Building a PR Complexity Score Bot with GitHub Webhooks

7th February 2025

This is the third post in my Automating GitHub Reviews series, where I build small tools that make pull request reviews easier to prioritize and understand.

If you missed the previous parts, check them out here:

Building an AI-powered GitHub PR Reviewer Bot – Automatically reviews PRs with OpenAI.
Automating PR Descriptions with OpenAI and Vercel – Generates clear, structured PR descriptions.

Today, we’re taking it a step further by introducing a PR Complexity Score bot.

Why Does PR Complexity Matter? 🎯

Not all pull requests are created equal. Some are small and easy to review, while others are massive, difficult to understand, and prone to bugs.

Code Complexity Affects:

Review Time – Large, complex PRs take longer to review and often introduce mistakes.
Merge Confidence – High-complexity PRs are riskier to merge.
Code Maintainability – Complex code is harder to refactor, debug, and extend in the future.

Our Solution 🔍

The PR Complexity Score Bot analyzes a few visible signals and assigns a score from 0–10, giving reviewers a quick indication of review effort before they open the diff. It is a heuristic, not a quality metric or a replacement for engineering judgment.

How the PR Complexity Score Works 🛠️

Our bot will:

✅ Listen for PR events (opened, ready_for_review).
✅ Fetch changed files, lines added/removed.
✅ Calculate Cyclomatic Complexity (measuring logical branching).
✅ Assign a complexity score (0-10) based on weighted factors.
✅ Post a comment on the PR with a structured breakdown.

🔢 What is Cyclomatic Complexity?

Cyclomatic Complexity measures the number of independent execution paths in the code. More branches (if/else, loops, switch cases) → harder to read, test, and maintain.

Example:

1function canAccessFeature(user) {
2  if (!user) {
3    return false;                  // Decision 1
4  }
5  if (user.role === "admin") {     // Decision 2
6    return true;
7  } else {
8    if (user.subscription) {       // Decision 3
9      if (user.subscription.active) {     // Decision 4
10        if (user.permissions.includes("feature_access")) { // Decision 5
11          return true;
12        }
13      }
14    }
15  }
16  return false;
17}

What’s Wrong?

Too many nested if statements (4 decision points → Cyclomatic Complexity 5)
Hard to scan & debug
Any small change increases the risk of breaking something

Let’s flatten the logic and use early returns to make the function more readable:

1function canAccessFeature(user) {
2  if (!user) return false;
3  if (user.role === "admin") return true;
4  if (!user.subscription?.active) return false;
5  return user.permissions.includes("feature_access");
6}

Why This is Better?

✅ Cyclomatic Complexity reduced from 5 → 2
✅ No deep nesting → Easier to read
✅ Less chance of errors → Easier to maintain

Let’s Build It! 🚀

1️⃣ Setting Up the Project

1mkdir github-pr-complexity-bot
2cd github-pr-complexity-bot
3npm init -y
4npm install @octokit/rest dotenv express

This installs:

**@octokit/rest **→ GitHub API client
dotenv → Manage API keys
**express **→ Handle webhook requests

2️⃣ Writing the Webhook Handler

Inside the project, create an api/ directory:

1mkdir api
2touch api/webhook.js

Add the following GitHub webhook handler to api/webhook.js:

1const express = require('express');
2const { Octokit } = require('@octokit/rest');
3const { computePRComplexityScore, calculateCyclomaticComplexity } = require('../utils/complexity');
4
5const router = express.Router();
6const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
7
8router.post('/webhook', async (req, res) => {
9  try {
10    const { pull_request, repository, action } = req.body;
11
12    if (!pull_request || !['opened', 'ready_for_review'].includes(action)) {
13      return res.status(400).send('Not a relevant PR event');
14    }
15
16    const owner = repository.owner.login;
17    const repo = repository.name;
18    const prNumber = pull_request.number;
19
20    // Fetch PR Files and Changes
21    const { data: files } = await octokit.pulls.listFiles({ owner, repo, pull_number: prNumber });
22    const totalFilesChanged = files.length;
23    const totalLinesAdded = files.reduce((sum, file) => sum + (file.additions || 0), 0);
24    const totalLinesRemoved = files.reduce((sum, file) => sum + (file.deletions || 0), 0);
25
26    // Compute Cyclomatic Complexity
27    const fileDiffs = files.map(f => f.patch || '').join('\n');
28    const cyclomaticComplexity = calculateCyclomaticComplexity(fileDiffs);
29
30    // Compute PR Complexity Score
31    const prComplexity = computePRComplexityScore({
32      filesChanged: totalFilesChanged,
33      linesAdded: totalLinesAdded,
34      linesRemoved: totalLinesRemoved,
35      cyclomaticComplexity
36    });
37
38    // Post a Comment on the PR
39    const complexityComment = `
40📊 **PR Complexity Score: ${prComplexity}/10**
41- 📝 **Files Changed:** ${totalFilesChanged}
42- ➕ **Lines Added:** ${totalLinesAdded}
43- ➖ **Lines Removed:** ${totalLinesRemoved}
44- 🔄 **Cyclomatic Complexity:** ${cyclomaticComplexity}
45
46🚀 **What This Means:**
47${prComplexity < 4 ? '🟢 Low complexity—should be easy to review ✅' : prComplexity < 7 ? '🟡 Moderate complexity—review carefully! 🧐' : '🔴 High complexity—consider breaking this into smaller PRs! ⚠️'}
48`;
49
50    await octokit.issues.createComment({ owner, repo, issue_number: prNumber, body: complexityComment });
51
52    res.status(200).send('PR Complexity Score added');
53  } catch (error) {
54    console.error(error);
55    res.status(500).send('Something went wrong');
56  }
57});
58
59module.exports = router;

3️⃣ Calculating the PR Complexity Score

Higher complexity → Higher score (0-10)

We assign weights to:

Files changed → 2 x weight
Lines added/removed → 1 x weight
Cyclomatic Complexity → 3 x weight

1function computePRComplexityScore({ filesChanged, linesAdded, linesRemoved, cyclomaticComplexity }) {
2  const weightFiles = 2;
3  const weightLines = 1;
4  const weightComplexity = 3;
5
6  const scoreFiles = Math.min((filesChanged / 10) * weightFiles, 10);
7  const scoreLines = Math.min(((linesAdded + linesRemoved) / 500) * weightLines, 10);
8  const scoreComplexity = Math.min((cyclomaticComplexity / 10) * weightComplexity, 10);
9
10  return Math.min((scoreFiles + scoreLines + scoreComplexity) / (weightFiles + weightLines + weightComplexity) * 10, 10).toFixed(1);
11}
12
13function calculateCyclomaticComplexity(code) {
14  const controlStructures = /(\bif\b|\belse if\b|\bfor\b|\bwhile\b|\bswitch\b|\bcase\b)/g;
15  return (code.match(controlStructures) || []).length + 1;
16}
17
18module.exports = { computePRComplexityScore, calculateCyclomaticComplexity };