Building a GitHub PR Reviewer Bot with OpenAI and Vercel

Code reviews are one of those things that we all know are important, but let’s be honest - they can be tedious. Wouldn’t it be nice to have a bot that does an initial review for you? 😉

In this tutorial, I’ll show you how to build a simple PR reviewer with the OpenAI Responses API and deploy it to Vercel. It is a first-pass assistant, not an approval gate: a human reviewer remains responsible for validating every finding.

What This Bot Does?

Our bot will:

• Listen for new pull requests on GitHub via webhooks
• Fetch the PR’s diff
• Ask OpenAI for an analysis of the changes
• Post review comments automatically

Let’s get started! 🚀

Before we dive in, make sure you have:

• A GitHub account (obviously)
• A Vercel account (for easy deployment)
• An OpenAI API key (to do the actual review work)
• Node.js installed on your machine

If you don’t have these set up yet, take a moment to do so. I'll wait ⏰

Step 1: Setting Up the Project

First, create a new directory for our project and initialize it:

1$ mkdir github-pr-reviewer
2$ cd github-pr-reviewer
3$ npm init -y
4$ npm install @octokit/rest openai dotenv express

We’re installing:

• @octokit/rest to talk to GitHub
• openai to send requests to the configured OpenAI model
• dotenv to manage secrets
• express to handle incoming webhook requests

Step 2: Handling Webhooks

GitHub needs somewhere to send PR events, so we’ll create a simple webhook handler. Inside the project folder, create an `api` directory. This will contain our API function that will be later deployed to Vercel.

1$ mkdir api

Then, create api/webhook.js and add this code:

1const express = require('express');
2const { Octokit } = require('@octokit/rest');
3const OpenAI = require('openai');
4const { verifyWebhookSignature } = require('../utils/security'); // This will be created in the next step
5
6const router = express.Router();
7
8const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
9const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
10
11router.post('/webhook', async (req, res) => {
12  try {
13    verifyWebhookSignature(req);
14    const { pull_request, repository } = req.body;
15
16    if (!pull_request) return res.status(400).send('Not a PR event');
17
18    const owner = repository.owner.login;
19    const repo = repository.name;
20    const prNumber = pull_request.number;
21
22    // Fetch all PR files
23    const { data: files } = await octokit.pulls.listFiles({ owner, repo, pull_number: prNumber });
24    // Concatenate the diffs
25    const fileDiffs = files.map(f => `${f.filename}:\n${f.patch}`).join('\n');
26
27    // Send files to OpenAI for review
28    const aiResponse = await openai.responses.create({
29      model: process.env.OPENAI_MODEL || "gpt-5.6-sol",
30      instructions: "Review the diff for concrete correctness, security, and maintainability risks. Cite the relevant file when possible. Do not invent missing context.",
31      input: fileDiffs,
32      max_output_tokens: 700,
33    });
34
35    // Extract review from response
36    const reviewComment = aiResponse.output_text;
37
38    // Create a comment in Github
39    await octokit.issues.createComment({ owner, repo, issue_number: prNumber, body: reviewComment });
40
41    res.status(200).send('Review added');
42  } catch (error) {
43    console.error(error);
44    res.status(500).send('Something went wrong');
45  }
46});
47
48module.exports = router;

This script:

1. Verifies the webhook request.
2. Fetches the changed files in the PR.
3. Sends the diff to OpenAI for analysis.
4. Posts a comment with AI-generated feedback.

We’ll also need a simple helper to verify GitHub’s webhook signature. Create utils/security.js:

1const crypto = require('crypto');
2
3const verifyWebhookSignature = (req) => {
4  const signature = req.headers['x-hub-signature-256'];
5
6  if (!signature) throw new Error('No signature found');
7
8  const hmac = crypto.createHmac('sha256', process.env.WEBHOOK_SECRET);
9  const digest = 'sha256=' + hmac.update(JSON.stringify(req.body)).digest('hex');
10
11  if (signature !== digest) throw new Error('Invalid signature');
12};
13
14module.exports = { verifyWebhookSignature };

Step 3: Setting Up Secrets

Create a .env file in the project root:

1GITHUB_TOKEN = your_github_token
2OPENAI_API_KEY = your_openai_api_key
3WEBHOOK_SECRET = your_random_secret
4

Generate a random secret and paste it into the .env with:

1openssl rand -hex 20

This ensures that only valid GitHub webhooks are accepted.

Step 4: Deploying to Vercel

Now let’s deploy this thing! First, create a `vercel.json` file:

1{
2  "version": 2,
3    "functions": {
4    "api/*.js": {
5      "maxDuration": 60
6    }
7  },
8  "routes": [
9    {
10      "src": "/webhook",
11      "dest": "/api/webhook.js"
12    }
13  ]
14}

Then run:

1$ vercel login
2$ vercel

Add the environment variables on Vercel’s dashboard under Project Settings → Environment Variables.

Step 5: Configuring the GitHub Webhook

1. Go to your repo → Settings → Webhooks
2. Click Add webhook
3. Set the Payload URL to https://your-vercel-url/webhook (you will find it in the terminal after running vervel command)
4. Choose application/json as the content type
5. Enter your WEBHOOK_SECRET
6. Select **Pull request **events
7. Save it 💾

Testing It Out

1. Create a new branch
2. Make some changes
3. Open a pull request
4. Watch the bot post a comment!

What’s Next?

This is just a starting point. You could:

• Add inline comments for specific lines
• Implement configurable review rules
• Support different AI models

You can check my code here. It does more than in this tutorial including:

• posting inline comments instead of a single one per PR
• optimized OpenAI prompt that uses less tokens

Enjoy your new AI-powered code reviewer! 🤖