A first pass on pull requests
For teams where code review is the bottleneck. The assistant reads the changes in a pull request, points out likely bugs, missing tests and broken rules (a committed password, a dependency you don’t allow), and leaves comments on the lines concerned. Your reviewers start from a sorted list instead of a blank page.
What you need
- Your GitHub repository and a GitHub token that may read pull requests and comment, and nothing more. The token is what really limits the assistant: with no permission to approve or merge, it can’t.
- Two AI providers from different companies, for the double check in step 4.
How to build it
- Connect GitHub. Open APIs, click Connect an API, pick SDK / npm and add
@octokit/rest. Then, under Tools, Create tool, SDK / npm package, make tools for “read a pull request’s changes” and “comment on a pull request”. Keep the token in Credentials. - Create the assistant. Open Agents, click Create agent and switch to Autonomous. In Instructions, write: “Review the changes for correctness, test coverage and our rules. Comment on the lines concerned. Never approve or merge.” Pick the GitHub tools under Tools and APIs.
- Write your rules as constraints. On its Constraints tab: “Flag any committed password or key”, “Flag new dependencies not on our list”, “Ask for tests for every new endpoint”.
- Cut the false alarms. On its Overview tab, click Configure verification, turn on Enable verification and add a reviewer from a different AI company. The second model checks the assistant’s final summary and sends it back for revision when a finding doesn’t hold up. Comments it already posted during the run are not held back, so for checked comments only, have it write the review as its final answer and post that (see For developers).
- Run it. On its Overview tab, turn on Schedule to go through open pull requests regularly.
Ideas
- Give it a tool that reads the surrounding code, so it judges more than the changed lines.
- Have several coding assistants fix what the review finds.
For developers
To review each pull request as soon as it opens, have a GitHub Action call the agent’s endpoint from the Integration card on its Overview tab, and post the answer it gets back as the review comment. The answer has passed verification by then. See the OpenAI-compatible API.