# lgtmaybe — full documentation
> AI code review for GitHub, GitLab, Gitea, and local Git diffs. It posts inline findings and a summary on pull and merge requests, or prints findings locally. Choose from seven hosted providers, ollama, or any OpenAI-compatible endpoint. Bedrock, Vertex, and Azure support keyless cloud auth.
Source: https://lgtmaybe.coles.codes/ — generated from the docs/ tree.
---
{ width="128" }
# lgtmaybe
lgtmaybe reviews pull and merge requests on **GitHub, GitLab, and Gitea**. You
can also review a local Git diff before pushing. Choose a hosted model, ollama,
or another OpenAI-compatible endpoint. Reviews on a code host post inline
comments and one summary; local reviews print findings in your terminal.
It reads the diff and a little surrounding code, but only comments on changed
lines. It never checks out or runs the change. Generated files and binaries are
skipped, secrets are redacted, and all author-supplied text is treated as
untrusted.
It checks for:
- **Correctness and security** — logic errors, missed `await`s, injection, auth mistakes, and leaked secrets.
- **Code health** — performance problems, needless complexity, deprecations, and risky dependencies.
- **Tests and documentation** — missing or weak tests, stale comments, and undocumented APIs.
- **Intent** — whether the change does what its title, description, and commits say it does.
- **Ponytail** — code you don't need, standard-library opportunities, and simpler ways to get the job done.
Findings are graded from `info` to `critical` and land on the exact changed
line. A clean change just gets a 👍 **LGTM!**.
![An inline lgtmaybe review comment flagging a [CRITICAL] SQL injection vulnerability, with an explanation and a committable parameterized-query suggestion](assets/marketplace/marketplace-screenshot-1.png){ width="720" }
Slash commands can refresh a review (`/review`, `/improve`), answer a question
(`/ask`), post a structured description (`/describe`), or post
[a change overview](how-to/generate-a-change-diagram.md) (`/diagram`). The
overview describes high-impact areas and includes a flowchart; it adds a
sequence diagram when the change alters a runtime flow. Run `lgtmaybe diagram`
to print a text version locally.
```mermaid
flowchart LR
web["Storefront
React
places orders"]
api["Order API
Python
creates orders (changed)"]
db["Order database
PostgreSQL
stores orders"]
queue["Order events
SQS
buffers events (new)"]
worker["Notification worker
Python
sends confirmations (new)"]
email["Email provider
delivers messages"]
web -->|calls| api
api -->|stores in| db
api -->|publishes to| queue
queue -->|delivers to| worker
worker -->|sends via| email
```
## Start here
- **Tutorial** — [Getting started](tutorial/getting-started.md): your first review with ollama, locally and free.
- **How-to** — task recipes: [choose a review model](how-to/choose-a-review-model.md), [run locally](how-to/run-locally-with-ollama.md), [Bedrock OIDC](how-to/review-with-bedrock-oidc.md), [Vertex WIF](how-to/review-with-vertex-wif.md), [Azure OpenAI](how-to/review-with-azure.md), [GitHub Action](how-to/use-as-github-action.md), [GitLab](how-to/review-on-gitlab.md), [Gitea](how-to/review-on-gitea.md).
- **Reference** — [Configuration](reference/config.md): every config field and schema.
- **Explanation** — [What gets reviewed](explanation/what-gets-reviewed.md), [Architecture](explanation/architecture.md), [Auth model](explanation/auth-model.md), [Data & privacy](explanation/data-and-privacy.md).
## Providers
| Provider | Auth |
|---|---|
| `openai` | `OPENAI_API_KEY` |
| `anthropic` | `ANTHROPIC_API_KEY` |
| `openrouter` | `OPENROUTER_API_KEY` |
| `zai` | `ZAI_API_KEY` — GLM / Zhipu AI; optional `--api-base` for the China / coding-plan endpoint |
| `bedrock` | Ambient AWS creds — GitHub OIDC, no static key |
| `vertex` | Ambient GCP creds — Workload Identity Federation, no key |
| `azure` | Ambient Azure AD creds — GitHub OIDC, no static key (or `AZURE_API_KEY`) + endpoint |
| `ollama` | None — local only, zero cost |
| `openai-compatible` | `--api-base` to any OpenAI `/v1` endpoint; key optional (placeholder for keyless local servers) |
## Where it posts
The model provider and the code host are independent — any provider above works
on any host below.
| Host | How it runs | Token | Guide |
|---|---|---|---|
| GitHub | GitHub Action | `GITHUB_TOKEN` | [GitHub Action](how-to/use-as-github-action.md) |
| GitLab | GitLab CI job | `GITLAB_TOKEN` | [Review on GitLab](how-to/review-on-gitlab.md) |
| Gitea | Gitea Actions | `GITEA_TOKEN` | [Review on Gitea](how-to/review-on-gitea.md) |
| None | `lgtmaybe review` locally | — | [Install the CLI](how-to/install-the-cli.md) |
The same review runs on all three hosts. Their APIs and workflows differ in
thread resolution, incremental reviews, and keyless cloud auth. See
[Architecture](explanation/architecture.md#code-hosts-forges) for the details.
## For AI agents
The site root publishes a curated [`llms.txt`](llms.txt) index of these docs,
plus a whole-corpus [`llms-full.txt`](llms-full.txt), for LLM crawlers and
coding agents.
---
# Getting Started with lgtmaybe
This tutorial walks you through your first review using **ollama** — a fully
local model that costs nothing and needs no API keys. By the end you will have
reviewed a branch and seen the findings in your terminal, with no GitHub token
and no pull request required.
## What you need
- Python 3.11 or later
- [ollama](https://ollama.com) running locally
- A local git repository with some changes on a branch to review
## Step 1 — Install lgtmaybe
```bash
pip install lgtmaybe
```
On macOS you can install from the Homebrew tap instead:
```bash
brew tap MattJColes/tap
brew trust MattJColes/tap # current Homebrew requires trusting third-party taps
brew install lgtmaybe
```
See [Install the CLI](../how-to/install-the-cli.md) for details. It explains
the `brew trust` step, and which providers the base install covers: API-key and
local ones. Keyless cloud providers need the `pip` extras.
Verify the install:
```bash
lgtmaybe --help
```
## Step 2 — Start ollama and pull a model
```bash
ollama serve # starts the local server on http://localhost:11434
ollama pull qwen3.6:27b # or any model you prefer
```
Leave `ollama serve` running in a separate terminal.
## Step 3 — Review your changes
From inside a git repo, on a branch with some changes, run:
```bash
lgtmaybe review \
--provider ollama \
--model qwen3.6:27b \
--api-base http://localhost:11434
```
lgtmaybe diffs your current branch against the remote primary branch
(`origin/HEAD`, falling back to `origin/main` / `origin/master`, then a local
`main` / `master`), sends the changed lines to your
local qwen3.6:27b instance, and prints the findings to your terminal:
```console
src/app.py:2 [MEDIUM] Import order
sys should be sorted before os
1 finding · provider ollama · model qwen3.6:27b
```
To review the whole worktree — your branch's commits plus uncommitted edits —
add `--working`. To review only the uncommitted edits, add `--uncommitted`. To
diff against a different base, pass `--base main`.
## Step 4 — Change the output format
`--format` controls what `review` prints. `--json` (shorthand for
`--format json`) emits a JSON array ready to pipe into other tooling:
```bash
lgtmaybe review --provider ollama --model qwen3.6:27b \
--api-base http://localhost:11434 --json
```
`--format agent` instead prints the findings as correction instructions an AI
coding agent can read and apply, for a local review-and-fix loop — see
[Fix findings with an AI agent](../how-to/fix-findings-with-an-ai-agent.md).
## Step 5 — See the whole change
`lgtmaybe diagram` runs the same local diff through three concurrent model calls
and prints the **change overview**: a description of what you changed, a **High
Impact Areas** section calling out anything that could bite (infrastructure,
security posture, outage risk, migrations, backups and more), and a picture of
the components your change touches — plus, when it alters a run-time flow, a
sequence diagram of that flow:
```bash
lgtmaybe diagram \
--provider ollama \
--model qwen3.6:27b \
--api-base http://localhost:11434
```
It takes the same `--base` / `--working` / `--uncommitted` flags as `review`, so
`review` then `diagram` is a natural pair before you open a pull request: what's
wrong with the change, then what the change is and what it could break. The
diagrams print as Mermaid source plus a text rendering — the text is what reads
in a terminal; paste the Mermaid into a GitHub comment or
[mermaid.live](https://mermaid.live) to see it drawn. See
[Generate a change overview](../how-to/generate-a-change-diagram.md).
## Step 6 — Post reviews on real pull requests
The CLI reviews local changes. To run lgtmaybe on actual pull requests — inline
comments and a summary posted back to the change — wire it into your code host.
The review is the same on all three; only the plumbing differs:
- **GitHub** — add the Action to your repo:
[Use as a GitHub Action](../how-to/use-as-github-action.md)
- **GitLab** — add a CI job:
[Review on GitLab](../how-to/review-on-gitlab.md)
- **Gitea** — add a Gitea Actions workflow:
[Review on Gitea](../how-to/review-on-gitea.md)
## What happened under the hood
lgtmaybe ran its pipeline over your local diff:
1. **fetch** — read the diff from your local repo with `git diff`
2. **compress** — stripped generated files, binaries, and lockfiles
3. **prompt** — built a structured prompt asking for JSON output
4. **parse** — validated the model's JSON against the `ReviewFinding` schema
5. **gate** — dropped defect findings without a concrete `failure_scenario`
6. **render** — printed the findings (the Action posts them to the PR instead)
## Next steps
- To configure severity thresholds, path filters, and token caps, see
[Configure .lgtmaybe.yml](../how-to/configure-lgtmaybe-yml.md).
- To use a cloud provider with no API keys, see
[Review with Bedrock OIDC](../how-to/review-with-bedrock-oidc.md) or
[Review with Vertex WIF](../how-to/review-with-vertex-wif.md).
- To use lgtmaybe in a GitHub Actions workflow, see
[Use as a GitHub Action](../how-to/use-as-github-action.md).
---
# Install the CLI
Install the `lgtmaybe` command-line tool once; then point it at any provider.
This page is only about **getting the CLI onto your machine**. Choosing a model
backend — a hosted API, keyless cloud, or a local model — comes after, in the
[provider how-tos](../index.md#providers).
## Install (pip)
Works on any OS with Python 3.11+:
```bash
pip install lgtmaybe
```
Verify it:
```bash
lgtmaybe --help
```
That prints the command list with usage examples; ` --help`
(e.g. `lgtmaybe review --help`) shows the full option reference for one command.
To record which build you are running — for a bug report, or to pin the
executable a benchmark ran against — ask it:
```bash
lgtmaybe --version
```
It prints one parseable line, `lgtmaybe `, and works the same for every
install (pip, Homebrew, or the portable Windows executable). A source checkout
that was never installed has no version metadata to read and says
`lgtmaybe unknown`.
Upgrade later with `pip install --upgrade lgtmaybe`.
## Install on Windows (WinGet)
The WinGet package is a portable executable for Windows x86_64 (64-bit).
Install it by its exact package identifier:
```powershell
winget install --id MattJColes.lgtmaybe --exact
```
Verify the command alias:
```powershell
lgtmaybe --help
```
Upgrade or uninstall it later with the same package identifier:
```powershell
winget upgrade --id MattJColes.lgtmaybe --exact
winget uninstall --id MattJColes.lgtmaybe --exact
```
If WinGet cannot find a newly published version, refresh its package sources
and retry:
```powershell
winget source update
```
The executable bundles ast-grep for cross-file symbol resolution, but it does
not bundle the optional cloud SDKs used by keyless Bedrock, Vertex, or Azure
authentication. Use the pip install with the matching extra for those providers.
The regular `pip install lgtmaybe` path also works on Windows.
## Install on macOS (Homebrew)
On macOS (and Linuxbrew) you can install from the project's
[Homebrew tap](https://github.com/MattJColes/homebrew-tap) instead of `pip`:
```bash
brew tap MattJColes/tap
brew trust MattJColes/tap
brew install lgtmaybe
```
The middle step is required: current Homebrew refuses to load a formula from a
third-party tap until you explicitly **trust** it (`Error: Refusing to load
formula … from untrusted tap`). This is a one-time, per-machine decision that
applies to every tap outside Homebrew's core — the tap author can't waive it for
you, by design. To trust only this one formula instead of the whole tap, use
`brew trust --formula MattJColes/tap/lgtmaybe`.
Tapping and installing in one line (`brew install MattJColes/tap/lgtmaybe`)
works too, but still needs the `brew trust …` step first — otherwise the install
stops at the untrusted-tap error.
Upgrade later with `brew upgrade lgtmaybe`.
If you tapped the earlier `MattJColes/lgtmaybe` name, switch over once with
`brew untap MattJColes/lgtmaybe && brew tap MattJColes/tap` (then `brew trust
MattJColes/tap`). Upgrades keep working on the old tap either way — GitHub
redirects the renamed repo — so this is tidying, not a break.
Homebrew installs lgtmaybe into its own isolated virtualenv, so it never touches
your system or project Python. The formula creates the venv and installs
lgtmaybe and its dependencies from prebuilt **PyPI wheels** (no compiling), so a
first install takes about a minute, mostly download time. It works on any
architecture and macOS version — there's no prebuilt bottle to match. It also
pulls the `ast-grep` formula, which lgtmaybe uses for cross-file symbol
resolution during review.
## Cloud providers need extras
The base pip install, Homebrew formula, and Windows executable cover every
**API-key** and **local** provider:
| Providers | Covered by base install? |
|---|---|
| `openai`, `anthropic`, `openrouter`, `zai` — set the provider's API key in your environment | ✅ Yes |
| `ollama`, `openai-compatible` — fully local, point `--api-base` at the server | ✅ Yes |
| `bedrock`, `vertex`, `azure` — keyless cloud (OIDC/WIF) | ❌ Needs an extra |
The **keyless cloud** providers need extra cloud SDKs that the Homebrew formula
and Windows executable do not bundle. Install the CLI from PyPI with the matching
extra:
```bash
pip install 'lgtmaybe[bedrock]' # or [vertex] / [azure]
```
or run them through the [GitHub Action](use-as-github-action.md), where the image
bundles all three and wires up the OIDC/WIF auth for you.
## Next steps
- Run your first local review: [Getting started](../tutorial/getting-started.md).
- Diagram what a change touches: [Generate a change diagram](generate-a-change-diagram.md).
- Post reviews on real pull requests: [Use as a GitHub Action](use-as-github-action.md).
---
# Choose a Review Model
**Start with `google/gemini-3.8-flash` on OpenRouter.** On the 24 September
2026 benchmark using lgtmaybe 2.8.5, it led the 11 tested models on both
small, varied changes (60.9%) and large Python diffs (71.4%). Its breadth
result is fully adjudicated. It still flagged findings on seven of nine clean
changes, so check its output before acting on it.
```bash
lgtmaybe review --provider openrouter --model google/gemini-3.8-flash
```
If large diffs matter more than small changes, `openai/gpt-6-luna` was second
on long horizon (64.6%) and completed those five cases in 440 seconds of
aggregate review time, against Gemini's 798. Its breadth recall was much
lower (42.9% against 62.9%), so it is not the default for everyday review.
These are recommendations from a synthetic corpus, not a guarantee for your
repository. Try the candidates on recent pull requests before choosing a team
default. See [Review with OpenRouter](review-with-openrouter.md) for setup.
## What the Suites Measure
- **Breadth:** 32 small changes across seven programming languages, GitHub
Actions, and Terraform, with 72 planted findings and nine verified-clean
changes. Each model reviews the suite three times; the table reports the
median result.
- **Long horizon:** Four increasingly large Python changes with eight planted
bugs each, plus one large clean change. Each model reviews the suite once.
The suites use different cases. **Compare scores within a suite, never across
suites.** Score is closed-world F0.5, which weights precision more than recall,
scaled by the share of review calls that returned parseable findings
(completeness). An unmatched finding counts as a false positive even if it
might be useful on real code. A provisional breadth score has a small number
of findings awaiting human adjudication.
## Cloud Models Tested on lgtmaybe 2.8.5
### Small changes: breadth
| Model | Score | Recall | Precision | False positives | Clean pass |
|---|---:|---:|---:|---:|---:|
| `google/gemini-3.8-flash` | **60.9%** | 62.9% | 79.3% | 12 | 22.2% |
| `x-ai/grok-4.7` † | 55.7% | 62.9% | 70.1% | 20 | 11.1% |
| `z-ai/glm-5.3-flash` † | 55.1% | 62.9% | 70.0% | 21 | 11.1% |
| `moonshotai/kimi-k3` † | 52.0% | 65.7% | 63.3% | 27 | 0.0% |
| `openai/gpt-6-sol` | 50.0% | 60.0% | 62.9% | 26 | 11.1% |
| `openai/gpt-6-luna` † | 50.0% | 42.9% | 70.5% | 13 | 22.2% |
| `z-ai/glm-5.3` † | 49.9% | 64.3% | 67.6% | 22 | 11.1% |
| `xiaomi/mimo-v2.6-flash` † | 46.2% | 62.9% | 61.3% | 29 | 0.0% |
| `qwen/qwen3.8-flash` † | 45.4% | 62.9% | 59.5% | 32 | 11.1% |
| `openai/gpt-6-astra` | 27.7% | 44.3% | 66.7% | 16 | 44.4% |
| `anthropic/claude-opus-5.5` | 6.0% | 1.4% | 100.0% | 0 | 100.0% |
† Provisional: some borderline findings remain unadjudicated. Clean pass is
the share of nine clean changes on which the model returned no findings. Opus's
perfect precision and clean pass came from finding almost none of the planted
bugs. Astra's score was limited in part by 45.3% completeness.
### Large diffs: long horizon
| Model | Score | Recall | Precision | False positives |
|---|---:|---:|---:|---:|
| `google/gemini-3.8-flash` | **71.4%** | 87.5% | 75.7% | 9 |
| `openai/gpt-6-luna` | 64.6% | 84.4% | 69.2% | 12 |
| `openai/gpt-6-sol` | 30.6% | 78.1% | 29.8% | 59 |
| `moonshotai/kimi-k3` | 30.4% | 100.0% | 37.6% | 53 |
| `qwen/qwen3.8-flash` | 28.3% | 78.1% | 58.1% | 18 |
| `openai/gpt-6-astra` | 25.5% | 56.2% | 40.0% | 27 |
| `z-ai/glm-5.3-flash` | 24.3% | 59.4% | 48.7% | 20 |
| `x-ai/grok-4.7` | 21.9% | 65.6% | 36.2% | 37 |
| `xiaomi/mimo-v2.6-flash` | 21.5% | 71.9% | 44.2% | 29 |
| `z-ai/glm-5.3` | 8.1% | 81.2% | 12.4% | 183 |
| `anthropic/claude-opus-5.5` | 0.0% | 0.0% | 100.0% | 0 |
Kimi found all 32 planted bugs but added 53 false positives. The low scores
for Qwen Flash and Xiaomi also reflect completeness below 50%: many review
calls produced no parseable findings. The [benchmark results][results] show
per-case recall, token use, truncation, and wall time.
## Local Models
The published local runs use older lgtmaybe versions and have not been repeated
on 2.8.5. `nvidia/Qwen3.6-35B-A3B-NVFP4` remains the strongest local option
with canonical runs on both suites in that historical set: 48.2% breadth on
2.2.0 and 43.8% long horizon on 2.1.4. Treat those as a starting point for a
local trial, not a current comparison with the cloud table above.
```bash
lgtmaybe review \
--provider openai-compatible \
--model nvidia/Qwen3.6-35B-A3B-NVFP4 \
--api-base http://127.0.0.1:8000/v1
```
See [Run locally with Ollama](run-locally-with-ollama.md) for hardware guidance
and [Other OpenAI-compatible servers](use-a-custom-openai-compatible-endpoint.md)
for vLLM, llama.cpp, and LM Studio setup.
## Source and Limits
The [benchmark repository][bench] holds the live leaderboard, [complete
results][results], reproducible commands, and append-only raw results. Older
runs remain there for historical comparison; their model versions, lgtmaybe
versions, and sometimes review profiles differ. This page uses the 22
canonical runs from 24 September 2026 for its cloud recommendations.
The corpus is synthetic and does not measure provider price, availability,
data handling, or performance on your codebase. No run in this cohort has an
immutable audit trace; the live leaderboard reports audit availability.
[bench]: https://github.com/MattJColes/lgtmaybe-benchmarks
[results]: https://github.com/MattJColes/lgtmaybe-benchmarks/blob/main/RESULTS.md
---
# Use lgtmaybe as a GitHub Action
Use this guide to add lgtmaybe to a repository as a GitHub Actions workflow
that reviews pull requests automatically.
!!! tip "Not on GitHub?"
lgtmaybe runs the same review on GitLab and Gitea — see
[Review on GitLab](./review-on-gitlab.md) or
[Review on Gitea](./review-on-gitea.md).
Use lgtmaybe from the
[GitHub Marketplace listing](https://github.com/marketplace/actions/lgtmaybe).
It is a **GitHub Action**: the reviewer runs in your workflow, and its provider,
model, and provider authentication settings live in the step's `with:` block.
GitHub Actions supplies the repository token by default, so reviews post as
`github-actions[bot]`. You can optionally install the public lgtmaybe App and
set `github_identity: lgtmaybe` to post as `lgtmaybe[bot]`; the App changes only
the GitHub author identity. The
[minimal OpenAI workflow](#minimal-workflow-openai) below shows the complete
default shape.
Ready-to-copy workflows for every cloud and API-key provider live in
[`examples/workflows/`](https://github.com/MattJColes/lgtmaybe/tree/main/examples/workflows).
`auto_diagram` is on by default, so opened and reopened pull requests receive a
change overview automatically — a description, a **High Impact Areas** section,
and a compact Mermaid flowchart. Later `synchronize` pushes refresh it and, when
the change alters a run-time flow, a sequence diagram appears beside it. Set it
to `false` for no overview at all, or drop a single section with
`auto_describe: false` / `high_impact: false`. When the PR description or
any comment lgtmaybe didn't post already carries a Mermaid diagram, the
automatic overview is skipped and the review summary says so; `/diagram` still
posts one on request.
ollama runs the model on your own machine, so it is local-only — use the
[CLI](run-locally-with-ollama.md) rather than a posting workflow.
## Contents
- [Security requirement: pull_request_target](#security-requirement-pull_request_target)
- [Who can trigger a review](#who-can-trigger-a-review)
- [Minimal workflow — openai](#minimal-workflow-openai)
- [Choose the GitHub author](#choose-the-github-author)
- [Other key-based providers](#other-key-based-providers)
- [Keyless cloud workflows](#keyless-cloud-workflows)
- [Action inputs](#action-inputs)
- [Adding a config file](#adding-a-config-file)
- [Pin to a specific version](#pin-to-a-specific-version)
## Security requirement: pull_request_target
All lgtmaybe workflows use the `pull_request_target` trigger, not
`pull_request`. This is non-negotiable:
- `pull_request_target` runs in the context of the **base branch**, so it can
access secrets and write to the PR.
- lgtmaybe **never checks out or executes PR code** — it fetches the diff via
the GitHub API only. The PR author cannot inject code that runs in the
reviewer's environment.
The action derives the PR from the triggering event, so there is no `pr-url`
input to set. On an `issue_comment` event it routes the slash command
(`/review`, `/ask`, `/describe`, `/diagram`, `/improve`) to the same engine. On a
`synchronize` push the review is **incremental** by default: only the commits
added since the last completed review are re-reviewed, and earlier findings
stay open until fixed. Comment `/review full` for a full re-review on demand,
or pin the behaviour with the `incremental` input / config key.
Replies inside finding threads do not trigger lgtmaybe. A comment is not
evidence that the finding is fixed: push the fix and the next incremental review
will verify the changed code, reply `✅ Looks resolved.`, and resolve the outdated
thread when the finding has disappeared. Use `/ask` when you deliberately want
the model to answer a question.
> **Upgrading:** remove `answer_replies` from `.lgtmaybe.yml` and Action inputs,
> and remove the `pull_request_review_comment` trigger from custom workflows.
> Configuration is strict, so leaving the removed option in place is an error.
> **Note on cost.** With ollama the model runs on your own hardware, so reviews
> are free. On a hosted provider each run uses tokens you pay for, so it's worth
> a moment's thought about who can trigger one (next section) — the default keeps
> that to people you trust, and `max_files` / `max_input_tokens` keep any single
> run modest.
## Who can trigger a review
You choose who reviews run for. The example workflows gate the `review` job on
the triggering user's
[author association](https://docs.github.com/en/graphql/reference/enums#commentauthorassociation)
and default to **trusted contributors** — `OWNER`, `MEMBER`, and `COLLABORATOR`.
A maintainer can also review an outside contributor's PR any time by commenting
`/review` on it (their own association passes the gate).
The same `if:` also requires a comment to actually carry a slash command before
any job starts. `issue_comment` fires on every comment on every pull request, so
without that clause an ordinary "lgtm, merging" would claim a runner, pull the
container and boot Python only to find no command and exit — no tokens spent, but
a job queued ahead of the reviews that do have work to do. The check is
substring-based and case-insensitive, so `/REVIEW` and `/review full` both pass.
To change the policy, edit the `if:` on the `review` job:
- **Everyone** — drop the author-association checks so any PR or `/ask` /
`/review` comment runs a review (keep the slash-command clause, or every
comment starts a job). A friendly choice for an open project — just remember
that on a hosted provider it means anyone can start a paid run, so pick it
deliberately.
- **Returning contributors too** — add `CONTRIBUTOR` to auto-review anyone whose
PR has merged before.
- **Admins only** — keep just `OWNER` (plus `MEMBER` for your org).
For extra guardrails, you can also require approval for fork-PR workflow runs in
**Settings → Actions → General → Fork pull request workflows**, or move the
provider key behind a protected `environment`. See
[Trust and Cost](../explanation/trust-and-cost.md) for the reasoning behind these
options.
## Minimal workflow — openai
```yaml
name: lgtmaybe
on:
pull_request_target:
issue_comment:
types: [created]
permissions:
contents: read
pull-requests: write
jobs:
review:
# Only trusted authors (owner / member / collaborator) can trigger a review,
# and a comment only starts a job when it carries a slash command.
if: >-
(github.event_name == 'pull_request_target' &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) ||
(github.event.issue.pull_request &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
(contains(github.event.comment.body, '/review') ||
contains(github.event.comment.body, '/improve') ||
contains(github.event.comment.body, '/ask') ||
contains(github.event.comment.body, '/describe') ||
contains(github.event.comment.body, '/diagram')))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7 # base repo only — for .lgtmaybe.yml config
- uses: MattJColes/lgtmaybe@v2
with:
provider: openai
model: gpt-5.5
auto_diagram: true
api_key: ${{ secrets.OPENAI_API_KEY }}
```
## Choose the GitHub author
The minimal workflow uses GitHub's built-in workflow token and posts as
`github-actions[bot]`. This path needs no App installation and remains the
default.
To post as `lgtmaybe[bot]`, install the public lgtmaybe App on selected
repositories, add `id-token: write` to the workflow, and set
`github_identity: lgtmaybe`. The provider, model, and provider key stay exactly
where they are. Follow [Post as lgtmaybe[bot]](./post-as-a-github-app.md) for
the complete workflow, permission boundary, and uninstall instructions.
If your organisation operates its own App, the same guide documents the
advanced `app_id` / `app_private_key` inputs.
## Other key-based providers
Swap the `provider`, `model`, and `api_key` inputs:
```yaml
# anthropic
- uses: MattJColes/lgtmaybe@v2
with:
provider: anthropic
model: claude-sonnet-4-6
api_key: ${{ secrets.ANTHROPIC_API_KEY }}
# openrouter
- uses: MattJColes/lgtmaybe@v2
with:
provider: openrouter
model: anthropic/claude-sonnet-4-6
api_key: ${{ secrets.OPENROUTER_API_KEY }}
# zai (GLM / Zhipu AI)
- uses: MattJColes/lgtmaybe@v2
with:
provider: zai
model: glm-4.6
api_key: ${{ secrets.ZAI_API_KEY }}
```
For these, the one-time setup is just: generate an API key in the provider's
console and add it as a repo secret (Settings → Secrets and variables → Actions),
then reference it as `api_key` above.
## Keyless cloud workflows
Bedrock (AWS OIDC), Vertex (GCP WIF), and Azure (Entra OIDC) need **no API keys
in secrets** — the action performs the keyless token exchange for you when you
pass `aws_role_arn`, `gcp_wif_provider`, or `azure_client_id`. All require
`id-token: write` permission. See:
- [Review with Bedrock OIDC](./review-with-bedrock-oidc.md)
- [Review with Vertex WIF](./review-with-vertex-wif.md)
- [Review with Azure OpenAI](./review-with-azure.md)
## Action inputs
| Input | Default | Description |
|---|---|---|
| `provider` | — | One of: `openai`, `openrouter`, `anthropic`, `zai`, `bedrock`, `vertex`, `azure`, `ollama`, `openai-compatible` |
| `model` | — | Model identifier for the chosen provider |
| `fallback_model` | — | Model to re-run a call on when `model` cannot answer it. For a truncation it is the *last* remedy, tried only after a smaller payload, a lower `reasoning_effort` and reasoning switched off have been tried on `model` itself; for any other failure the adapter switches straight to it. Also settable as `fallback_model` in `.lgtmaybe.yml`, which this input overrides. See [Configure fallback_model](configure-lgtmaybe-yml.md#fallback_model) for provider scope, cost, and failure behavior |
| `api_key` | — | API key for key-based providers (leave empty for bedrock/vertex/ollama and keyless azure) |
| `api_base` | — | Resource endpoint for azure (`https://.openai.azure.com`), or a custom base URL for other providers |
| `timeout` | provider default (ollama/openai-compatible/openrouter 1800s, cloud 600s) | Enforced wall-clock timeout for each model call. Transient failures (capacity 429s, connection blips, 5xx) are retried with exponential backoff; permanent ones (bad key, quota/billing 429, unknown model) and a call that blows this whole timeout fail fast — re-sending the identical request against the identical budget can only burn it twice |
| `temperature` | `0.0` | Sampling temperature (0.0 = deterministic) |
| `num_ctx` | `32768` | Ollama context window (ollama only; ignored for hosted providers) |
| `max_input_tokens` | `100000` | Token budget per model call before the diff is split into batches (any provider) |
| `resolve_fixed` | `true` | Auto-resolve a review conversation once its finding is fixed (set `false` to resolve manually) |
| `recursive` | `true` | Walk a file whose diff exceeds `max_input_tokens` hunk-by-hunk (RLM-style) instead of sending it whole; set `false` to disable |
| `structured_output` | `true` | Constrain output to the findings JSON schema via `response_format` (JSON mode); set `false` for an `openai-compatible` gateway that rejects it |
| `preset` | `fast` | `fast` uses four calls — security, correctness, code health, artefacts — on every provider; `full` runs one call per lens |
| `triage_model` | — | Cheap model that runs first to skip plainly-non-substantive files and rank the rest by risk; security-relevant files always escalate past triage. Unset = no triage |
| `reflect_model` | defaults to `model` | Model for the self-reflection (false-positive audit) pass — point it at a stronger model to audit a weaker reviewer's findings |
| `max_review_seconds` | `3600` | Soft wall-clock ceiling for the whole review; once passed, queued calls are skipped and partial results post with a notice (a cancelled or timed-out job does the same, via SIGINT/SIGTERM). `0` disables |
| `max_concurrency` | auto (6, every provider) | Concurrent review calls across the whole fan-out. Set `1` for a serial local run |
| `symbol_resolution` | `true` | During reflection, resolve a deferred finding's symbol via ast-grep in a read-only shallow clone of the base branch, so cross-file findings are re-judged against the real definition |
| `incremental` | auto | Commit-scoped incremental review on `synchronize` pushes (full review elsewhere); `true`/`false` forces it. `/review full` forces a full re-review on demand |
| `static_analysis` | `false` | Run deterministic tools (ruff, bandit, mypy, gitleaks, zizmor, ast-grep, osv-scanner, semgrep) sandboxed over the changed files: linters ground the model as untrusted hints, while gitleaks, zizmor, ast-grep and osv-scanner post directly with no model call. The image bundles these tools and an offline vulnerability database |
| `auto_describe` | `true` | Head the change overview with a structured description (title, change type, summary, per-file walkthrough, intent check); set `false` to drop the section |
| `auto_diagram` | `true` | After each opened, reopened, or synchronized (push) PR review, post or refresh the change overview: description, High Impact Areas, a Mermaid flowchart and, when the change alters a flow, a sequence diagram; set `false` to opt out of the whole comment |
| `high_impact` | `true` | Include the High Impact Areas section in the change overview — infrastructure, security posture, outage risk, data migrations, backups and recovery, compatibility, observability, dependencies, cost, compliance; set `false` to drop the section |
| `risk` | `true` | Assess the PR's risk of change (low/medium/high/critical) from importer counts, sensitive paths, size and untested code — deterministic, no model call; shown on the summary line, in the overview, as a hidden `lgtmaybe-risk` marker, and (with `pr_labels`) a `risk/` label. Set `false` to turn it off |
| `pr_labels` | `false` | Attach derived labels: `review-effort/1-5`, `possible-security-issue`, `consider-splitting`, `risk/` (best-effort, no extra model calls) |
| `fail_on` | — (off) | Merge-gate threshold (`info`/`low`/`medium`/`high`/`critical`). Creates a `lgtmaybe` Check Run that **fails** when any finding is at or above this severity — make it a required check to block merges. See [Gate merges on findings](#gate-merges-on-findings) |
| `profile` | `false` | Print a timing profile (per-stage and per-call tables, token and cache usage) in the Action log |
| `aws_role_arn` | — | IAM role ARN to assume via OIDC for bedrock (keyless) |
| `aws_region` | `us-east-1` | AWS region for bedrock |
| `gcp_wif_provider` | — | Workload Identity Federation provider resource name for vertex |
| `gcp_service_account` | — | GCP service account email to impersonate via WIF |
| `azure_client_id` | — | Entra (Azure AD) client ID with a federated credential — keyless azure via OIDC |
| `azure_tenant_id` | — | Entra (Azure AD) tenant ID for keyless azure |
| `config_path` | `.lgtmaybe.yml` | Path to the config file, relative to repo root |
| `github_token` | `${{ github.token }}` | Token for reading the PR and posting the review |
| `github_identity` | `actions` | GitHub author identity: `actions` posts as `github-actions[bot]`; `lgtmaybe` posts as `lgtmaybe[bot]` after the public App is installed and `id-token: write` is granted |
| `identity_broker_url` | managed endpoint | Advanced override for the public App identity exchange |
| `app_id` | — | Advanced: ID of your own GitHub App used with `app_private_key`; do not combine with `github_identity: lgtmaybe` |
| `app_private_key` | — | Advanced: private key of your own App named by `app_id` |
| `app_owner` | — | Owner for a cross-repo App token (defaults to the current repo's owner) |
| `app_repositories` | — | Repositories the App token may access, newline/comma-separated (defaults to the current repo); use with `app_owner` |
| `image` | `ghcr.io/mattjcoles/lgtmaybe:v2` | Override the container image (advanced) |
The action sets the `GITHUB_TOKEN` and provider credentials for the container
itself — you do not pass them as `env`.
## Gate merges on findings
Set `fail_on` to a severity to turn the review into a merge gate. After posting
the review, lgtmaybe creates a **Check Run** named `lgtmaybe` whose conclusion is
`failure` when any surviving finding is at or above that severity, and `success`
otherwise. Enforcement rides the Check Run — lgtmaybe never sets PR approval
state, so a clean review stays comment-only.
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: openai
model: gpt-4o
api_key: ${{ secrets.OPENAI_API_KEY }}
fail_on: high # block merge on any high/critical finding
```
To make it block merges, mark the check as required in **branch protection**:
1. Open **Settings → Branches → Branch protection rules** (or a ruleset) for the
target branch.
2. Enable **Require status checks to pass before merging**.
3. Search for and add the **`lgtmaybe`** check. It appears in the list once the
Action has run at least once with `fail_on` set on a PR against that branch.
A PR with a finding at or above the threshold then shows a failing `lgtmaybe`
check and cannot merge until the finding is resolved (or `fail_on` is lowered).
Leave `fail_on` unset to keep reviews advisory (the default) — no check run is
created.
Creating the Check Run requires `checks: write`. For the default Actions
identity, add it to the workflow `permissions:` block. The public
`lgtmaybe[bot]` App intentionally does not hold that permission, so
`github_identity: lgtmaybe` cannot be combined with `fail_on`; use the Actions
identity or a self-managed App granted `Checks: write`.
### What the public App cannot do
The public `lgtmaybe[bot]` App holds a deliberately minimal permission set, so
two features are unavailable under `github_identity: lgtmaybe`:
| Feature | Behaviour under the public App |
|---|---|
| `fail_on` (merge-gate Check Run) | Refused up front with a setup error — needs `checks: write` |
| `resolve_fixed` (auto-resolve fixed conversations) | `resolveReviewThread` is refused, so threads stay open. The review is unaffected: nothing is posted, the refusal is logged once with what to do, and the rest of the run continues |
Both work under the default `actions` identity (with `pull-requests: write` in
the workflow `permissions:` block) or a self-managed App holding the
corresponding permission. Set `resolve_fixed: false` if you would rather not
attempt it at all.
## Adding a config file
Place a `.lgtmaybe.yml` at the repo root to control severity thresholds, path
filters, and cost caps. See
[Configure .lgtmaybe.yml](./configure-lgtmaybe-yml.md) for all options.
## Pin to a specific version
`@v2` is a floating tag that tracks the latest `v2.x.x` release. To pin exactly,
use a full version tag:
```yaml
uses: MattJColes/lgtmaybe@v2.0.0
```
## On another code host
The review is identical on GitLab and Gitea; only the wiring and a few
host-capability differences change:
- [Review on GitLab](./review-on-gitlab.md) — a CI job running
`lgtmaybe gitlab-ci` on merge request pipelines.
- [Review on Gitea](./review-on-gitea.md) — Gitea Actions, running the same
container as GitHub.
Keyless cloud auth (Bedrock, Vertex, Azure) relies on GitHub Actions' OIDC
token, so it is available here only. On the other hosts use an API-key provider
or `ollama`.
---
# Post as lgtmaybe[bot]
lgtmaybe is a GitHub Action. The Action runs the reviewer and keeps the
provider, model, and provider authentication in the workflow's `with:` block.
A GitHub App is optional: it changes the author of GitHub comments from
`github-actions[bot]` to a branded bot.
There are three identity choices:
- Do nothing: the Action uses GitHub's workflow token and posts as
`github-actions[bot]`.
- Install the public lgtmaybe App: the Action posts as `lgtmaybe[bot]`; you do
not receive or manage an App private key.
- Use your own GitHub App: provide its App ID and private key as an advanced,
self-managed setup.
## Install the public lgtmaybe App
1. [Install the lgtmaybe App](https://github.com/apps/lgtmaybe/installations/new)
and select only the repositories where it should review pull requests.
2. Add `id-token: write` to the workflow permissions.
3. Set `github_identity: lgtmaybe` on the Action step.
Here is a complete OpenAI workflow:
```yaml
name: lgtmaybe
on:
pull_request_target:
issue_comment:
types: [created]
permissions:
contents: read
id-token: write
jobs:
review:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7 # base repository only
- uses: MattJColes/lgtmaybe@v2
with:
github_identity: lgtmaybe
provider: openai
model: gpt-5.5
api_key: ${{ secrets.OPENAI_API_KEY }}
```
The provider configuration is unchanged. `github_identity` selects who posts
on GitHub; it does not select or host the model.
The public App has only:
- `contents: read`, to fetch repository and pull-request content
- `pull requests: write`, to post reviews, comments, and labels
- `issues: write`, to answer pull-request issue comments
The Action exchanges GitHub's signed workflow identity for a short-lived App
token restricted to the triggering repository and those exact permissions.
The identity broker verifies the repository, event, and default-branch
workflow. It never receives your diff, provider key, prompt, model response, or
review findings.
Selecting `github_identity: lgtmaybe` is explicit: if `id-token: write` is
missing, the App is not installed on the repository, the workflow is not
trusted, or the identity service is unavailable, the job fails with setup
guidance. A misconfiguration is never papered over with `github-actions[bot]`.
Reviews, `/review`, `/ask`, `/describe`, and `/diagram` arrive on
`pull_request_target` or `issue_comment`, which run from the default branch, so
they post as `lgtmaybe[bot]`. Replies inside review threads do not start a model
run; a pushed fix is verified by the next incremental review.
To remove access, uninstall the App from the repository or remove that
repository from the installation. Remove `github_identity: lgtmaybe` and
`id-token: write` from the workflow to return to `github-actions[bot]`. The
Action also revokes its temporary App token after each run; GitHub expires it
automatically if cleanup cannot run.
The public App intentionally has no `Checks: write` permission, so
`github_identity: lgtmaybe` cannot be combined with `fail_on`. Keep the
default `github_identity: actions` and grant the workflow `checks: write`, or
use a self-managed App with `Checks: write`, when lgtmaybe must create a merge
gate.
## Use your own GitHub App
Use this path only when your organisation already operates a GitHub App and
wants reviews attributed to it.
Configure the App with repository permissions `contents: read`, `pull
requests: write`, and `issues: write`. Add `checks: write` only when using
`fail_on`. Then install it on the repositories it may review. Store its ID as
the `LGTMAYBE_APP_ID` repository variable and its PEM private key as the
`LGTMAYBE_APP_PRIVATE_KEY` Actions secret:
```yaml
permissions:
contents: read
steps:
- uses: actions/checkout@v7
- uses: MattJColes/lgtmaybe@v2
with:
provider: openai
model: gpt-5.5
api_key: ${{ secrets.OPENAI_API_KEY }}
app_id: ${{ vars.LGTMAYBE_APP_ID }}
app_private_key: ${{ secrets.LGTMAYBE_APP_PRIVATE_KEY }}
```
Do not combine `app_id` / `app_private_key` with
`github_identity: lgtmaybe`. The Action validates this before minting a token.
---
# Review pull requests on Gitea
lgtmaybe posts reviews to Gitea as well as GitHub. The review itself is
identical — same lenses, same reflection pass, same findings — because none of
that ever depended on the host. Only the adapter that fetches the diff and posts
the comments is different.
## Set it up
1. **Create a token.** In Gitea, go to *Settings → Applications → Access Tokens*
and create a token with `write:repository` and `write:issue` scopes. Add it to
your repository as a secret named `GITEA_TOKEN`.
Both are needed, and `write:repository` covers reading too. Gitea scopes the
review endpoints lgtmaybe posts inline comments through (`POST
/pulls/{index}/reviews`) and the commit-status endpoint under the
*repository* category, not the *issue* one — a token with only
`read:repository` is refused on the one call the whole setup exists to make.
`write:issue` covers the summary comment and the PR labels.
2. **Add a provider key.** Add the API key for whichever model you want as a
second secret — for example `ANTHROPIC_API_KEY`.
3. **Add the workflow.** Copy
[`examples/gitea/lgtmaybe.yml`](https://github.com/MattJColes/lgtmaybe/blob/main/examples/gitea/lgtmaybe.yml)
into your repository at `.gitea/workflows/lgtmaybe.yml`.
That is the whole setup. Gitea Actions reimplements the GitHub Actions runtime,
so lgtmaybe's container image runs unchanged, and it reads `GITHUB_SERVER_URL`
— which Gitea points at your own instance — to know which host to post back to.
## Try it locally first
Reviewing a pull request by URL happens in the Actions run — that is where the
token and the PR context live. Locally, `lgtmaybe review` reviews the branch you
have checked out, against the remote primary branch, and prints the findings
instead of posting them:
```bash
git switch my-feature-branch
export ANTHROPIC_API_KEY=...
lgtmaybe review --provider anthropic --model claude-sonnet-4-6
```
Same lenses, same reflection pass, same findings — so it is the quick way to see
what a model will say about your change before you wire up the workflow.
## What works, and what does not
Everything about the *review* works: every lens, the self-reflection pass,
secret redaction, prompt-injection hardening, static-analysis fusion, path
filters, finding rules, and the `/review`, `/improve`, `/ask`, `/describe` and
`/diagram` slash commands.
Three things are unavailable on Gitea, and lgtmaybe skips them rather than
failing:
| Not available | Why | What happens instead |
|---|---|---|
| Incremental review | Gitea's compare endpoint returns commit metadata, not a unified diff | Every run is a full review |
| Resolve-on-fix | Gitea has no thread-resolution API | Fixed findings' comments stay open |
| Keyless cloud auth | The OIDC/WIF exchange is a GitHub Actions feature | Use an API key, or `ollama` for a local model |
Because a submitted Gitea review cannot be edited, lgtmaybe reads the hidden
finding ids already on the pull request and declines to post a finding twice.
The summary lives in an ordinary comment, which *is* editable, so it updates in
place on each run.
## Fork safety
Gitea has no `pull_request_target`, so the `pull_request` event runs with the
repository's secrets. The safety rule is unchanged and holds for the same
reason: lgtmaybe never checks out or executes pull-request code. It reads the
diff through the API and treats every byte of it as untrusted input.
---
# Review merge requests on GitLab
lgtmaybe posts reviews to GitLab as well as GitHub. The review itself is
identical — same lenses, same reflection pass, same findings — because none of
that ever depended on the host. Only the adapter that fetches the diff and posts
the comments is different.
## Set it up
1. **Create a token.** In your project, go to *Settings → Access Tokens* and
create a project access token with the `api` scope and at least the Developer
role. Add it under *Settings → CI/CD → Variables* as `GITLAB_TOKEN`, masked.
2. **Add a provider key.** Add the API key for whichever model you want as a
second masked variable — for example `ANTHROPIC_API_KEY`.
3. **Add the job.** Copy
[`examples/gitlab/.gitlab-ci.yml`](https://github.com/MattJColes/lgtmaybe/blob/main/examples/gitlab/.gitlab-ci.yml)
into your repository, or merge the `lgtmaybe` job into your existing
pipeline.
The job must run on merge request pipelines — `rules: - if: $CI_PIPELINE_SOURCE
== "merge_request_event"`. On a branch pipeline there is no merge request to
review, and lgtmaybe will tell you so by name rather than guessing.
## Try it locally first
Reviewing a merge request by URL happens in CI — that is where the token and the
MR context live. Locally, `lgtmaybe review` reviews the branch you have checked
out, against the remote primary branch, and prints the findings instead of
posting them:
```bash
git switch my-feature-branch
export ANTHROPIC_API_KEY=...
lgtmaybe review --provider anthropic --model claude-sonnet-4-6
```
Same lenses, same reflection pass, same findings — so it is the quick way to see
what a model will say about your change before you wire up the pipeline.
## How it differs from GitHub
GitLab has no batched review object, so **each finding is posted as its own
discussion**, anchored by the merge request's diff refs and the old or new file
line. The summary lives in a note, which updates in place on each run.
Resolve-on-fix **does** work here: GitLab discussions resolve over plain REST,
where GitHub needs GraphQL. A finding is only resolved once the follow-up
validation pass has confirmed it is actually fixed — never merely because it
stopped being reported.
One thing is not available yet:
| Not available | Why | What happens instead |
|---|---|---|
| Incremental review | Not yet built — GitLab's compare endpoint returns per-file diffs that could be reassembled, so this is a "not yet", not a "cannot" | Every run is a full review |
| Keyless cloud auth | The OIDC/WIF exchange is a GitHub Actions feature | Use an API key, or `ollama` |
## Fork safety
lgtmaybe never checks out or executes merge request code. It reads the diff
through the API and treats every byte of it as untrusted input — which is why
the example job sets `GIT_STRATEGY: none` and does not clone the repository at
all.
---
# Review with OpenAI
OpenAI is a key-based provider: add an `OPENAI_API_KEY` and pick a model. No
OIDC, no endpoint to configure — the simplest path to a hosted review.
## Contents
- [Get an API key](#get-an-api-key)
- [GitHub Action](#github-action)
- [Run locally](#run-locally)
- [Choosing the model](#choosing-the-model)
- [Persist non-secret defaults](#persist-non-secret-defaults)
## Get an API key
Create a key at . In your repository, add
it as an Actions secret named `OPENAI_API_KEY`
(**Settings → Secrets and variables → Actions → New repository secret**).
## GitHub Action
Copy [`examples/workflows/review-openai.yml`][wf] to
`.github/workflows/lgtmaybe.yml`. The core step is:
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: openai
model: gpt-5.5
api_key: ${{ secrets.OPENAI_API_KEY }}
```
That review runs on `pull_request_target`, so the secret is available while PR
code is **never** checked out. lgtmaybe only reads the diff via the API. See
[Use as a GitHub Action](use-as-github-action.md) for the full workflow,
including [who can trigger a review](use-as-github-action.md#who-can-trigger-a-review).
## Run locally
```bash
export OPENAI_API_KEY=sk-...
lgtmaybe review --provider openai --model gpt-5.5
```
You can pass the key inline with `--api-key sk-...` instead of the env var. The
key is read from the environment or the flag and is **never persisted** to
config.
## Choosing the model
Use any chat model your key can access: `gpt-5.5` for the strongest reviews, or
a smaller model to cut cost. Pass litellm's native OpenAI name — the same string
you'd send to the API.
## Persist non-secret defaults
Provider and model are non-secret, so they can live in `.lgtmaybe.yml` (the key
stays in the environment):
```yaml
provider: openai
model: gpt-5.5
```
With that file in place, `lgtmaybe review` needs no flags. See
[Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md) for every knob.
[wf]: https://github.com/MattJColes/lgtmaybe/blob/main/examples/workflows/review-openai.yml
---
# Review with Claude (Anthropic)
Anthropic is a key-based provider: add an `ANTHROPIC_API_KEY` and pick a Claude
model. No OIDC, no endpoint to configure.
## Contents
- [Get an API key](#get-an-api-key)
- [GitHub Action](#github-action)
- [Run locally](#run-locally)
- [Choosing the model](#choosing-the-model)
- [Want it through a gateway instead?](#want-it-through-a-gateway-instead)
- [Persist non-secret defaults](#persist-non-secret-defaults)
## Get an API key
Create a key in the [Anthropic Console](https://console.anthropic.com/). In your
repository, add it as an Actions secret named `ANTHROPIC_API_KEY`
(**Settings → Secrets and variables → Actions → New repository secret**).
## GitHub Action
Copy [`examples/workflows/review-anthropic.yml`][wf] to
`.github/workflows/lgtmaybe.yml`. The core step is:
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: anthropic
model: claude-sonnet-4-6
api_key: ${{ secrets.ANTHROPIC_API_KEY }}
```
That review runs on `pull_request_target`, so the secret is available while PR
code is **never** checked out. lgtmaybe only reads the diff via the API. See
[Use as a GitHub Action](use-as-github-action.md) for the full workflow,
including [who can trigger a review](use-as-github-action.md#who-can-trigger-a-review).
## Run locally
```bash
export ANTHROPIC_API_KEY=sk-ant-...
lgtmaybe review --provider anthropic --model claude-sonnet-4-6
```
You can pass the key inline with `--api-key sk-ant-...` instead of the env var.
The key is read from the environment or the flag and is **never persisted** to
config.
## Choosing the model
Pass litellm's native Anthropic name as the model. `claude-sonnet-4-6` is a
strong default. Reach for an Opus-class model when you want the deepest review
and are happy to pay more, or a Haiku-class model to cut cost on smaller PRs.
## Want it through a gateway instead?
If you route Claude through OpenRouter or a Bedrock/Vertex deployment rather than
the Anthropic API directly, use that provider instead:
- [OpenRouter](review-with-openrouter.md) — `anthropic/claude-sonnet-4-6`
- [Bedrock (keyless OIDC)](review-with-bedrock-oidc.md)
- [Vertex (keyless WIF)](review-with-vertex-wif.md)
## Persist non-secret defaults
```yaml
provider: anthropic
model: claude-sonnet-4-6
```
With that file in place, `lgtmaybe review` needs no flags. See
[Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md) for every knob.
[wf]: https://github.com/MattJColes/lgtmaybe/blob/main/examples/workflows/review-anthropic.yml
---
# Review with OpenRouter
[OpenRouter](https://openrouter.ai/) is a key-based gateway to many model
vendors behind one API. Add an `OPENROUTER_API_KEY`, then pick any model
OpenRouter offers using its `vendor/model` name.
## Contents
- [Get an API key](#get-an-api-key)
- [GitHub Action](#github-action)
- [Run locally](#run-locally)
- [Choosing the model](#choosing-the-model)
- [Credit reservations](#credit-reservations)
- [Rate limits](#rate-limits)
- [Persist non-secret defaults](#persist-non-secret-defaults)
## Get an API key
Create a key at . In your repository, add it as an
Actions secret named `OPENROUTER_API_KEY`
(**Settings → Secrets and variables → Actions → New repository secret**).
## GitHub Action
Copy [`examples/workflows/review-openrouter.yml`][wf] to
`.github/workflows/lgtmaybe.yml`. The core step is:
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: openrouter
model: anthropic/claude-sonnet-4-6
api_key: ${{ secrets.OPENROUTER_API_KEY }}
```
That review runs on `pull_request_target`, so the secret is available while PR
code is **never** checked out. lgtmaybe only reads the diff via the API. See
[Use as a GitHub Action](use-as-github-action.md) for the full workflow,
including [who can trigger a review](use-as-github-action.md#who-can-trigger-a-review).
## Run locally
```bash
export OPENROUTER_API_KEY=sk-or-...
lgtmaybe review --provider openrouter --model anthropic/claude-sonnet-4-6
```
You can pass the key inline with `--api-key sk-or-...` instead of the env var.
The key is read from the environment or the flag and is **never persisted** to
config.
## Choosing the model
OpenRouter models are named `vendor/model`. Pick whichever fits your budget and
quality bar, for example:
- `anthropic/claude-sonnet-4-6`
- `openai/gpt-5.5`
- `z-ai/glm-4.6`
Browse the full catalogue and per-model pricing at
.
One catalogue quirk: a model can accept JSON-schema mode through OpenRouter yet
return replies that parse as empty, so a review comes back with zero findings
and no error (seen with `anthropic/claude-fable-5.1`). If a model is silent on
everything, try one diff with `--no-structured-output` — see
[Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md#structured_output) — before
writing the model off.
## Credit reservations
OpenRouter checks your balance **before** it generates anything, costing the
worst case: the prompt plus the most tokens the reply could use. A request that
sends no cap is assumed to want the model's full output ceiling, so a review can
be refused for credit it was never going to spend:
```
This request requires more credits, or fewer max_tokens.
You requested up to 65536 tokens, but can only afford 25905.
```
OpenRouter reviews send a **default cap of 16384 tokens per call**, so the
reservation already matches a realistic findings payload rather than the model's
full ceiling. If a refusal still names a number larger than that, something has
overridden the default — check the `per-call budget resolved` log line, which
names the ceiling and where it came from:
```
per-call budget resolved timeout_s=1800 max_tokens=16384 max_tokens_source="provider default"
```
Top up, or lower the cap further so the reservation fits your balance:
```yaml
max_tokens: 8192
```
`--max-tokens 8192` does the same for one run, and `max_tokens` is a GitHub
Action input too. Set it lower with care: a cap set too low truncates the
findings JSON mid-object, and reasoning models spend this same budget on thinking
tokens, so they need more headroom than a plain model. `max_tokens: 0` removes
the cap entirely and puts the reservation back to the model's full ceiling.
lgtmaybe treats this refusal as permanent and stops after one attempt — your
balance cannot grow mid-review, so retrying every lens would only waste runner
time before reporting the same failure.
## Rate limits
A *capacity* 429 is different: it is temporary, and lgtmaybe handles it for you.
Rate-limited calls back off on a 5s–60s ladder — long enough to reach a fresh
per-minute window — and honour OpenRouter's own `Retry-After` when it sends one
(clamped at 120s). A lens still failing on the provider when the fan-out drains
is re-run once more. That rescue is only for provider-side failures: unparseable
output, a blown `max_tokens` ceiling, a batch the oversized-diff split already
retried, an unrecoverable failure like a spent quota, and any ceiling you set
(`max_review_seconds`, `max_review_tokens`, a cancelled job) are all left alone,
because a second attempt would buy the same answer at full price. You will
usually see none of this; a review that could not recover says so in its
summary, and names the lens it lost.
If you *are* seeing rate limits, the review's own burst is worth a look before
the model is. Every `(batch, lens)` call shares one concurrency pool and one API
key, so a wide fan-out can meter itself:
```yaml
max_concurrency: 3
```
Beyond that it is an account question rather than a lgtmaybe one. OpenRouter's
limits on **free** model variants (the `:free` suffix) are account-wide rather
than per-model, and the daily allowance depends on how much credit the account
has bought — so the same review is far more likely to be throttled on a free
variant than a paid one. **Paid** models carry no OpenRouter platform request
cap, though the upstream provider behind a given model can still rate-limit you.
Neither is the same thing as a per-key credit *spending* cap, which limits what a
key may spend, not how often it may call.
## Persist non-secret defaults
```yaml
provider: openrouter
model: anthropic/claude-sonnet-4-6
```
With that file in place, `lgtmaybe review` needs no flags. See
[Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md) for every knob.
[wf]: https://github.com/MattJColes/lgtmaybe/blob/main/examples/workflows/review-openrouter.yml
---
# Review with z.ai (GLM)
[z.ai](https://z.ai/)'s GLM models (GLM-4.6 and friends, from Zhipu AI) are a
first-class provider: add a `ZAI_API_KEY` and pick a model. lgtmaybe reaches them
through litellm's native `zai/` route — no `api_base` needed for the
international endpoint.
## Contents
- [Get an API key](#get-an-api-key)
- [GitHub Action](#github-action)
- [Run locally](#run-locally)
- [Choosing the model](#choosing-the-model)
- [China / coding-plan endpoint](#china-coding-plan-endpoint)
- [Persist non-secret defaults](#persist-non-secret-defaults)
## Get an API key
Create a key in the [z.ai developer console](https://z.ai/) (see the
[API docs](https://docs.z.ai/)). In your repository, add it as an Actions secret
named `ZAI_API_KEY`
(**Settings → Secrets and variables → Actions → New repository secret**).
## GitHub Action
Copy [`examples/workflows/review-zai.yml`][wf] to
`.github/workflows/lgtmaybe.yml`. The core step is:
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: zai
model: glm-4.6
api_key: ${{ secrets.ZAI_API_KEY }}
```
That review runs on `pull_request_target`, so the secret is available while PR
code is **never** checked out. lgtmaybe only reads the diff via the API. See
[Use as a GitHub Action](use-as-github-action.md) for the full workflow,
including [who can trigger a review](use-as-github-action.md#who-can-trigger-a-review).
## Run locally
```bash
export ZAI_API_KEY=...
lgtmaybe review --provider zai --model glm-4.6
```
You can pass the key inline with `--api-key ...` instead of the env var. The key
is read from the environment or the flag and is **never persisted** to config.
## Choosing the model
Pass any GLM chat model your key can access — e.g. `glm-4.6`, `glm-4.7`,
`glm-4.5-air`, or a newer `glm-5.x`. Use the model name as z.ai's API expects it
(litellm prefixes the `zai/` route for you).
## China / coding-plan endpoint
The native route targets z.ai's **international** endpoint. To use the China or
coding-plan endpoint instead, point `--api-base` (or `ZAI_API_BASE`, or the
Action's `api_base` input) at it:
```bash
lgtmaybe review \
--provider zai \
--model glm-4.6 \
--api-base https://open.bigmodel.cn/api/paas/v4
```
## Persist non-secret defaults
```yaml
provider: zai
model: glm-4.6
```
With that file in place, `lgtmaybe review` needs no flags. See
[Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md) for every knob.
[wf]: https://github.com/MattJColes/lgtmaybe/blob/main/examples/workflows/review-zai.yml
---
# Review with AWS Bedrock (keyless OIDC)
Use this guide to run lgtmaybe with **AWS Bedrock** using GitHub's OIDC token
— no static AWS credentials stored in secrets.
## Contents
- [How it works](#how-it-works)
- [One-time AWS setup](#one-time-aws-setup)
- [Workflow example](#workflow-example)
- [Choosing a Bedrock model ID](#choosing-a-bedrock-model-id)
- [Running locally with ambient AWS credentials](#running-locally-with-ambient-aws-credentials)
- [Troubleshooting](#troubleshooting)
## How it works
GitHub Actions issues a short-lived OIDC token. AWS STS exchanges that token
for temporary IAM credentials scoped to a role you control. The action performs
that exchange for you (pass `aws_role_arn`) and lgtmaybe picks up the ambient
credentials automatically — no `AWS_ACCESS_KEY_ID` or `AWS_SECRET_ACCESS_KEY`
in your secrets.
## One-time AWS setup
This is the human-only part — do it once in your AWS account:
1. Create an IAM **OIDC identity provider** for `token.actions.githubusercontent.com`.
2. Create an IAM **role** with a trust policy scoped to your repo
(`repo:/lgtmaybe:*`).
3. Attach the least-privilege policy below.
4. Confirm the models you want are enabled in the target region (model access
request in the Bedrock console).
5. Note the **role ARN** (e.g. `arn:aws:iam::123456789012:role/lgtmaybe-bedrock`)
— it becomes the `aws_role_arn` action input. No static key is ever stored.
The role needs only:
```json
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": [
"arn:aws:bedrock:*::inference-profile/*",
"arn:aws:bedrock:*::foundation-model/*"
]
}
```
**Both ARNs are required for the recommended inference-profile model ids**
(the `us.`/`eu.`/`apac.`-prefixed ids below). A cross-region inference profile
fans the call out to the foundation model in several regions, so the role needs
`bedrock:InvokeModel*` on **both** the `inference-profile/*` ARN **and** the
underlying `foundation-model/*` ARN. Granting only one of them still fails with
`AccessDeniedException`. A bare `anthropic.…` model id needs only the
`foundation-model/*` ARN, but most current Claude models are invocable only
through an inference profile — see the model table below.
Scope each `Resource` to specific model / inference-profile ARNs for tighter
least-privilege once it works, e.g.
`arn:aws:bedrock:*::inference-profile/us.anthropic.claude-opus-4-8*`
and `arn:aws:bedrock:*::foundation-model/anthropic.claude-opus-4-8*`.
## Workflow example
The action assumes the role for you — no separate `configure-aws-credentials`
step needed. Store the role ARN in an `AWS_ROLE_ARN` secret.
```yaml
name: lgtmaybe
on:
pull_request_target:
issue_comment:
types: [created]
permissions:
id-token: write # required for the OIDC token exchange (keyless)
pull-requests: write # required to post review comments
contents: read
jobs:
review:
# Only trusted authors (repo owner / org member / collaborator) can trigger
# a review, so fork PRs and drive-by comments cannot spend your provider
# budget. A maintainer can still opt in to an external PR with /review.
if: >-
(github.event_name == 'pull_request_target' &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) ||
(github.event.issue.pull_request &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
(contains(github.event.comment.body, '/review') ||
contains(github.event.comment.body, '/improve') ||
contains(github.event.comment.body, '/ask') ||
contains(github.event.comment.body, '/describe') ||
contains(github.event.comment.body, '/diagram')))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MattJColes/lgtmaybe@v2
with:
provider: bedrock
model: us.anthropic.claude-sonnet-4-6
aws_role_arn: ${{ secrets.AWS_ROLE_ARN }}
aws_region: us-east-1
```
## Choosing a Bedrock model ID
The `model` must be a **Bedrock** model identifier. Bedrock hosts Anthropic
Claude, Amazon Nova/Titan, Meta Llama, Mistral, Cohere and others — it does
**not** host OpenAI's GPT models, so an id like `openai.gpt-5.5` is rejected with
`The provided model identifier is invalid`. Use one of the Claude ids below
(prefixed with a cross-region inference profile — see the note):
| Model | Inference-profile ID (recommended) | Base model ID |
|---|---|---|
| Claude Opus 4.8 | `us.anthropic.claude-opus-4-8` | `anthropic.claude-opus-4-8` |
| Claude Sonnet 4.6 | `us.anthropic.claude-sonnet-4-6` | `anthropic.claude-sonnet-4-6` |
| Claude Haiku 4.5 | `us.anthropic.claude-haiku-4-5` | `anthropic.claude-haiku-4-5` |
**Prefer the inference-profile form.** Most current Claude models on Bedrock are
invocable only through a cross-region inference profile, not via on-demand
throughput on the bare model id — so a bare id often fails with the same
`invalid model identifier` (or `on-demand throughput isn't supported`) error.
Prefix with your geography: `us.` (US), `eu.` (Europe) or `apac.` (Asia
Pacific), matching `aws_region`. Use the bare `anthropic.…` id only where
on-demand access to that model is enabled in your region.
## Running locally with ambient AWS credentials
If your local shell has AWS credentials (via `~/.aws`, SSO, or an assumed role),
you can review your current branch's changes with Bedrock. Bedrock signing needs
`boto3`, so install the extra (the Action image already bundles it):
```bash
pip install 'lgtmaybe[bedrock]'
lgtmaybe review \
--provider bedrock \
--model us.anthropic.claude-sonnet-4-6
```
lgtmaybe does not require or accept a static API key for Bedrock.
> **`No matching distribution found for lgtmaybe[bedrock]`** (`from versions:
> none`) — this is not a packaging problem; the `bedrock` extra is published on
> PyPI. It means `pip` found no version compatible with your environment. The
> usual cause is an interpreter older than the required **Python 3.11+** (check
> with `python --version`); install under 3.11+ (e.g. via `pipx`/`uv`). If your
> Python is fine, your `pip` can't reach PyPI — check your network/proxy or
> `--index-url`.
## Troubleshooting
**`ExpiredTokenException`** — the OIDC exchange failed or the role session
expired. Check that `id-token: write` permission is present in the workflow and
that the IAM trust policy references the correct GitHub repository.
**`AccessDeniedException`** — the role lacks `bedrock:InvokeModel` for the
selected model, or the model is not enabled in the Bedrock console for your
account and region. For an inference-profile id (`us.`/`eu.`/`apac.`-prefixed),
the policy must also allow the `inference-profile/*` ARN, not just
`foundation-model/*`.
**`output_config.format: Extra inputs are not permitted`**, **`outputConfig`**,
or a `toolConfig` / "doesn't support tool choice" refusal — the model's Converse
route doesn't accept the structured-output shape lgtmaybe asks for, and Bedrock
rejects the whole request with a 400. No action is needed: enforcement degrades
one rung at a time rather than being switched off. A rejected `response_format`
is re-sent as the *same* schema through the mechanism Converse does implement —
a forced tool call, whose arguments are the findings — and the rest of the lens
fan-out uses that shape straight away. Only a route that refuses the tool too
falls back to prompt-instructed JSON (the prompt asks for JSON anyway and the
parser is lenient); that fallback is logged as a warning naming the model and
the reason. If you're on a build from before this (the symptom is every lens
failing at once, and the review reporting `every review call failed`), upgrade,
or pass `--no-structured-output` / set `structured_output: false` as a stopgap.
**Every review comes back unstructured after one odd reply** — an empty reply,
or one that parsed to something that wasn't findings, is evidence against the
mechanism in play, not against enforcement. It now steps the model down to the
forced tool call and keeps the schema; only a second such reply gives it up. On
a build from before this, one blip on one lens cost enforcement for every later
call of the run. The provider log names what happened:
`structured output stepped down to a forced tool call` is the middle rung and
costs you nothing, while `structured output disabled for this model` is the
floor and carries a `reason` (`empty-response`, `unparseable-output`,
`tool-rejected`, `internal-tool-rejected`) worth reading.
**`The provided model identifier is invalid`** — the `model` is not a Bedrock
model id. Two common causes: (1) it's a non-Bedrock id such as `openai.gpt-5.5`
or another provider's name — Bedrock hosts Claude / Nova / Llama / Mistral /
Cohere, not OpenAI GPT, so pick a Claude id from the table above; (2) the model
needs a cross-region inference profile — prefix with `us.`/`eu.`/`apac.` (e.g.
`us.anthropic.claude-sonnet-4-6`) rather than the bare `anthropic.…` id.
---
# Review with Google Vertex AI (keyless WIF)
Use this guide to run lgtmaybe with **Google Vertex AI** using Workload Identity
Federation — no service-account JSON or static keys in secrets.
## Contents
- [How it works](#how-it-works)
- [One-time GCP setup](#one-time-gcp-setup)
- [Workflow example](#workflow-example)
- [Environment variables](#environment-variables)
- [Available Vertex AI models](#available-vertex-ai-models)
- [Running locally with ADC](#running-locally-with-adc)
- [Troubleshooting](#troubleshooting)
## How it works
GitHub Actions issues an OIDC token. GCP's Workload Identity Federation
exchanges that token for a short-lived GCP access token, impersonating a service
account that has only the Vertex AI permissions it needs. lgtmaybe uses
Application Default Credentials (ADC) to pick up those tokens automatically.
## One-time GCP setup
This is the human-only part — do it once in your GCP project:
1. Enable the Vertex AI API on the project.
2. Create a **workload identity pool** + a GitHub provider in it.
3. Create a **service account** with `roles/aiplatform.user` (or narrower — that
role grants `aiplatform.endpoints.predict`; do not assign broader project-level
roles).
4. Grant the GitHub principal permission to impersonate that service account,
scoped to your repo.
5. Note the **WIF provider resource name** (→ `gcp_wif_provider`) and the
**service account email** (→ `gcp_service_account`). No key file is ever stored.
## Workflow example
The action authenticates to GCP for you — no separate
`google-github-actions/auth` step needed. Store the provider resource name and
service account email in `GCP_WIF_PROVIDER` and `GCP_SERVICE_ACCOUNT` secrets.
```yaml
name: lgtmaybe
on:
pull_request_target:
issue_comment:
types: [created]
permissions:
id-token: write # required for the WIF token exchange (keyless)
pull-requests: write # required to post review comments
contents: read
jobs:
review:
# Only trusted authors (repo owner / org member / collaborator) can trigger
# a review, so fork PRs and drive-by comments cannot spend your provider
# budget. A maintainer can still opt in to an external PR with /review.
if: >-
(github.event_name == 'pull_request_target' &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) ||
(github.event.issue.pull_request &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
(contains(github.event.comment.body, '/review') ||
contains(github.event.comment.body, '/improve') ||
contains(github.event.comment.body, '/ask') ||
contains(github.event.comment.body, '/describe') ||
contains(github.event.comment.body, '/diagram')))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MattJColes/lgtmaybe@v2
with:
provider: vertex
model: gemini-3-pro
gcp_wif_provider: ${{ secrets.GCP_WIF_PROVIDER }}
gcp_service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
```
## Environment variables
| Variable | Required | Description |
|---|---|---|
| `VERTEXAI_PROJECT` | Yes | GCP project ID |
| `VERTEXAI_LOCATION` | No | Region (default: `us-central1`) |
## Available Vertex AI models
| Model | Vertex model ID |
|---|---|
| Gemini 3 Pro | `gemini-3-pro` |
| Gemini 3.1 Pro | `gemini-3.1-pro` |
| Gemini 3 Flash | `gemini-3-flash` |
| Gemini 3.5 Flash | `gemini-3.5-flash` |
| Gemini 2.5 Pro | `gemini-2.5-pro` |
## Running locally with ADC
Sign in with `gcloud auth application-default login` so your shell has
Application Default Credentials. Vertex token minting needs `google-auth`, so
install the extra (the Action image already bundles it):
```bash
pip install 'lgtmaybe[vertex]'
export VERTEXAI_PROJECT=my-project
export VERTEXAI_LOCATION=us-central1
lgtmaybe review \
--provider vertex \
--model gemini-3-pro
```
This reviews your current branch's changes with Vertex; lgtmaybe does not accept
a static API key for it.
## Troubleshooting
**`UNAUTHENTICATED`** — ADC credentials are missing or expired. Run
`gcloud auth application-default login` locally, or verify the WIF provider and
service account impersonation binding in CI.
**`PERMISSION_DENIED`** — the service account lacks `roles/aiplatform.user`, or
the Vertex AI API is not enabled in the project. Enable it with:
```bash
gcloud services enable aiplatform.googleapis.com --project=my-project
```
**`Model not found`** — the model ID may not be available in your selected
region. Check the [Vertex AI model garden](https://console.cloud.google.com/vertex-ai/model-garden).
---
# Review with Azure OpenAI (keyless OIDC)
Use this guide to run lgtmaybe against **Azure OpenAI** using GitHub's OIDC token
federated to Entra (Azure AD) — no static Azure key stored in secrets. A
key-based fallback is covered at the end.
## Contents
- [How it works](#how-it-works)
- [One-time Azure setup](#one-time-azure-setup)
- [Workflow example](#workflow-example)
- [Choosing the model name](#choosing-the-model-name)
- [Running locally with ambient Azure credentials](#running-locally-with-ambient-azure-credentials)
- [Key-based alternative](#key-based-alternative)
- [Troubleshooting](#troubleshooting)
## How it works
GitHub Actions issues a short-lived OIDC token. Entra (Azure AD) exchanges it
for an access token scoped to your Azure OpenAI resource, via a **federated
credential** on an app registration or managed identity. The action does that
exchange for you (pass `azure_client_id` / `azure_tenant_id`), and lgtmaybe
picks up the ambient token through `azure-identity`'s `DefaultAzureCredential`.
No `AZURE_API_KEY` in your secrets.
Azure still needs two non-secret values: the **deployment name** (`model`) and
the **resource endpoint** (`api_base`, `https://.openai.azure.com`).
## One-time Azure setup
This is the human-only part — do it once in your Azure tenant:
1. Register an **Entra app** (or use a user-assigned **managed identity**). Note
its **client ID** (`azure_client_id`) and your **tenant ID**
(`azure_tenant_id`).
2. Add a **federated credential** to it for GitHub Actions:
- Issuer: `https://token.actions.githubusercontent.com`
- Subject: scope it to your repo, e.g.
`repo:/:pull_request` (or `:ref:refs/heads/main`, `:environment:…`)
- Audience: `api://AzureADTokenExchange`
3. On the **Azure OpenAI resource**, grant the app the
**Cognitive Services OpenAI User** role (least privilege — it can call models,
not manage the resource).
4. **Deploy** the model you want and note the **deployment name** — that is what
you pass as `model`, not the upstream OpenAI model id.
5. Note the resource **endpoint** (`https://.openai.azure.com`) — it
becomes `api_base`. No static key is ever stored.
## Workflow example
The action performs the OIDC exchange for you — no separate `azure/login` step
needed. `id-token: write` is required so GitHub will mint the OIDC token.
```yaml
name: lgtmaybe
on:
pull_request_target:
issue_comment:
types: [created]
permissions:
id-token: write # required for the OIDC token exchange (keyless)
pull-requests: write # required to post review comments
contents: read
jobs:
review:
# Only trusted authors (repo owner / org member / collaborator) can trigger
# a review, so fork PRs and drive-by comments cannot spend your provider
# budget. A maintainer can still opt in to an external PR with /review.
if: >-
(github.event_name == 'pull_request_target' &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) ||
(github.event.issue.pull_request &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
(contains(github.event.comment.body, '/review') ||
contains(github.event.comment.body, '/improve') ||
contains(github.event.comment.body, '/ask') ||
contains(github.event.comment.body, '/describe') ||
contains(github.event.comment.body, '/diagram')))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MattJColes/lgtmaybe@v2
with:
provider: azure
model: my-gpt-4o-deployment # your deployment name
api_base: ${{ secrets.AZURE_API_BASE }} # https://.openai.azure.com
azure_client_id: ${{ secrets.AZURE_CLIENT_ID }}
azure_tenant_id: ${{ secrets.AZURE_TENANT_ID }}
```
## Choosing the model name
For Azure, `model` is the **deployment name** you chose in the portal, not the
underlying OpenAI model id. A deployment of `gpt-4o` named `my-gpt-4o-deployment`
is referenced as `model: my-gpt-4o-deployment`. litellm routes it as
`azure/`.
## Running locally with ambient Azure credentials
Keyless works locally too. Install the azure extra, sign in (or use a managed
identity), set the endpoint, and review your current branch's changes:
```bash
pip install 'lgtmaybe[azure]'
az login
export AZURE_API_BASE="https://.openai.azure.com"
lgtmaybe review \
--provider azure \
--model my-gpt-4o-deployment
```
`DefaultAzureCredential` finds your `az login` session (or a managed identity, or
`AZURE_*` env vars) and lgtmaybe never stores a static key.
## Key-based alternative
If you would rather use a resource key, set `AZURE_API_KEY` and `AZURE_API_BASE`
(no `id-token` permission, no `azure_client_id` needed). The `azure-identity`
extra is not required in this mode.
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: azure
model: my-gpt-4o-deployment
api_key: ${{ secrets.AZURE_API_KEY }}
api_base: ${{ secrets.AZURE_API_BASE }}
```
Locally: `export AZURE_API_KEY=… AZURE_API_BASE=…` (or pass `--api-key` /
`--api-base`).
## Troubleshooting
**`azure requires credentials`** — neither a key nor an ambient AD credential was
found. For keyless, check `id-token: write` is present and the federated
credential's subject matches the triggering ref/event. For key mode, set
`AZURE_API_KEY`.
**`azure requires the resource endpoint`** — set `api_base` (or `AZURE_API_BASE`)
to `https://.openai.azure.com`.
**`keyless azure needs the azure-identity package`** — running keyless on the CLI
without the extra; install `lgtmaybe[azure]`. (The Action image already bundles
it.)
**`AADSTS70021` / no matching federated identity** — the federated credential's
issuer/subject/audience don't match this workflow. Audience must be
`api://AzureADTokenExchange` and the subject must match the repo and event.
**`DeploymentNotFound` / 404** — `model` must be the **deployment name** on the
resource that `api_base` points at, not the upstream OpenAI model id.
**`Unsupported API version`** — pin one by setting `AZURE_API_VERSION`
(e.g. `2024-08-01-preview`) in the environment; litellm otherwise uses a default.
---
# Run Locally with ollama
Use this guide to review your local changes with a local ollama model — zero API
cost, zero egress, no keys required. The CLI reviews your `git` diff and prints
the findings; to post reviews on real pull requests, use the
[GitHub Action](use-as-github-action.md).
> **ollama is the easiest local start.** Running a different local server —
> **vLLM, llama.cpp, or LM Studio** — or a hosted OpenAI-compatible API like
> DeepSeek? Use the `openai-compatible` provider instead:
> [Other OpenAI-compatible servers](use-a-custom-openai-compatible-endpoint.md).
> The [model-choice and hardware guidance below](#which-model-and-will-it-fit)
> applies to any local runtime.
## Contents
- [Prerequisites](#prerequisites)
- [Pull the model you want](#pull-the-model-you-want)
- [Which model, and will it fit?](#which-model-and-will-it-fit)
- [Run the review](#run-the-review)
- [Reviewing large files (recursive walk)](#reviewing-large-files-recursive-walk)
- [Use a remote ollama instance](#use-a-remote-ollama-instance)
- [Inside the GitHub Action's container](#inside-the-github-actions-container)
- [Get findings as JSON](#get-findings-as-json)
- [Let an AI agent apply the fixes](#let-an-ai-agent-apply-the-fixes)
- [Concurrency on a local server](#concurrency-on-a-local-server)
- [The output ceiling (why a review can't run forever)](#the-output-ceiling-why-a-review-cant-run-forever)
- [Slow models and timeouts](#slow-models-and-timeouts)
- [Troubleshooting](#troubleshooting)
## Prerequisites
- lgtmaybe installed (`pip install lgtmaybe`)
- [ollama](https://ollama.com) installed and running
- A local git repository with changes to review
## Pull the model you want
```bash
ollama pull qwen3.6:27b # strong all-round coding model
ollama pull gemma4:e4b # smaller — for devices with limited RAM
```
List available models:
```bash
ollama list
```
## Which model, and will it fit?
Two simple rules:
1. **Pick a coding model.** Reviewing a PR is a coding task, so use a model built
for code (e.g. the Qwen3 coder line), not a general chat model. Models are
tuned for different jobs — match the model to the use case.
2. **Bigger and newer is more accurate.** Use the largest, most recent coding
model your hardware can run. Our accuracy numbers are for a *small* model —
we benchmarked **qwen3.5:4b**, and it did well, but only *with recursive
review on* (88% vs 61% recall). A larger, newer model catches more across the
board and leans on that trick less.
A solid mid-2026 default is **Qwen3.6-27B** (`qwen3.6:27b`): near frontier API
models on coding benchmarks (SWE-bench Verified ~77%) yet small enough to run on a
workstation or a well-specced laptop, so it clears lgtmaybe's bar across all the
review lenses without a data-center GPU. Smaller models work too — accuracy just
falls off (you'll miss subtler findings and may need `--no-reflect`, because the
reflection pass over-prunes on a weak model).
**Hardware, quantised (the usual way to run it locally):**
| You have | What to expect |
|---|---|
| **< 32 GB** RAM/VRAM | Drop to a smaller model (`gemma4:e4b`) or route to a hosted provider — 27B at a usable quant won't leave room for the diff. |
| **32 GB** RAM/VRAM | The practical floor. Run `qwen3.6:27b` at a 4-bit quant (≈16–18 GB of weights) with a modest context window. Keep `num_ctx` conservative so the model plus the diff and findings fit. |
| **48 GB+** RAM/VRAM (preferred) | Comfortable. Room for the weights plus the default `--num-ctx` (32k) — or a raised 64k window for very large diffs, with headroom for the KV cache. |
This applies to both discrete VRAM and Apple-Silicon unified memory. A bigger
context window costs memory on top of the weights, so if you bump `--num-ctx` for
a large diff (see [Slow models and timeouts](#slow-models-and-timeouts)), size it
to the table above. On a hosted provider none of this matters — the model runs on
the provider's hardware.
## Run the review
From inside the repo, on the branch you want reviewed:
```bash
lgtmaybe review \
--provider ollama \
--model qwen3.6:27b \
--api-base http://localhost:11434
```
This diffs your current branch against the remote primary branch and prints the
findings. Add `--working` to review the whole worktree (branch commits plus
uncommitted edits) against that same base, `--uncommitted` to review only your
uncommitted edits against HEAD, or `--base [` to diff against a different
base.
## Reviewing large files (recursive walk)
When a single file's diff is larger than the per-call token budget
(`--max-input-tokens`, default 100000), lgtmaybe **walks it hunk-by-hunk** —
each hunk reviewed in its own focused call — instead of sending the whole file at
once and letting the tail drop out of the model's attention. The findings are
merged back together, and inline-comment positions still bind to the real diff.
This **RLM-style recursive review is on by default** (`recursive: true`); files
that already fit the budget are still reviewed whole, so nothing changes for small
diffs.
It helps **small local models the most**, because a smaller, focused prompt is
easier to review thoroughly. In our A/B benchmark a local **qwen3.5:4b** caught
**all 6** planted bugs reviewing recursively versus **4/6** reviewing each file
whole — the two it missed whole were both in the file's *tail*, even though the
diff fit the context window (so the gain is focus, not just avoiding truncation).
It's a single non-deterministic run on one fixture, so treat it as directional;
the harness behind it is in
[DEVELOPMENT.md](https://github.com/MattJColes/lgtmaybe/blob/main/DEVELOPMENT.md#benchmarking-the-recursive-rlm-walk).
To use the **original whole-file method** instead — one call per file, which keeps
all of a file's hunks in view together but tends to miss more on big files with
small models — pass `--no-recursive`:
```bash
lgtmaybe review --provider ollama --model qwen3.5:4b \
--api-base http://localhost:11434 --no-recursive
```
```yaml
# or in .lgtmaybe.yml (also how the GitHub Action picks it up):
recursive: false
```
## Use a remote ollama instance
If ollama runs on another machine (e.g. a Tailscale peer):
```bash
lgtmaybe review \
--provider ollama \
--model qwen3.6:27b \
--api-base http://100.x.x.x:11434
```
No authentication is added — ollama has no built-in auth. Ensure network access
is restricted at the host or firewall level.
## Inside the GitHub Action's container
The Action runs lgtmaybe in a container, so ollama on the runner host is reached
at `host.docker.internal` rather than `localhost`. Set it in `.lgtmaybe.yml`,
since the Action reads its provider settings from config:
```yaml
provider: ollama
model: qwen3.6:27b
api_base: http://host.docker.internal:11434
```
## Get findings as JSON
The CLI prints a readable listing by default and never posts anywhere. Add
`--json` for a machine-readable array you can pipe into other tooling:
```bash
lgtmaybe review \
--provider ollama \
--model qwen3.6:27b \
--api-base http://localhost:11434 \
--json
```
## Let an AI agent apply the fixes
`--format agent` prints the findings as correction instructions an AI coding
agent (such as Claude Code) can read and apply, so you can review and fix a
branch locally before opening a PR. See
[Fix findings with an AI agent](fix-findings-with-an-ai-agent.md).
## Concurrency on a local server
lgtmaybe fans its review calls out across a pool sized by `max_concurrency`,
**6 by default on every provider, local included**. That is a ceiling on what
lgtmaybe will have in flight — it is not a promise your server will run them at
once, and the distinction matters:
- **A default ollama runs one at a time.** `OLLAMA_NUM_PARALLEL` is `1` unless you
set it, so five of six calls simply queue (up to `OLLAMA_MAX_QUEUE`, 512). They
are not lost and nothing fails; the wall clock is your server's throughput
either way. Raising `max_concurrency` alone therefore changes nothing.
- **The knob that matters is on the server.** Start ollama with
`OLLAMA_NUM_PARALLEL=4` and four calls genuinely run together.
- **It costs memory, in proportion.** Ollama allocates the context window *per
parallel slot*: four slots at `num_ctx` 32768 needs 128k of context allocated,
not 32k. This is the usual reason a machine that reviewed happily at one slot
falls over at four.
So the two settings want to match. If you have raised the server, tell lgtmaybe:
```bash
lgtmaybe review --max-concurrency 4
```
**How to tell which you got.** `--profile` already answers it, no extra flag
needed: compare the per-call `elapsed` column against the `review` stage total.
If the calls' elapsed times sum to roughly the stage time, they ran one after
another; if they sum to well over it, they genuinely overlapped.
```
review 95.32s <- stage total
correctness 52.02s |
spec 45.64s | sum is ~3x the stage,
artefacts 43.67s | so these overlapped
security 43.30s |
```
**Queue time is already paid for.** A queued call's timeout clock starts when the
request is *sent*, not when your server gets to it, so on a single-slot box the
last call in a six-wide fan-out would otherwise have to be served within the same
budget as the first. lgtmaybe scales the local default by the fan-out width to
cover that (see below), so you should not need to intervene. If you would rather
not have the calls queue at all:
```yaml
# .lgtmaybe.yml
max_concurrency: 1
```
## The output ceiling (why a review can't run forever)
A local model under structured output sometimes fails to stop: the response just
keeps decoding. With no ceiling the only thing that ends it is the per-call
timeout — half an hour of sustained GPU for a single lens on a one-file diff, and
hours for a whole review.
So a review runs with a **default output ceiling of 16384 tokens per call** —
half the default 32768 `num_ctx`. Measured findings payloads are hundreds of
tokens, so the rest is headroom for a thinking model — reasoning is drawn from
this same budget.
The same ceiling applies to `openai-compatible` and `openrouter`. The failure is
not specific to local models: benchmarking found a model behind
`openai-compatible` emitting 223,558 tokens on a single lens call, and a hosted
model on `openrouter` returning a 393k-token response that parsed into 699 false
positives on a diff with nothing wrong in it. The first-party APIs (openai,
anthropic, bedrock, vertex, azure) get no default ceiling, because they have not
shown this failure and a cap would risk truncating long findings payloads.
The ceiling is a fixed number, not a fraction of your window: raising `num_ctx`
for a big diff adds room for the **prompt**, not for a longer answer. Use
`max_tokens` when you want a longer answer.
Every run says which ceiling it resolved and where it came from:
```
per-call budget resolved timeout_s=3600 max_tokens=16384 max_tokens_source="provider default"
```
If a lens does hit it you get a **truncation notice**, not a silent clean review:
```
response truncated at the 16384-token `max_tokens` ceiling — the batch is
re-reviewed in smaller pieces automatically, so a lens that keeps doing it is
usually generation instability in the model, which a higher ceiling makes more
expensive rather than prevents
```
**One truncation is not a reason to raise the cap.** The batch is re-reviewed in
smaller pieces on its own, so a lens that trips it occasionally has still been
reviewed. A model that runs away repeatedly is generating past the ceiling rather
than being cut short by it. A bigger ceiling only produces a longer wasted call;
changing the model is what fixes it.
Where a raise *is* right is a genuinely long answer being cut off — a large diff
with many real findings, salvaging most of them each time. Then raise it, or turn
the cap off entirely with `0`, which puts the run back on the timeout as its only
stop:
```bash
lgtmaybe review --provider ollama --model qwen3.5:4b --max-tokens 32768
lgtmaybe review --provider ollama --model qwen3.5:4b --max-tokens 0 # uncapped
```
```yaml
# or in .lgtmaybe.yml (also how the GitHub Action picks it up):
max_tokens: 32768
```
Lowering the ceiling is the quickest way to get a result out of a slow model:
`--max-tokens 512` turns a stuck review into one that finishes in minutes and
reports what it truncated.
## Slow models and timeouts
Local models are slow, especially large ones on CPU, so lgtmaybe gives **ollama a
long default per-request timeout (1800 seconds)** automatically — you don't need
to set anything for a normal run. (Direct cloud providers default to 600 s.)
**That default scales with the fan-out.** Because the calls queue, the budget has
to cover the wait as well as the work: at the default width of six the resolved
per-call timeout is `1800 × 6`, **bounded by `max_review_seconds`** (3600 s by
default; `0` disables the deadline and with it the cap) — though never trimmed
below ollama's own 1800 s, so a deadline set under that does not shrink it. (And it bounds the budget rather than the wall clock: the
deadline decides when a call may *start*, so one that begins just inside it still
runs its budget out.) At
`max_concurrency: 1` there is no queue and the budget is the plain 1800 s. Every
run logs which number it resolved, and why:
```
per-call budget resolved timeout_s=3600 timeout_source="provider default" concurrency=6
```
An explicit `timeout` is never scaled — `timeout: 600` means 600 at any width.
If a big model still times out — you'll see
`litellm.Timeout: Connection timed out after 3600.0 seconds` — raise it explicitly:
```bash
# CLI flag (seconds):
lgtmaybe review --provider ollama --model qwen3.6:35b \
--api-base http://localhost:11434 --timeout 3600
```
```yaml
# or in .lgtmaybe.yml (also how the GitHub Action picks it up):
provider: ollama
model: qwen3.6:35b
timeout: 1800
```
The review fans out **four calls** under the default `fast` preset (nine under
`--preset full`). On a default ollama those queue rather than overlap, so a slow
model takes roughly `lens calls × per-call time` — which is exactly why `fast` is
the default: four calls instead of nine is the single biggest local speed-up.
To go faster still, narrow the lenses with `categories:` in `.lgtmaybe.yml`
(e.g. just `security` and `correctness`), use a smaller model, or give ollama
more GPU. If you have the VRAM to truly serve requests in parallel, raise
`OLLAMA_NUM_PARALLEL` on the **ollama server** as described above — the same four
calls then overlap instead of queueing, which is close to a 4× wall-clock win.
Add `--profile` to any run to see the per-call breakdown.
## Troubleshooting
**`Connection refused` on port 11434** — ensure `ollama serve` is running and
the `--api-base` URL is reachable.
**Model not found** — run `ollama pull ` before using it.
**`review incomplete — every review call failed`** — every category
call timed out or returned output that wasn't valid JSON. Raise `--timeout`, try a
model that follows instructions more reliably, or check `LITELLM_LOG=DEBUG` output
for the underlying error. lgtmaybe reports this (and exits non-zero) rather than
pretending the PR is clean.
For a **large diff** this can mean the prompt plus the findings don't fit in
ollama's context window and the output gets truncated. lgtmaybe runs ollama with
a generous context (`num_ctx` of 32768) and **structured JSON output**, which
covers most reviews.
lgtmaybe does not pass ollama's `think` flag either way. Ollama already defaults
thinking **on** for a model that supports it and rejects the flag outright for one
that does not, so sending nothing is the only choice that is right for both.
lgtmaybe used to force it off — that made reasoning models review with their
reasoning switched off, which measurement says is the single biggest lever on
finding quality there is.
For a big multi-file change ("vibe-coded" commits across many files), raise the
context window with `--num-ctx` so the whole diff and the findings fit — this is
**ollama-only** (hosted providers manage their context window server-side and
ignore it):
```bash
# A large multi-file diff on a local model — more time and more context
# (32768 is already the default; go above it for very large diffs):
lgtmaybe review --provider ollama --model qwen3.6:35b \
--api-base http://localhost:11434 --timeout 1800 --num-ctx 65536
```
```yaml
# or in .lgtmaybe.yml (also how the GitHub Action picks it up):
provider: ollama
model: qwen3.6:35b
timeout: 1800
num_ctx: 65536
```
`--num-ctx` needs enough RAM/VRAM on the ollama host — a bigger window costs
memory, so size it to your machine. The token budget that decides when lgtmaybe
splits a diff into separate model calls is `--max-input-tokens` (default 100000),
which applies to **any** provider — raise it to send a large diff in fewer calls,
lower it for a small-context model. If a very large diff still truncates, narrow
it with `include_paths` / `exclude_paths` or a lower `max_files` in `.lgtmaybe.yml`,
or run a model with a bigger context window.
> **Keep `--max-input-tokens` under `--num-ctx`.** The two are independent:
> `--max-input-tokens` caps each batch lgtmaybe *sends*, while `--num-ctx` is the
> window ollama actually *allocates*. lgtmaybe estimates tokens with a generic
> tokenizer, and local models tokenize differently, so leave headroom — a batch
> budget comfortably below your context window (e.g. `--max-input-tokens 24000`
> with `--num-ctx 32768`) avoids ollama silently truncating the findings JSON,
> which otherwise surfaces only as an unhelpful "review failed".
**Review is empty or truncated** — the diff may exceed the model's context
window. Add a path filter in `.lgtmaybe.yml` to reduce diff size, or set
`max_files` to a lower value.
---
# Local Models & other OpenAI providers
Lots of model servers speak the OpenAI `/v1` wire format: local and self-hosted
runtimes like [vLLM][vllm], [llama.cpp][llamacpp], and [LM Studio][lmstudio], plus
hosted APIs like [DeepSeek][deepseek] and many proxies. The `openai-compatible`
provider points lgtmaybe at any of them — you supply the base URL, and (if the
server wants one) a key.
Use this provider when the model you want is not in the built-in provider list:
anything that exposes an OpenAI-compatible `/v1` endpoint works through one flag.
> Some endpoints that *could* run through here have a first-class provider
> instead — use it for less setup: [z.ai / GLM](review-with-zai.md) (`zai`) and
> [ollama](run-locally-with-ollama.md) (`ollama`).
## Contents
- [Local models at a glance](#local-models-at-a-glance)
- [How it works](#how-it-works)
- [DeepSeek (hosted, keyed)](#deepseek-hosted-keyed)
- [llama.cpp (local, keyless)](#llamacpp-local-keyless)
- [LM Studio (local, keyless)](#lm-studio-local-keyless)
- [vLLM (local or self-hosted, keyless)](#vllm-local-or-self-hosted-keyless)
- [Concurrency: what each server can actually take](#concurrency-what-each-server-can-actually-take)
- [Persist it in `.lgtmaybe.yml`](#persist-it-in-lgtmaybeyml)
- [Gateways that don't support JSON mode (`response_format`)](#gateways-that-dont-support-json-mode-response_format)
## Local models at a glance
Run a model on your own hardware — zero cost, no key, nothing leaves the machine:
| Runtime | Provider | Key needed? | See |
|---|---|---|---|
| [ollama][ollama] | `ollama` (native) | No | [Run locally with ollama](run-locally-with-ollama.md) |
| [vLLM][vllm] | `openai-compatible` | No | [below](#vllm-local-or-self-hosted-keyless) |
| [llama.cpp][llamacpp] | `openai-compatible` | No | [below](#llamacpp-local-keyless) |
| [LM Studio][lmstudio] | `openai-compatible` | No | [below](#lm-studio-local-keyless) |
ollama has its own first-class `--provider ollama` (it's the easiest local
start), so it gets its [own guide](run-locally-with-ollama.md). vLLM, llama.cpp,
and LM Studio are reached through `openai-compatible` and the `--api-base` of
their local server, as shown below.
**Which model, and will it fit?** The same model-choice and hardware guidance
applies to any local runtime — pick a coding model, bigger and newer is more
accurate, and size it to your RAM/VRAM. See
[Which model, and will it fit?](run-locally-with-ollama.md#which-model-and-will-it-fit)
in the ollama guide.
## How it works
`--provider openai-compatible` routes through litellm's OpenAI client, but sends
your requests to the `--api-base` you give instead of `api.openai.com`. The
**base URL is required** (that's the whole point); the **API key is optional**:
- **Hosted endpoints** (DeepSeek, a paid proxy) need a key — pass `--api-key` or
set `OPENAI_COMPATIBLE_API_KEY`.
- **Local servers** (llama.cpp, LM Studio, vLLM) usually need none. lgtmaybe
sends a harmless placeholder key in that case, because the OpenAI client rejects
an empty one.
The API key, when you do supply one, is read from the environment or `--api-key`
and is **never persisted** to config.
Because the endpoint might be a slow local model, `openai-compatible` defaults to
the same generous **1800s** per-call timeout as ollama. For a fast hosted endpoint
like DeepSeek you can dial it down with `--timeout` (or `timeout:` in config).
## DeepSeek (hosted, keyed)
```bash
export OPENAI_COMPATIBLE_API_KEY=sk-... # your DeepSeek key
lgtmaybe review \
--provider openai-compatible \
--model deepseek-chat \
--api-base https://api.deepseek.com/v1
```
You can pass the key inline with `--api-key sk-...` instead of the env var.
## llama.cpp (local, keyless)
Start the server:
```bash
llama-server -m ./model.gguf --port 8000 # serves the OpenAI API at /v1
```
Then review against it — no key needed:
```bash
lgtmaybe review \
--provider openai-compatible \
--model local-model \
--api-base http://localhost:8000/v1
```
## LM Studio (local, keyless)
Enable the local server in LM Studio (it serves the OpenAI API, default port
`1234`), then:
```bash
lgtmaybe review \
--provider openai-compatible \
--model your-loaded-model \
--api-base http://localhost:1234/v1
```
## vLLM (local or self-hosted, keyless)
```bash
vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
```
```bash
lgtmaybe review \
--provider openai-compatible \
--model meta-llama/Llama-3.1-8B-Instruct \
--api-base http://localhost:8000/v1
```
## Concurrency: what each server can actually take
lgtmaybe fans out across a pool sized by `max_concurrency`, **6 by default on
every provider**. That is a ceiling on what it will have in flight; whether your
server runs them together is the server's business, and the three above differ
sharply.
| server | concurrent by default? | raising it |
|---|---|---|
| **vLLM** | yes — continuous batching | close to free; each request keeps the full `--max-model-len` |
| **llama.cpp** | no, one slot | `-np N` adds slots, but **splits one KV cache between them** |
| **LM Studio** | no, single-slot | not exposed; leave `max_concurrency` at 1 |
The llama.cpp trap is worth spelling out, because it fails as a quality problem
rather than an error. `-np` divides the context you asked for: `-c 32768 -np 4`
leaves each slot **8k**, and a lgtmaybe review prompt is comfortably larger than
that, so the diff is silently truncated and the model reviews something it was
never fully shown. Size `-c` as `slots × per-slot-context`:
```bash
llama-server -m ./model.gguf --port 8000 -np 4 -c 131072 # 4 slots × 32k each
```
vLLM allocates KV blocks dynamically rather than carving them up front, so
concurrent requests each keep the full `--max-model-len`. That is why it is the
local server to reach for when review latency matters:
```bash
vllm serve --port 8000 --max-model-len 32768
lgtmaybe review --provider openai-compatible --model \
--api-base http://localhost:8000/v1 --max-concurrency 6
```
**Queueing does not cost you a timeout.** A queued request's clock starts when it
is sent, not when the slot frees, so lgtmaybe scales the `openai-compatible`
per-call default (1800 s) by the fan-out width — `1800 × 6` at the default,
bounded by `max_review_seconds` (3600 s; `0` disables the deadline and the cap
with it) — though never trimmed below the 1800 s provider default, so a deadline
set under that does not shrink it, and a bound on
the budget rather than the wall clock, since the deadline gates when a call may
start rather than cutting a running one short. An
explicit `timeout` is honoured exactly as written, at any width. Each run logs the
number it resolved and the width it assumed:
```
per-call budget resolved timeout_s=3600 timeout_source="provider default" concurrency=6
```
For a single-slot server, you can still say so and let lgtmaybe queue nothing:
```yaml
# .lgtmaybe.yml
max_concurrency: 1
```
## Persist it in `.lgtmaybe.yml`
The provider, model, and base URL are non-secret defaults, so they can live in
config (the key stays in the environment):
```yaml
provider: openai-compatible
model: deepseek-chat
api_base: https://api.deepseek.com/v1
```
With that file in place, `lgtmaybe review` needs no flags. In the GitHub Action,
set the same values as inputs (or in `.lgtmaybe.yml`) and pass `api_key` from a
secret for hosted endpoints; leave it empty for keyless local servers reached at
`http://host.docker.internal:/v1`.
## Gateways that don't support JSON mode (`response_format`)
To keep models returning clean findings instead of prose, lgtmaybe asks for
structured output via the OpenAI `response_format` parameter (JSON mode). Most
endpoints honour it. Some enterprise gateways and custom proxies don't. They
either **ignore** it — the model then answers with the JSON wrapped in a
```` ```json ```` fence or surrounded by conversational prose — or **reject**
the request outright with a `400 Bad Request`.
lgtmaybe handles the first case for you: the parser strips fences and pulls the
JSON out of surrounding prose, so a gateway that merely ignores `response_format`
still produces a normal review. (Older versions could fail here with
`unparseable model output` on every lens — that's fixed.)
There is a third case, common with **LM Studio fronting a "thinking" model**
(e.g. qwen3.x): the server *accepts* `response_format` but the schema-constrained
decoder returns **empty content** — every lens would otherwise fail with
`unparseable model output`. lgtmaybe handles this for you too: when a structured
call comes back empty, it drops the schema and retries once, and the model then
emits the findings as normal (fenced) text the parser reads. No flag needed.
If your gateway **rejects** `response_format` with a `400`, turn it off so the
request never carries the parameter — the prompt still asks for JSON and the
lenient parser still does its job:
```bash
lgtmaybe review \
--provider openai-compatible \
--model gemini-3.5-flash \
--api-base https://api.myllm.com/v1 \
--no-structured-output
```
Persist it as `structured_output: false` in `.lgtmaybe.yml`, or set the
`structured_output` input to `false` in the GitHub Action.
[deepseek]: https://api-docs.deepseek.com/
[llamacpp]: https://github.com/ggml-org/llama.cpp
[lmstudio]: https://lmstudio.ai/
[vllm]: https://docs.vllm.ai/
[ollama]: https://ollama.com/
---
# Configure .lgtmaybe.yml
Place a `.lgtmaybe.yml` file at the root of your repository to control how
lgtmaybe reviews pull requests. CLI flags override file values; the file
provides defaults for all runs.
## Upgrading from 1.14
Remove `prompt_cache` from existing `.lgtmaybe.yml` files and remove
`--prompt-cache` or `--no-prompt-cache` from scripts. Prompt caching is now
automatic where the provider supports it.
## Contents
- [Upgrading from 1.14](#upgrading-from-114)
- [Full example](#full-example)
- [Field reference](#field-reference)
- [provider](#provider)
- [model](#model)
- [fallback_model](#fallback_model)
- [min_severity](#min_severity)
- [include_paths / exclude_paths](#include_paths-exclude_paths)
- [max_files](#max_files)
- [max_input_tokens](#max_input_tokens)
- [preset](#preset)
- [categories](#categories)
- [context_lines](#context_lines)
- [function_context](#function_context)
- [timeout](#timeout)
- [structured_output](#structured_output)
- [reflect](#reflect)
- [min_confidence](#min_confidence)
- [mid_review_retrieval](#mid_review_retrieval)
- [incremental](#incremental)
- [static_analysis](#static_analysis)
- [triage_model](#triage_model)
- [auto_diagram](#auto_diagram)
- [auto_describe](#auto_describe)
- [high_impact](#high_impact)
- [risk](#risk)
- [pr_labels](#pr_labels)
- [finding_rules](#finding_rules)
- [summary_template](#summary_template)
- [resolve_fixed](#resolve_fixed)
- [extra_lenses](#extra_lenses)
- [lens_paths](#lens_paths)
- [directory_rules](#directory_rules)
- [CLI flag overrides](#cli-flag-overrides)
## Full example
```yaml
provider: openai
model: gpt-5.5
min_severity: low
include_paths:
- "src/**"
- "lib/**"
exclude_paths:
- "**/__pycache__/**"
- "**/*.min.js"
max_files: 30
max_input_tokens: 80000
categories:
- security
- correctness
- tests
```
## Field reference
See [Reference: Config](../reference/config.md) for the full schema with
types and defaults.
### provider
Which LLM backend to use. One of `openai`, `openrouter`, `anthropic`, `zai`,
`bedrock`, `vertex`, `azure`, `ollama`, `openai-compatible`.
```yaml
provider: anthropic
```
### model
The model identifier for the chosen provider. Format varies by provider:
| Provider | Example model IDs |
|---|---|
| openai | `gpt-5.5` |
| anthropic | `claude-sonnet-4-6`, `claude-haiku-4-5` |
| openrouter | `anthropic/claude-sonnet-4-6` |
| zai | `glm-4.6`, `glm-4.7`, `glm-4.5-air` (GLM / Zhipu AI; newer `glm-5.x` pass through too) |
| bedrock | `us.anthropic.claude-sonnet-4-6`, `us.anthropic.claude-haiku-4-5` (prefer the cross-region inference profile; a non-Bedrock id like `openai.gpt-5.5` is invalid — see [Review with Bedrock](review-with-bedrock-oidc.md)) |
| vertex | `gemini-3-pro`, `gemini-3.5-flash` |
| azure | your deployment name, e.g. `my-gpt-4o-deployment` (not the upstream model id — see [Review with Azure](review-with-azure.md)) |
| ollama | `qwen3.6:27b`, `gemma4:e4b` |
| openai-compatible | the served model name, e.g. `deepseek-chat` or `meta-llama/Llama-3.1-8B-Instruct` (requires `api_base` — see [Use a custom OpenAI-compatible endpoint](use-a-custom-openai-compatible-endpoint.md)) |
### fallback_model
`fallback_model` names one second model for a review call that the primary
model cannot finish. It is off by default and adds no calls to a healthy
review.
Both model names go through the configured `provider`, `api_base`, and
credentials. This means an OpenRouter review can fall back from a Google model
to a Qwen model because OpenRouter serves both. An `openai` review cannot fall
back to an Anthropic account. Cross-provider failover needs a separate review
run.
```yaml
provider: openrouter
model: google/gemini-3.7-flash
fallback_model: qwen/qwen3.8-max
```
The fallback runs at different points depending on the failure:
| Primary model result | What lgtmaybe does |
|---|---|
| Transient provider failure, such as a connection error, capacity rate limit, or 5xx | Exhausts the primary model's bounded retry policy, then tries the fallback model. |
| Permanent provider failure, such as an unknown model, rejected request, or exhausted quota | Skips repeated primary attempts and tries the fallback model. A shared credential, quota, or endpoint failure will usually reject both. |
| Output reaches the `max_tokens` ceiling | First retries a smaller payload or lowers `reasoning_effort` on the primary, based on the reported token counts. It tries the fallback only if that remedy also fails. |
The fallback does not judge the primary answer. A clean review, a low-quality
finding, or zero findings will not trigger it. Output parsing has its own
bounded repair path.
The fallback gets a fresh request and retry budget, so it adds to the primary
model's spent time and tokens. The profile keeps both models' calls and the
review summary names every lens answered by the fallback. Use `--profile` for
the human-readable `models:` line or `--profile-json` for the model attached to
each call. Its usage contributes to `max_review_tokens` after the call returns.
See [Profile Reference](../reference/profile.md#the-models-line).
If the fallback fails, lgtmaybe marks the lens incomplete and stops that
recovery path. A review with no usable results fails instead of posting a clean
result. lgtmaybe does not walk a list of models or fall back to the primary
again. A lens that repeatedly needs the second model is a signal to make that
model the primary.
Set the same value for one run with `--fallback-model`. The CLI flag and the
GitHub Action's `fallback_model` input override `.lgtmaybe.yml`.
```bash
lgtmaybe review --fallback-model qwen/qwen3.8-max
```
See [Choose a Review Model](choose-a-review-model.md) for current model results.
### min_severity
The minimum severity level to report. Findings below this threshold are
suppressed. Ordered low to high: `info`, `low`, `medium`, `high`, `critical`.
```yaml
min_severity: medium # suppresses info and low findings
```
Default: `low` (suppresses only `info` findings).
### include_paths / exclude_paths
Glob patterns to restrict which files in the diff are reviewed.
`include_paths` acts as an allowlist; `exclude_paths` acts as a denylist applied
after the allowlist, so an exclude always wins. Both default to empty (all files
included). Patterns match against the full repo-relative path, and a
`**/`-prefixed pattern also matches at the repo root (so `**/*.lock` covers a
root-level lockfile). The built-in skip filter for generated, vendored, and
binary files runs first either way — an `include_paths` entry can't resurrect a
lockfile.
```yaml
include_paths:
- "src/**"
exclude_paths:
- "src/generated/**"
- "**/*.lock"
```
### max_files
Maximum number of changed files to include in the review. Files beyond this
limit are skipped. Reduces token usage on large PRs.
```yaml
max_files: 30
```
Default: `50`.
### max_input_tokens
Token budget **per model call**. When the compressed diff exceeds this limit,
lgtmaybe splits it across multiple batched calls (and, with `recursive` on,
walks an over-budget single file hunk-by-hunk) — nothing is truncated or
dropped.
```yaml
max_input_tokens: 80000
```
Default: `100000`.
### preset
How many model calls the review spends. `fast` (the default) covers all nine
categories in **four calls**, one per concern:
| Call | Covers |
|---|---|
| security | security |
| correctness | correctness, and stated intent when the PR states one |
| code health | performance, complexity, ponytail, deprecation |
| artefacts | tests, documentation |
The same four run on every provider — worker count changes only how they are
scheduled, not how many there are. `full` runs one focused call per lens for
release branches and deep audits.
```yaml
preset: full
```
Default: `fast`. CLI: `--preset fast|full` (`--full` is shorthand). An explicit
`categories` list (below) overrides the preset grouping.
### categories
Which review lenses to run. An explicit list disables the preset grouping: the
reviewer asks for each listed category in its own concurrent model call and
merges the findings, so a focused prompt concentrates on one concern at a
time. One or more of `security`, `correctness`, `deprecation`, `tests`,
`documentation`, `performance`, `complexity`, `intent`, `ponytail`. Narrowing
the list trades thoroughness for fewer model calls (and lower token usage).
The `ponytail` lens is the "lazy senior dev" check — *the best code is the code
you never wrote* — flagging code that needn't exist at all (YAGNI, reach for the
standard library, do it in fewer lines). See
[What gets reviewed](../explanation/what-gets-reviewed.md#ponytail-the-laziest-senior-dev-in-the-room).
The `intent` lens checks the diff against the PR's stated intent — title,
description, and commit names on GitHub; your `git log` commit names on the
CLI (in both branch and `--working` mode). When nothing states an intent (e.g.
no commits beyond the base branch yet), it is skipped automatically, so it
never costs an extra call (under the `fast` preset it shares correctness's
call). It is also the only lens that sends the PR title/description/commit
names to the provider — drop it from `categories` if you don't want that text
sent at all.
```yaml
categories:
- security
- correctness
```
Default: all nine categories.
### context_lines
Ceiling on the number of unchanged lines added around each changed hunk. The
lines are read from the head revision of the file, so the model reviews a
change in the context of its surrounding code. The pad is **asymmetric**: the full budget goes
before the hunk (the enclosing signature and setup explain a change best) and a
quarter of it — at least one line — goes after. The actual number used is the
smaller of this ceiling and what the token budget allows, so it shrinks
automatically on large PRs. Set it to `0` to disable context expansion and
review the bare diff (no extra file content is fetched).
```yaml
context_lines: 10 # at most 10 lines before each hunk (2 after); 0 disables
```
Default: `20`.
### function_context
Extend each hunk's leading pad up to the **enclosing function or class
signature** when it sits above the fixed `context_lines` window — the
signature and setup explain a change better than an arbitrary cut. Boundaries
are found structurally with ast-grep (already bundled for symbol resolution;
parsing only, never executing) for Python, JS/TS/TSX, Go, Rust, Java, and
Ruby, with a bounded reach so a distant definition can't drown the diff.
Unsupported languages and any ast-grep failure keep the plain fixed-line pad.
```yaml
function_context: false # fixed-line padding only
```
Default: `true`.
### timeout
Per-request timeout in seconds for each model call. Left unset, lgtmaybe picks a
**provider-aware default**: **1800 s for ollama, openai-compatible, and
openrouter** (local models are slow, and openrouter can route to slow reasoning
models) and 600 s for direct cloud providers. Set it explicitly to raise it for a
large local model.
```yaml
timeout: 3600 # an hour per call, e.g. for a big model on CPU
```
Default: auto (ollama/openai-compatible/openrouter 1800 s, cloud 600 s). See
[Run locally with ollama](run-locally-with-ollama.md#slow-models-and-timeouts).
### structured_output
Constrain the model to emit the findings JSON schema using the provider's native
JSON mode (litellm `response_format`). This keeps models — especially local ones —
from returning prose or reasoning instead of findings. Leave it on unless the
combination in front of you fails in one of two ways:
1. A gateway **rejects** `response_format` outright (some `openai-compatible`
gateways return a `400`). The review fails loudly; turn this off.
2. A gateway **accepts** JSON-schema mode but returns replies the parser reads
as empty. This fails *silently*: every lens logs
`findings: 0 parsed / 0 returned`, the run exits clean, and a capable model
looks like one that finds nothing. Seen with
`anthropic/claude-fable-5.1` behind OpenRouter on lgtmaybe 2.3.0 and 2.7.0 —
the same diff with the same flags returned 11 parsed findings once this was
off. If a model is unexpectedly silent, re-run one diff with
`--no-structured-output` before concluding the model finds nothing.
In both cases the lenient parser still strips fences and pulls JSON out of any
surrounding prose. CLI: `--no-structured-output`.
```yaml
structured_output: false # if your gateway rejects JSON-schema mode, or a model returns zero findings under it
```
Default: `true`. See
[Use a custom OpenAI-compatible endpoint](use-a-custom-openai-compatible-endpoint.md#gateways-that-dont-support-json-mode-response_format).
### reflect
Run the **self-reflection pass** that audits the merged findings and drops the
ones the model marks low-confidence, before anything is posted. This trims false
positives, so leave it on for most models. Turn it **off** for a weaker or local
model that over-prunes and drops valid findings during the audit. CLI:
`--no-reflect`.
```yaml
reflect: false # keep every finding; skip the false-positive audit
```
Default: `true`. To audit a weak reviewer's findings with a stronger model
instead of disabling the pass, set `reflect_model` to that model id (it uses the
same provider and credentials as `model`).
### min_confidence
During reflection the auditor also scores each kept finding's confidence from
0 (certainly a false positive) to 10 (certain it is real), reached by actively
trying to disprove the finding against the diff and the file text. Findings
scored **below** `min_confidence` are dropped before posting; the surviving
score is shown in the CLI output, the JSON export, and — as a percentage — on the
posted GitHub comment itself: `**[HIGH · security · 80%] Title**`, alongside the
lens that raised it. A finding the auditor keeps but doesn't score always survives the
threshold — a missing score never drops a real finding, and the comment simply
omits that half of the badge (as it does for every finding when `reflect:
false`). CLI: `--min-confidence`.
```yaml
min_confidence: 5 # drop findings the auditor scores 0-4
```
Default: `0` (no numeric filtering — reflection prunes only via its keep/drop
verdicts, as before the score existed).
### mid_review_retrieval
Let a **review lens defer once** for bounded, read-only codebase context.
By default every lens is told the diff is only a slice of the codebase: if a
finding depends on code it cannot see — a guard, a base class, the helper it
calls — it must hedge the wording, lower the severity, or say nothing. That
protects you from confident cross-file nonsense, and it also means a real bug
whose evidence lives one file away is never reported.
With this on, the lens gets a third option: alongside its findings it names the
files (or symbols) it must read. lgtmaybe fetches them **read-only** — the same
API/worktree read the reflection auditor's deferral uses, never a checkout of PR
code — redacts them, and re-runs that one lens with them in front of it. Both
calls' findings are kept and deduped, so a deferral can only add findings.
```yaml
mid_review_retrieval: true # trade tokens for cross-file recall
```
Default: `false`. It is opt-in because of the price: **up to one extra model
call per (batch, lens)**, carrying the fetched text as well. Every other bound is
fixed — one hop (the re-run cannot defer again), at most five files inside a
quarter of `max_input_tokens`, and a deferral arriving past `max_review_seconds`
or `max_review_tokens` is skipped and reported in the summary. Read
[Reduce Review Cost](reduce-review-cost.md#what-costs-more-on-purpose) before
turning it on, and pair it with `max_review_tokens` if the worst case matters.
CLI: `--mid-review-retrieval`; Action input: `mid_review_retrieval`.
### incremental
Commit-scoped incremental review, for the GitHub posting path. On a re-run
lgtmaybe reads a hidden watermark (the head SHA its last completed review
covered) from its own summary comment and reviews **only the diff of the
commits pushed since**, instead of the whole PR — faster, cheaper, and no
re-noise on code that was already reviewed. New findings post as inline
comments; findings on files outside the increment stay open, and are only
auto-resolved by a run that actually re-reviewed their file.
It always degrades to a **full review** when there is no watermark yet (first
review), after a force-push/rebase (the increment would be meaningless), or if
the compare fails. A failed review never moves the watermark, so no commit is
ever silently skipped. Comment `/review full` on the PR to force a full
re-review on demand.
```yaml
incremental: false # every run reviews the whole PR
```
Default: auto — incremental on a `synchronize` push (new commits on an
already-reviewed PR), full review everywhere else (open/reopen, slash
commands, and the local CLI, which never uses it).
### static_analysis
Static-analysis fusion: run fast, deterministic tools over the changed files.
Each tool reaches the review in one of two **modes**:
- **`hint`** — findings become **hints for the model to confirm, contextualise,
or discard**. This raises recall on exactly the mechanical bugs LLMs miss
without posting raw linter noise; only findings the model itself confirms are
reported. The default for **ruff**, **bandit**, **mypy** and **semgrep**.
- **`finding`** — findings are posted directly, with **no model call at all**.
The default for **gitleaks**, **zizmor**, **ast-grep** and **osv-scanner**.
Deterministic, free, and identical run to run.
The split is about the tool, not taste: a committed credential is present or it
isn't, so asking a model to "confirm or discard" a regex match only adds latency
and a chance of it talking itself out of a real hit. A lint or a SAST heuristic
is the opposite — often technically true and beside the point — which is exactly
what the model is good at filtering. Override either way with `tool_mode`.
Supported tools: **ruff**, **bandit**, and **mypy** (Python), **gitleaks**
(secrets, any language), **zizmor** (GitHub Actions workflow security — template
injection, unpinned `uses`, over-broad permissions; it runs only when the PR
changes a workflow file), **ast-grep** (your own structural rules — the
deterministic sibling of `extra_lenses`; set `ast_grep_rules`), **osv-scanner**
(known vulnerabilities in your dependencies), and **semgrep** (multi-language
SAST).
**osv-scanner** is the one check a model genuinely cannot do: whether a pinned
dependency has a published advisory depends on what was disclosed this week, and
a model's knowledge cutoff makes any answer it gives unreliable. It reads the
dependency manifests and lockfiles your PR changes — fetched into a **scan-only
channel** that never enters the diff, a prompt, or the reflection pass, because
nobody wants a model commenting on line 84,000 of a lockfile.
Its findings are about the dependency rather than a line, so they render in the
review body rather than as inline comments, and only `high`/`critical`
advisories surface by default (`unanchored_min_severity`).
It needs a local vulnerability database — the sandbox has no network, so it can
never fetch one. The GitHub Action image ships one, baked in at build time. On
the CLI, seed it once with `osv-scanner --download-offline-databases`; without a
database the scanner reports nothing, and the review says so rather than passing
it off as clean.
When osv-scanner is set to post findings, the review prompt **stops asking the
model** whether a version has a known advisory or a package is abandoned — a
knowledge cutoff cannot answer either, so asking would only put a confident
guess beside an accurate answer. Everything a vulnerability database cannot
answer stays with the model: deprecated APIs, end-of-life runtimes, typosquats
and licence conflicts. Demote osv-scanner to `hint` and the model takes those
claims back.
The same happens for **gitleaks**. When it is set to post findings, the review
prompt **stops asking the model** for hardcoded secrets — and here the ask was
close to unanswerable anyway: lgtmaybe redacts every secret it matches *before*
the diff leaves for the provider, so the model was being asked to spot what it
had been prevented from seeing, while gitleaks reads the unredacted file text
and answers exactly. Secrets reaching a **log** are a different defect that no
secret scanner reports, so the model keeps that one. Demote gitleaks to `hint`
and the model takes the claim back.
**semgrep now works out of the box.** It used to skip itself unless you set
`semgrep_rules`, which almost nobody did — so the one multi-language tool never
ran. It now falls back to a small, high-precision **MIT rule pack shipped with
lgtmaybe**, and `semgrep_rules` overrides which rules it uses. Rules are always
read from a local path: semgrep's registry configs need the network, which the
sandbox forbids.
We ship our own rules rather than bundling a well-known upstream pack because
the widely-used semgrep-rules / opengrep-rules collections are LGPL-2.1 **plus a
Commons Clause** — not an open-source licence, and not something an MIT package
can redistribute honestly. Point `semgrep_rules` at a directory to use a fuller
pack you have obtained yourself.
Two rules keep direct posting honest. Tools read **whole files**, but only the
diff is under review, so a finding on a line this PR did not change is dropped
and counted in the summary — otherwise a fake credential in a test fixture would
post on every PR that touches that file, forever. And direct findings are capped
per review, most severe first, since no model is there to filter volume.
Posted findings carry a `scan:` category, so `finding_rules` can drop or
re-grade a scanner without turning it off:
```yaml
finding_rules:
- match: {category: "scan:gitleaks", path: "tests/fixtures/**"}
action: drop
```
**mypy** earns its place on unguarded-`Optional` bugs: a `dict.get()` narrowed
to `str | None` and then dereferenced is a crash a review lens reads straight
past, and mypy proves it from the file's own text in seconds. It runs with
`--ignore-missing-imports --follow-imports=skip`, because the sandbox holds only
the changed files and everything they import is absent by construction —
so it reports what it can prove from a single file, and stays quiet about the
rest (untyped code and unresolvable imports produce nothing).
The tools run against the already-fetched file texts in a throwaway directory
(never a checkout, never executing PR code), in a subprocess with a scrubbed
environment (no proxy or credential variables) and a hard timeout. A tool that
isn't installed is skipped silently — install them with
`pip install lgtmaybe[static-analysis]`, or rely on whatever is already on
PATH. Tool output is treated as untrusted text: redacted and
injection-wrapped before it reaches the model. CLI:
`--static-analysis/--no-static-analysis`.
```yaml
static_analysis:
enabled: true
tools: [ruff, bandit, mypy, gitleaks, zizmor] # default: all supported tools
min_severity: low # floor on mapped tool severity (default info)
tool_min_severity: # per-tool overrides of the global floor
ruff: medium # only medium+ from ruff; bandit keeps `low`
tool_mode: # per-tool overrides of hint vs finding
gitleaks: hint # route secrets through the model instead
# semgrep_rules: .semgrep.yml # override the bundled MIT pack
# ast_grep_rules: .ast-grep/ # your own structural rules (skipped when unset)
```
Default: `enabled: false` — no subprocess ever runs and behaviour is
unchanged.
### triage_model
Two-stage model routing so routine PRs don't pay frontier prices while risky
ones still get the strong model. When set, this **cheap** model runs first
over the compressed per-file diffs. It skips files that plainly need no review
(pure formatting, trivial renames, generated churn) and scores the rest 0–10
by risk; the strong `model` then does the deep per-lens review only on the
survivors, riskiest first. Skipped files are listed in the review summary, and
`/review full` reviews everything on demand.
A deterministic **security floor** always escalates past triage, whatever the
cheap model says: security-relevant paths (auth/crypto/session code,
migrations, IaC, CI workflows, dependency manifests), patches carrying
security-relevant tokens, files with static-analysis hits, and large hunks.
Any triage failure — an unparseable verdict, a provider error — reviews
everything.
All three model slots (`triage_model`, `model`, `reflect_model`) resolve
through the same provider and credentials, so pointing them all at one ollama
model costs nothing. **Trade-off:** cheaper, faster reviews at the risk of the
triage model under-rating a subtle change; the floor and the
review-when-unsure prompt bound that risk, but for maximum recall leave triage
off. CLI: `--triage-model`.
```yaml
triage_model: claude-haiku-4-5 # cheap gatekeeper; unset = no triage
```
Default: unset (no triage — every file gets the full review, exactly as
before).
### auto_diagram
Post the **change overview** as a comment after the review, and refresh it on
every push: a description of the change, a **High Impact Areas** section, a
Mermaid flowchart of the components the PR touches, and — when the change alters
a run-time flow — a Mermaid sequence diagram of it. One comment, updated **in
place**, never duplicated. `/diagram` posts the same overview on demand.
This is the switch for the whole comment: with it off, nothing auto-posts
besides the review itself.
```yaml
auto_diagram: false
```
Default: `true`. See [Generate a change
overview](generate-a-change-diagram.md).
### auto_describe
Head the change overview with a **structured description**: a suggested title,
the change type, a short summary, a per-file walkthrough table, and — when the
PR states an intent — a "does it do what it says" check. Its own model call,
best-effort: a failure leaves a short note in that slot and never blocks the
rest of the comment.
Because it rides the overview comment, it refreshes on every push rather than
only on open. `/describe` still posts a description as its own separate comment,
whether or not this is enabled.
```yaml
auto_describe: false
```
Default: `true`.
### high_impact
Include the **High Impact Areas** section in the change overview — the changes a
reviewer must not miss, in ten areas: infrastructure, security posture,
availability (anything that could cause a production outage), data migrations,
backups and recovery, compatibility, observability, dependencies, cost, and
compliance.
Deterministic path patterns ground the model call as untrusted hints *and* floor
its output, so a sensitive file (a `*.tf`, a workflow, a migration, a backup
policy, a lockfile) is named even when the model says nothing about it. A failed
call degrades to that floor rather than dropping the section, and an empty
result says what was checked.
```yaml
high_impact: false
```
Default: `true`. See [Generate a change
overview](generate-a-change-diagram.md).
### risk
Assess each PR's **risk of change** as `low`, `medium`, `high` or `critical`,
with the reasons behind it. It is deterministic and makes no model call. It
combines how many files in the workspace import each changed file (Python and
TS/JS, via ast-grep), the High Impact path signals, the size of the change, and
code changed without any test changing.
The level and its top reason ride the review summary line, the change overview
explains every reason, a hidden `` marker lets a
workflow read it, and with `pr_labels` on the PR gets a `risk/` label.
`core_paths` names modules the importer count can't see as central, such as a
config loader or a plugin registry. A changed file matching one is at least
`high`.
```yaml
risk:
core_paths:
- src/shared/**
- "**/settings.py"
```
Default: on, with no `core_paths`. Turn it off with `risk: {enabled: false}`,
`--no-risk`, or the Action input `risk: false`. See [Risk of
Change](../explanation/risk-of-change.md) for how the level is decided and how to
gate on it.
### pr_labels
Attach labels derived from the finished review — **no extra model calls**:
- `review-effort/1` … `review-effort/5` — a size estimate from the changed
lines, so reviewers can gauge the PR at a glance;
- `possible-security-issue` — a high/critical finding from the security lens
was posted;
- `consider-splitting` — the diff spans many unrelated top-level directories.
- `risk/` — the [risk of change](../explanation/risk-of-change.md)
verdict, when `risk` is on. One level at a time: a new verdict replaces the
old label on GitHub.
Labels are reconciled on each run (a stale `review-effort/2` is removed when
the score changes) and only lgtmaybe's own label families are ever touched.
Best-effort: a labelling failure never fails the review.
```yaml
pr_labels: true
```
Default: `false`.
### finding_rules
Declarative post-processing applied to findings just before posting — the
safe alternative to arbitrary post-processing hooks (rules can only filter or
re-grade; **no user code ever runs**). Each rule has a `match` (all specified
fields must match) and an `action`; rules apply in order.
Match fields: `path` (glob, `**/`-prefix also matches at the repo root),
`category` (the lens that produced the finding — `security`, `correctness`,
…, or a custom lens id), `title_contains` (case-insensitive substring), and
`min_severity` (at or above). Actions: `drop: true` or `set_severity`.
```yaml
finding_rules:
# complexity nits in tests aren't worth a comment
- match: {path: "tests/**", category: complexity}
action: {drop: true}
# documentation findings are informational for this repo
- match: {category: documentation}
action: {set_severity: info}
```
Default: no rules.
### summary_template
Custom template for the review summary line, for teams matching a house
style. Placeholders: `{count}` (findings posted), `{provider}`, `{model}`,
`{version}` (the lgtmaybe release that produced the review). A template that
fails to format falls back to the built-in line.
```yaml
summary_template: "🤖 {count} finding(s) · {model} · lgtmaybe {version}"
```
Keep `{version}` if you can: the same model on the same provider reviews
differently across releases, so it is the handle that makes a surprising review
traceable to the code that produced it.
Default: unset (the built-in
`N findings · provider X · model Y · lgtmaybe Z` line).
### resolve_fixed
Auto-resolve a review conversation once its finding is fixed. On a re-run, when a
finding lgtmaybe raised is no longer produced **and** GitHub marks that thread
outdated (the code under it changed), lgtmaybe posts a short `✅ Looks resolved.`
reply and resolves the conversation. Both conditions must hold, so a thread is
never collapsed just because nearby lines shifted. Set it to `false` to leave
conversations for manual resolution.
GitHub posting only — the local CLI review has no conversations to resolve, so it
ignores this. Resolving a thread uses GitHub's GraphQL API; the default
`GITHUB_TOKEN` (`pull-requests: write`, already needed to post the review) is
sufficient.
```yaml
resolve_fixed: false # leave fixed conversations open for manual resolution
```
Default: `true`.
### extra_lenses
Define your own review lenses ("BYO skills") that run **alongside** the built-in
`categories`. Each one fans out as its own focused model call and its findings
merge into the same review. A lens needs an `id` (unique, and not one of the
built-in category names) and `instructions` describing what to look for; a
`title`, plus a worked example (`example_diff` + `example_finding`, supplied
together) are optional but sharply improve a small model's output.
```yaml
extra_lenses:
- id: simplify
title: Simplify or delete
instructions: |
Flag code that should not exist at all. The best code is the code you never
wrote: prefer the standard library, an existing dependency, or one line over
a new abstraction. Call out needless wrappers, premature generality, and
"just in case" code with no caller.
example_diff: |
--- a/util.py
+++ b/util.py
@@ -4,1 +4,3 @@
def get_name(user):
+ name = user.name
+ return name
example_finding:
path: util.py
line: 5
severity: low
title: Needless local variable
body: The temporary adds nothing; return user.name directly.
suggestion: " return user.name"
```
Lens definitions are **trusted config**: they go into the system prompt, so only
define them in files you control (committed `.lgtmaybe.yml` or repo skill files),
never from PR-author content. See
[Add a custom review lens](add-a-custom-lens.md) for a full walk-through.
Default: none.
### lens_paths
Load `extra_lenses` from separate **skill files** instead of inlining them — handy
for sharing a lens across repos or wiring lgtmaybe into an agent harness. Each
entry is a YAML file (one lens, or a list of lenses) or a directory of `*.yml` /
`*.yaml` lens files. Paths are resolved relative to where lgtmaybe runs (your repo
root). Lenses loaded this way are appended to any inline `extra_lenses`.
```yaml
lens_paths:
- .lgtmaybe/skills # a directory of one-lens-per-file skill files
- team-lenses/house-style.yml # or a single file
```
Default: none.
### directory_rules
Scope extra review instructions and reference files to **part of** the repo —
what a monorepo needs when `payments/**` and `tests/**` deserve different
treatment. Each rule has `paths` (fnmatch globs; a `**/` prefix also matches at
the repo root, and an **empty or omitted** list applies the rule everywhere),
free-text `instructions`, and `context_files` read from the checked-out
workspace. A rule reaches every lens reviewing a file it matches, and no other.
```yaml
directory_rules:
- paths: ["payments/**", "billing/**"]
instructions: |
Money-handling code. Treat rounding, currency-conversion and retry
changes as high severity, and flag any write path that is not idempotent.
- paths: ["tests/**"]
instructions: |
Test code — do not flag duplication, long functions, or missing
docstrings here.
- paths: ["src/**"]
instructions: Check changes against the architecture described below.
context_files: [ARCHITECTURE.md]
```
Context files are redacted like the diff, share `max_input_tokens / 8`, and are
capped at five files per review; a missing or over-budget path is skipped
silently. Both the instructions and the context text are **trusted config** — on
`pull_request_target` they come from the base branch, never the PR head. See
[Scope review instructions to a directory](scope-review-instructions-to-a-directory.md).
Default: no rules.
## CLI flag overrides
Every config field can be overridden at the command line:
```bash
lgtmaybe review \
--provider anthropic \
--model claude-sonnet-4-6 \
--min-severity high
```
Flags take precedence over `.lgtmaybe.yml`.
---
# Add a custom review lens (BYO skills)
lgtmaybe ships nine built-in review lenses (security, correctness, deprecation,
tests, documentation, performance, complexity, intent, and `ponytail` — the
"lazy senior dev" / write-less-code lens). A **custom lens** lets you add your
own — a "skill file" that runs alongside the built-ins, fans out as its own
focused model call, and merges its findings into the same review. Use it to bake
in a house style or a team convention the built-in lenses don't cover.
## Contents
- [How a lens works](#how-a-lens-works)
- [Inline in `.lgtmaybe.yml`](#inline-in-lgtmaybeyml)
- [As reusable skill files](#as-reusable-skill-files)
- [Bundled lens packs (`pack:`)](#bundled-lens-packs-packname)
- [Run it](#run-it)
- [Security: lenses are trusted input](#security-lenses-are-trusted-input)
- [See also](#see-also)
## How a lens works
Every lens — built-in or custom — is the same shape: a focused instruction set and
one worked example, sent as its own system prompt so the model concentrates on a
single concern. A custom lens needs:
| Field | Required | What it is |
|---|---|---|
| `id` | yes | A unique short name. Must not collide with a built-in category. |
| `instructions` | yes | What to look for, in plain language. This is the lens. |
| `title` | no | Human-readable heading (falls back to `id`). |
| `example_diff` + `example_finding` | no (together) | A worked example: a hunk and the finding the model should return for it. Optional, but it sharply improves smaller models. |
## Inline in `.lgtmaybe.yml`
The quickest way — define the lens directly in your repo config:
```yaml
provider: ollama
model: qwen3.6:27b
extra_lenses:
- id: house-style
title: House style
instructions: |
Enforce our team conventions on changed code. Flag new public functions
that return bare dicts instead of a typed dataclass, and any logging at
WARNING or above that doesn't include a correlation id.
example_diff: |
--- a/api.py
+++ b/api.py
@@ -10,1 +10,2 @@
def make_user(name):
+ return {"name": name, "active": True}
example_finding:
path: api.py
line: 11
severity: low
title: Public function returns a bare dict
body: House style returns a typed dataclass from public functions, not a dict.
suggestion: " return User(name=name, active=True)"
```
!!! note "The Ponytail lens is built in"
The "lazy senior dev / write less code" instinct from
[Ponytail](https://github.com/DietrichGebert/ponytail) ships as the built-in
`ponytail` lens — you don't need a custom lens for it. Reach for `extra_lenses`
when you want something the nine built-ins don't already cover, like the
house-style rule above.
## As reusable skill files
To share lenses across repos — or to let an agent harness drop its own lens in —
put each one in its own file and point `lens_paths` at the file or a directory:
```yaml
# .lgtmaybe.yml
provider: ollama
model: qwen3.6:27b
lens_paths:
- .lgtmaybe/skills
```
```yaml
# .lgtmaybe/skills/house-style.yml
id: house-style
title: House style
instructions: |
Flag new public functions that return bare dicts instead of a typed dataclass,
and WARNING+ logging without a correlation id.
```
A skill file may hold one lens (a mapping) or several (a list). Lenses loaded from
`lens_paths` are appended to any inline `extra_lenses`; `id`s must be unique across
the whole set.
## Bundled lens packs (`pack:`)
lgtmaybe ships a curated, **opt-in** library of extra lenses — distilled from the
wider engineering-review ecosystem (Ousterhout, Metz, Carmack, the NASA Power of
Ten / TigerStyle, Google's eng-practices, and several skill/rule collections; see
[ATTRIBUTION.md](https://github.com/MattJColes/lgtmaybe/blob/main/ATTRIBUTION.md)).
They are **off by default** — they are more opinionated than the nine built-ins,
and every lens you enable is another model call per review. Enable a pack by name
with the `pack:` scheme in `lens_paths` (this works for a `pip install`, with no
repo-relative path to point at):
```yaml
# .lgtmaybe.yml
provider: ollama
model: qwen3.6:27b
lens_paths:
- pack:design # combine as many as you like
- pack:robustness
```
| Pack | Lenses | Use it when |
|---|---|---|
| `pack:design` | `wrong-abstraction`, `shallow-module`, `information-leakage`, `errors-out-of-existence`, `hidden-state`, `naming` | You want taste/structure review (the "right abstraction", deep modules, explicit state). |
| `pack:robustness` | `assertions`, `bounded`, `idempotency`, `migrations`, `portability`, `observability` | Operational safety: defensive invariants, bounded work, retry-safe side effects, safe migrations. |
| `pack:interface` | `api-design`, `type-safety`, `magic-values`, `comment-why` | Contracts & clarity: backward compatibility, tight types, named constants, why-not-what comments. |
| `pack:frontend` | `accessibility`, `i18n` | The diff touches UI / user-facing copy. |
Each pack is a directory of skill files under the package, so you can also browse
or copy any single lens from
[`src/lgtmaybe/lenses/`](https://github.com/MattJColes/lgtmaybe/tree/main/src/lgtmaybe/lenses).
An unknown `pack:` name fails loudly, listing the packs that exist.
## Run it
Custom lenses run automatically on every review — CLI or GitHub Action — once they
are in your config. On the CLI you'll see findings titled by your lens just like
the built-ins:
```bash
lgtmaybe review --provider ollama --model qwen3.6:27b --api-base http://localhost:11434
```
To narrow a run to fewer **built-in** lenses while keeping your custom ones, set
`categories` — the two lists are independent.
## Security: lenses are trusted input
A lens's `instructions` and example go straight into the model's system prompt, so
treat a lens like code you run: **only define lenses in files you control.** Keep
them in your committed `.lgtmaybe.yml` or repo skill files, never source them from
a pull request's contents. On `pull_request_target` the Action reads config from
the base repository, not the PR head, so a fork PR cannot introduce or alter a
lens. Diff content itself is always treated as untrusted data, separately from
your lenses.
## See also
- [Configure .lgtmaybe.yml](configure-lgtmaybe-yml.md#extra_lenses) — the field reference.
- [What gets reviewed](../explanation/what-gets-reviewed.md) — the built-in lenses.
---
# Scope review instructions to a directory
A monorepo is not uniform. `payments/**` deserves a strictness that would be
pure noise on `tests/**`, and reviewing `src/**` well may need a design document
the diff never shows. `directory_rules` scopes both **extra instructions** and
**reference files** to path globs, so one repo config can say different things
about different directories.
## Contents
- [The shape of a rule](#the-shape-of-a-rule)
- [Scope instructions to a directory](#scope-instructions-to-a-directory)
- [Give the reviewer background reading](#give-the-reviewer-background-reading)
- [How rules reach the model](#how-rules-reach-the-model)
- [Limits](#limits)
- [Security: rules are trusted configuration](#security-rules-are-trusted-configuration)
- [See also](#see-also)
## The shape of a rule
| Field | Required | What it is |
|---|---|---|
| `paths` | no | fnmatch globs against the repo-relative path. A `**/` prefix also matches at the repo root. **Omit it** (or leave it empty) and the rule applies everywhere. |
| `instructions` | no | Free text handed to every lens reviewing a file the rule matches. |
| `context_files` | no | Repo-relative files whose text is read from the checked-out workspace and included alongside the instructions. |
A rule needs at least one of `instructions` and `context_files` to do anything.
## Scope instructions to a directory
```yaml
provider: bedrock
model: bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0
directory_rules:
- paths: ["payments/**", "billing/**"]
instructions: |
This is money-handling code. Treat any rounding, currency-conversion or
retry change as high severity, and flag any write path that is not
idempotent under a duplicated webhook.
- paths: ["tests/**"]
instructions: |
Test code. Do not flag duplication, long functions, or missing
docstrings here — explicit and repetitive is the house style for tests.
Do still flag assertions that can never fail.
- paths: ["infra/**/*.tf"]
instructions: |
Terraform. Flag any IAM policy with a `*` action or resource, and any
storage bucket or database that becomes publicly reachable.
```
Rules are independent, not exclusive: a review batch touching both
`payments/charge.py` and `tests/test_charge.py` gets both rules, in the order
you wrote them.
## Give the reviewer background reading
`context_files` puts a document in front of the model that the diff would never
show it — the architecture the code is supposed to follow, a data-model
invariant, a deprecation plan:
```yaml
directory_rules:
- paths: ["src/**"]
instructions: |
Check changes against the architecture described below. Flag a new
import that crosses a boundary the document forbids.
context_files:
- ARCHITECTURE.md
- docs/data-model.md
```
The files are read from the workspace lgtmaybe is running in — the same place
your `.lgtmaybe.yml` comes from. They are redacted for secrets before they leave
the process, exactly like the diff.
## How rules reach the model
The matched rules are rendered into a single block that leads each review call,
labelled as trusted configuration so the model never confuses it with the
(untrusted) diff below it. The block joins the same cacheable prefix as the diff
itself, so on providers with prompt caching you pay for it roughly once per
batch rather than once per lens.
Instructions go in verbatim. Context-file text is defanged first, so a document
that happens to quote one of lgtmaybe's own prompt delimiters cannot break out
of its block.
## Limits
- **Instructions and context only.** A rule cannot change `min_severity`,
`categories`, or any other setting for a directory. If you want a lens that
runs everywhere with its own worked example, that is
[a custom lens](add-a-custom-lens.md).
- **The context budget is a slice, not the whole thing.** Context files share
`max_input_tokens / 8` and are capped at five files per review; the diff
always dominates the call. A file that does not fit is skipped.
- **A missing path is skipped silently.** A context file is an aid, and a stale
path in your config must never fail a review. Check `--profile` or the run log
if a document does not seem to be reaching the model.
- **YAML only.** Like `finding_rules` and `extra_lenses`, `directory_rules` is a
list of objects, so it has no CLI flag and no Action input. Put it in
`.lgtmaybe.yml`.
## Security: rules are trusted configuration
Both the instructions and the context text enter the prompt as **trusted**
content, so they must never come from a pull request's author.
They do not. On the recommended `pull_request_target` trigger, the workflow
checks out the **base** branch — the code already merged — and never the PR
head. `directory_rules` and the files it names are read from that checkout, the
same source `.lgtmaybe.yml` and `lens_paths` already come from. lgtmaybe never
fetches context files through the host's API, which would resolve them at the
(untrusted) PR head. The same holds on GitLab and Gitea: context comes from the
checked-out workspace, never from the change's head.
The practical rule: a fork PR can change `ARCHITECTURE.md` in its own branch and
lgtmaybe will still read the base branch's copy. Review changes to your config
and context files the way you review any other change to CI.
## See also
- [Configure `.lgtmaybe.yml`](configure-lgtmaybe-yml.md) — every field
- [Add a custom review lens](add-a-custom-lens.md) — a whole extra lens, repo-wide
- [Configuration reference](../reference/config.md) — generated schema
---
# Generate a change overview
lgtmaybe can post a **change overview** on a pull request: what the change is,
what is risky about it, and what it touches. It is one comment, updated in
place, that a reviewer reads before the diff. A separate concern from the
review — the review finds bugs in the lines it reads, the overview describes the
change as a whole.
## Contents
- [What you get](#what-you-get)
- [High Impact Areas](#high-impact-areas)
- [Structure and sequence](#structure-and-sequence)
- [On GitHub: `/diagram` and `auto_diagram`](#on-github-diagram-and-auto_diagram)
- [Locally: `lgtmaybe diagram`](#locally-lgtmaybe-diagram)
- [Why Mermaid (and what the ASCII is for)](#why-mermaid-and-what-the-ascii-is-for)
- [See also](#see-also)
## What you get
Three focused model calls, run concurrently, plus one deterministic section,
compose one comment in reading order:
1. a **description** — a suggested title, the change type, a short summary, a
per-file walkthrough, and a "does it do what it says" check when the PR
states an intent (`auto_describe`);
2. **Risk of Change** — the change's `low` to `critical` verdict and every
reason behind it, computed without a model call (`risk`; see
[Risk of Change](../explanation/risk-of-change.md));
3. **[High Impact Areas](#high-impact-areas)** — the changes a reviewer must not
miss, in a bold-titled section (`high_impact`);
4. up to two **diagrams**, each in two renderings — a **Mermaid flowchart** of
the structure, a **Mermaid sequence diagram** of the run-time flow when the
change alters one, and a **plain-text rendering** of each, which is what
shows in a terminal and serves as the fallback if the Mermaid can't be
rendered.
Three calls rather than one because a single prompt doing three jobs does all
three worse. They run at the same time, so the overview costs about as much
wall-clock as its slowest call rather than the sum of the three. Each section
has its own switch, and the description and High Impact Areas sections are
best-effort: if one of those calls fails, its slot says so and the rest of the
comment still posts.
GitHub renders both Mermaid diagrams natively inside the comment — no image, no
external hosting.
The flowchart is intentionally sparse: it uses at most six nodes, short
relationship labels, and Mermaid's automatic layout. Changed elements are
marked in their labels (`(new)` / `(changed)`), keeping labels and arrows clear
of neighbouring cards. Because GitHub's renderer offers zoom buttons but no
full-screen, the diagram is followed by an **⛶ Open full screen** link that
renders the same diagram alone in a browser tab, with pan and zoom, on
[mermaid.live](https://mermaid.live). The source travels compressed in the URL
fragment, decoded in your browser, so nothing is sent anywhere until you click.
The diagram also stays honest about the diff being only a slice of the
codebase: relationships inferred from imports rather than shown in the diff
are called out in the notes.
Here is what the posted comment looks like for a PR that puts a Redis cache in
front of a user service — the Mermaid renders in place, with the ASCII tucked
in a collapsible "Text version" underneath:
> ## Cache user lookups in Redis
> **Change type:** feature
> User reads now check Redis before PostgreSQL, and successful database reads
> populate the cache for later requests.
> ### **Risk of Change: Critical**
>
> - touches infrastructure: `infra/redis.tf`, `ops/backup_retention.tf`
> - touches backup and recovery: `ops/backup_retention.tf`
> ### **High Impact Areas**
>
> - **Infrastructure** — New ElastiCache cluster (`infra/redis.tf`). Adds a
> managed dependency to the production VPC. _Check:_ security group and
> failover settings.
> - **Availability** — Cache on the user read path (`api/users.py`). User reads
> now depend on Redis, so a Redis outage takes reads down with it. _Check:_
> confirm a Redis error falls back to the database.
> - **Backup and recovery** — touched: `ops/backup_retention.tf` (not assessed
> by the model)
> ### Structure
```mermaid
flowchart LR
client["Client"]
api["User API]
Python
serves user reads"]
cache["Redis cache
Redis
caches rows (new)"]
db["User DB
PostgreSQL"]
client -->|requests| api
api -->|checks| cache
api -->|on miss| db
```
> [⛶ Open full screen](https://mermaid.live) *(the real link carries the
> diagram source compressed into the URL)*
Text version
```
[Client] --> [User API] --check--> [Redis cache] (new)
|
+--miss--> [User DB]
```
> ### Sequence
```mermaid
sequenceDiagram
participant client as Client
participant api as User API (changed)
participant cache as Redis cache (new)
participant db as User DB
client->>api: GET /users/{id}
api->>cache: read cached row
cache-->>api: miss
api->>db: SELECT user
api->>cache: store row, 60s TTL
api-->>client: user JSON
```
Text version
```
1. [Client] -> [User API (changed)]: GET /users/{id}
2. [User API (changed)] -> [Redis cache (new)]: read cached row
3. [Redis cache (new)] --> [User API (changed)]: miss
4. [User API (changed)] -> [User DB]: SELECT user
5. [User API (changed)] -> [Redis cache (new)]: store row, 60s TTL
6. [User API (changed)] --> [Client]: user JSON
```
> *The User DB link is inferred from an import, not shown in the diff.*
## High Impact Areas
A review reads the changed lines and tells you what is wrong in them. It cannot
tell you that halving a node count removes your peak headroom, or that the PR
quietly shortened a backup retention policy. **High Impact Areas** asks the
other question — *what could this change break beyond itself?* — and answers it
in one bold-titled section at the top of the overview.
Ten areas are checked:
| Area | What it covers |
|---|---|
| **Infrastructure** | IaC, Kubernetes/Helm, container images, CI/CD pipelines, deploy scripts, networking, DNS, load balancers, resource sizing |
| **Security** | Authentication, authorisation, IAM or permission scope, cryptography, secret handling, removed input validation, CORS/CSP/TLS, CI workflow permissions |
| **Availability** | Anything that could cause a production outage: startup and config defaults, timeouts, retries, circuit breakers, connection pools, rate limits, health checks, removed error handling, concurrency and locking, runtime or major dependency upgrades, hot-path performance, rollback and deploy ordering |
| **Data migration** | Schema or data migrations, destructive or irreversible data operations, backfills |
| **Backup and recovery** | Backup jobs, retention and lifecycle policies, snapshots, restore paths, disaster recovery, failover, replication |
| **Compatibility** | Breaking an external contract: HTTP APIs, event or message schemas, CLI flags, SDK surface, wire formats, feature-flag removal |
| **Observability** | Logs, metrics, alerts, dashboards, tracing, audit trails or runbooks being removed, silenced or changed |
| **Dependencies** | Supply chain: new dependencies, major bumps, loosened pins, lockfiles, base images, build toolchain |
| **Cost** | Autoscaling limits, instance types, provisioned capacity, storage classes, quotas |
| **Compliance** | PII or privacy handling, audit trails, data residency, licence changes |
Each call-out names the area, a short headline, the files it comes from, the
blast radius, and the one thing a reviewer should verify.
### The path floor
The model is not the only source. Before the call, lgtmaybe matches the PR's
changed **paths** against deterministic patterns per area — `*.tf`,
`.github/workflows/`, `alembic/`, anything named for backups or retention,
lockfiles, and so on. Those matches do two jobs:
- they are sent to the model as untrusted hints, saying *where* to look (never
that anything is wrong);
- they **floor** the output. An area whose files the model said nothing about is
still listed, marked `(not assessed by the model)`, so a weak model cannot
make your Terraform change invisible.
The floor is also the fallback: if the call fails or returns nothing usable, the
section renders the path signals alone under a "model assessment unavailable"
note. It degrades — it never disappears.
When nothing qualifies, the section says so and names what it checked, rather
than silently vanishing:
> ### **High Impact Areas**
>
> None detected — checked: infrastructure, security posture, availability, data
> migrations, backups and recovery, compatibility, observability, dependencies,
> cost, compliance.
### Turning it off
The section is on by default. To drop it and its model call:
```yaml
high_impact: false
```
The description section has its own switch, `auto_describe: false`. With both
off, the comment is just the diagrams, and costs one call.
## Structure and sequence
The two diagrams answer different questions, which is why lgtmaybe draws both:
| | Answers | Best on |
|---|---|---|
| **Flowchart** (structure) | What does this change touch, and what talks to what? | Structural PRs — a component added, a dependency introduced, a service split |
| **Sequence** | What happens at run time, in what order, and where did that change? | Behavioural PRs — a retry added, an ordering fixed, a signal now handled |
Most pull requests are behavioural, and structure alone under-describes them: a
PR that fixes terminal restoration on `SIGTERM` might touch two files and draw
three boxes, while its sequence diagram — signal, flag set, loop exit, guard
dropped, terminal restored — *is* the explanation of the fix. A PR that puts a
cache in front of a service is the opposite: the new box is the headline, and
the flow shows how a request now reaches it.
The sequence view is **omitted entirely** when the change has no meaningful
run-time flow — documentation, configuration, formatting — rather than inventing
one. Both diagrams share one model call, one PR comment, and the same
six-component budget; the sequence view adds at most eight ordered steps.
## On GitHub: `/diagram` and `auto_diagram`
Comment **`/diagram`** on a pull request to post (or update in place) the change
overview. It's an idempotent upsert — re-running edits the same comment instead
of stacking new ones. (`/describe` still posts a description as its own separate
comment, for when that is all you want.)
`auto_diagram` is **on by default** — no workflow input or `.lgtmaybe.yml`
needed — so the overview posts automatically when a PR is opened or reopened and
refreshes after later pushes, keeping every section current with the head
commit. To opt out of the whole comment, set it in your workflow:
```yaml
- uses: MattJColes/lgtmaybe@v2
with:
provider: anthropic
model: claude-sonnet-4-6
auto_diagram: "false"
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
or in `.lgtmaybe.yml`:
```yaml
auto_diagram: false
```
The automatic overview stands back when the pull request already diagrams the
change: a fenced `mermaid` block in the PR description, or in any comment
lgtmaybe didn't post itself (yours, a teammate's, or another review bot's
walkthrough).
The review still runs and its summary names where the existing diagram lives.
Once lgtmaybe has posted its own overview on a PR it keeps refreshing that one,
so it never goes stale. Comment `/diagram` to post the overview anyway — slash
commands skip the check. Gitea runs the same check through the shared Action
entrypoint. (The `gitlab-ci` command posts no automatic overview yet.)
The description and High Impact Areas sections are best-effort: a failure there
is logged, leaves a visible note in its slot, and never blocks the rest. The
diagram itself is a required completion step, so a failed diagram leaves the
commit un-finished and the next run redoes it rather than silently skipping it.
## Locally: `lgtmaybe diagram`
`lgtmaybe diagram` prints the change overview of your local changes — no GitHub
involved:
```console
$ lgtmaybe diagram --provider ollama --model llama3
```
It diffs your branch against the base (the same base resolution as `lgtmaybe
review`; `--base` overrides, `--working` includes uncommitted edits,
`--uncommitted` reviews only the working-tree edits). The output is the same
body the `/diagram` comment carries — description, Risk of Change, High Impact
Areas, then each
diagram's Mermaid source and its text rendering — with one adaptation: a
terminal renders no HTML, so the collapsible "Text version" wrapper is flattened
into a plain labelled section:
````console
$ lgtmaybe diagram --provider ollama --model llama3
### Sequence
```mermaid
sequenceDiagram
participant n0 as Order API (changed)
participant n1 as Payments
n0->>n1: POST /charge
n1-->>n0: 503 Unavailable
n0->>n0: backs off 200ms
```
Text version:
```
1. [Order API (changed)] -> [Payments]: POST /charge
2. [Payments] --> [Order API (changed)]: 503 Unavailable
3. [Order API (changed)] -> [Order API (changed)]: backs off 200ms
```
````
The Mermaid source stays in the output on purpose: your terminal can't draw it,
but it is what you paste into a GitHub comment,
[mermaid.live](https://mermaid.live), or a Markdown file to render it.
## Why Mermaid (and what the ASCII is for)
GitHub renders **Mermaid** natively in comments and Markdown — both flowcharts
and sequence diagrams — so a `mermaid` fence renders in the comment with no
image to generate or host. That matters for a `pull_request_target` reviewer:
hosting an image would mean committing a file or calling an external service,
neither of which fits a fork-safe, idempotently-updated comment.
A terminal, though, can't render Mermaid — which is exactly why each diagram
also comes as **plain text**. In the GitHub comment that text sits in a
collapsible "Text version" block, out of the way of the rendered diagram; in the
terminal the same block is flattened to a labelled section, because HTML tags
are noise in a shell. Either way the text is what you actually read without a
renderer, and it doubles as the fallback body, so a reviewer never sees a red
"unable to render" box if a diagram comes back malformed.
The model never writes Mermaid. It returns typed graph data — components,
relationships, ordered steps — and lgtmaybe renders the syntax itself, escaping
every label, so a diff that tries to smuggle diagram source or markup into a
label can't reach the fence.
D2 isn't used because GitHub doesn't render it in Markdown, so it would show as
source anyway.
## See also
- [Configure `.lgtmaybe.yml`](configure-lgtmaybe-yml.md)
- [Use as a GitHub Action](use-as-github-action.md)
- [Configuration reference](../reference/config.md)
---
# Fix findings with an AI agent
`lgtmaybe review` runs locally and prints findings; it never posts anywhere. The
`--format agent` output turns those findings into plain correction instructions
an AI coding agent (such as Claude Code) can read and apply, so you get a
review-and-fix loop on your own machine before you ever open a pull request.
This works with any provider. ollama keeps it local and free; a cloud provider
gives you a stronger reviewer at a small cost.
## Contents
- [Print the findings as agent instructions](#print-the-findings-as-agent-instructions)
- [Hand it to the agent](#hand-it-to-the-agent)
- [Close the loop](#close-the-loop)
- [See also](#see-also)
## Print the findings as agent instructions
From inside the repo, on the branch you want reviewed:
```bash
lgtmaybe review \
--provider ollama \
--model qwen3.6:27b \
--api-base http://localhost:11434 \
--format agent
```
The output is directive rather than a bare listing:
```text
Code review findings for your local changes. Act as the developer and apply each
correction below: open the file at the given path and line, fix the issue, and
apply the suggested change where one is given.
[1] src/app.py:42 (HIGH) possible NPE
Issue: `user` may be None here.
Suggested fix:
if user is not None:
do_thing(user)
1 finding(s) to address. After applying the fixes, re-run `lgtmaybe review` to
confirm they are resolved.
```
## Hand it to the agent
Save the instructions and point your agent at them:
```bash
lgtmaybe review --provider ollama --model qwen3.6:27b \
--api-base http://localhost:11434 --format agent > review.txt
```
Then ask the agent to work through `review.txt` — for example, in Claude Code:
> Apply the corrections in review.txt, then delete it.
Because each finding carries a path, a line, the issue, and (often) a suggested
replacement, the agent has everything it needs to make the edit without guessing.
## Close the loop
Once the agent has applied the fixes, run the review again to confirm the
findings are gone:
```bash
lgtmaybe review --provider ollama --model qwen3.6:27b \
--api-base http://localhost:11434 --format agent
```
A clean branch prints `No review findings — nothing to correct.` followed by
the summary line. Repeat until
you are happy, then open your PR. To post reviews on the PR itself, wire up the
[GitHub Action](use-as-github-action.md).
## See also
- [Run locally with ollama](run-locally-with-ollama.md) — the local CLI setup
- [What gets reviewed](../explanation/what-gets-reviewed.md) — scope, caps, and output formats
---
# Reduce Review Cost
A review can make several model calls per diff batch. Each call has an input
cost, even when prompt caching reduces the bill. Large diffs create more
batches, so cost can rise quickly on a hosted provider.
This guide shows you how to measure that cost and lower it while keeping the
findings you need.
> Deciding **who** can trigger a review is a related but separate question —
> see [Trust and Cost](../explanation/trust-and-cost.md). Every setting below is
> documented field-by-field in the [configuration reference](../reference/config.md).
## Contents
- [First, measure](#first-measure)
- [Where the tokens go](#where-the-tokens-go)
- [The levers, in order of payoff](#the-levers-in-order-of-payoff)
- [Put a hard ceiling on it](#put-a-hard-ceiling-on-it)
- [If a call runs past `max_tokens`](#if-a-call-runs-past-max_tokens)
- [If one lens floods the review](#if-one-lens-floods-the-review)
- [What costs more, on purpose](#what-costs-more-on-purpose)
- [What isn't worth changing](#what-isnt-worth-changing)
## First, measure
Do not tune blind. Run once with `--profile`:
```bash
lgtmaybe review --profile
```
The last line is the meter:
```
tokens: 158,076 billable (154,200 in / 3,876 out) across 12 calls
```
Every local review prints that same line to stderr even without `--profile`, so
the meter is always in view; redirect with `2>/dev/null` if you want it gone.
When litellm's pricing map knows the model, the meter gains a second line:
```
estimated cost: $0.5207 (litellm pricing map)
```
It is an estimate off a price list that can lag a brand-new model, so its
absence means "unpriced", not "free" — and when only some calls could be
priced, the line says how many rather than passing a partial total off as the
sum. Cache reads are billed at their discount, so a run with a warm prefix
shows it.
`in` can dwarf `out` because each review call includes diff content. The
per-call table shows which lens and batch each call belongs to, so you can see
whether the cost comes from call count, batch count, or one large file.
On a reasoning model, read the table's `think_tok` column next to `out_tok`.
That is how much of the output budget went on thought before the model wrote a
single finding, and a summary line totals it:
```
reasoning: 41,300 of 47,900 output tokens (86%)
```
The line appears only when a route reports the breakdown — its absence means
"not reported", not "no thinking". When the share is high, it explains a lens
that ran for a minute and returned three findings, and it is the number to
check before [raising `max_tokens`](#if-a-call-runs-past-max_tokens).
In the GitHub Action, set `profile: true` and the same breakdown lands in the
job log. Every provider call also emits a structured `provider call` log line
with its own token counts, so you can total a run without the summary.
## Where the tokens go
For one review:
```
input tokens ≈ batches × calls per batch × (diff + context padding + hints)
```
- **calls per batch** — the `fast` preset (the default) makes **four** core
calls per batch; `full` makes one per built-in category, up to nine. A
matching committed spec adds its own call. This is the biggest multiplier
in the formula.
- **batches** — a diff larger than `max_input_tokens` is split, and *each*
batch pays the full lens fan-out again. The default is 100k, fitted down to
what the model's context window can take when that is known (litellm's model
map for hosted models, `num_ctx` for ollama, less the output ceiling); a value
you set is used as-is.
- **reflection** — one more pass over the findings, on top.
- **triage** — an extra cheap call when enabled.
- **the change overview** (`auto_diagram`, on by default) — up to three extra
calls per push: the description (`auto_describe`), High Impact Areas
(`high_impact`), and the diagrams. Each opts out on its own; `auto_diagram:
false` drops all three.
- **mid-review retrieval** — off by default; on, it adds up to one more call per
(batch, lens). See [What costs more, on purpose](#what-costs-more-on-purpose).
Two things that look expensive but are not: `max_concurrency` only changes how
many calls run at once, never how many there are, and `temperature` costs
nothing.
## The levers, in order of payoff
### 1. Turn on triage
A cheap model reads every file first and drops the ones that plainly do not
need a review; the expensive model only sees the survivors, riskiest first.
```yaml
triage_model: claude-haiku-4-5
```
This is usually the single biggest win on a repo with lots of low-substance
churn (generated files, config bumps, test fixtures). A deterministic security
floor always escalates past triage, any triage failure reviews everything, and
skipped files are named in the summary — so it cannot quietly hide a problem.
All three model slots share one provider and one set of credentials.
### 2. Let deterministic tools do the deterministic work
Findings a linter can prove do not need a language model at all:
```bash
pip install 'lgtmaybe[static-analysis]'
```
```yaml
static_analysis:
enabled: true
```
Installed tools (ruff, bandit, mypy, semgrep with local rules) run over the
changed files and their findings are fed to the lenses as hints to confirm or
discard. The tools cost no tokens, and grounding the model tends to reduce
low-value findings as well as spend.
### 3. Check prompt caching is actually working
Prompt caching is **always enabled**, and when it works it takes most of the
sting out of the lens multiplier: lenses 2..N read the shared system-preamble
plus diff prefix from cache instead of paying full input price for it.
Verify it from the profile's cache line:
```
cache: 412000 tokens read / 38000 created across 12 calls
```
Reads climbing across a run means it is working. **Reads stuck near zero means
you are paying full price for every lens.** Caching needs an explicit
breakpoint on some routes (anthropic, bedrock Claude/Nova, vertex, and several
openrouter models) and identical prefixes on the rest — a route without either
gets no benefit, which is a strong argument for picking one that has it.
### 4. Narrow the lenses
Each lens you drop removes one call per batch:
```yaml
categories: [security, correctness]
```
Keep the default `fast` preset unless you have a reason not to; `preset: full`
roughly doubles the call count for a modest recall gain.
### 5. Review less
```yaml
exclude_paths: ["**/migrations/**", "**/*.generated.ts"]
max_files: 25
```
Generated, vendored and binary files are already skipped. Path filters cut the
rest before any model call is made, so they are pure saving.
### 6. Let re-runs be incremental
On the Action's `synchronize` event, a re-run reviews only the commits since the
last reviewed SHA rather than the whole PR again. That is the default — leave
`incremental` unset. Use `/review full` when you deliberately want everything
re-reviewed.
Locally there is no such watermark: **every `lgtmaybe review` re-reviews the
whole branch at full price.** If you are iterating, review only what you just
changed:
```bash
lgtmaybe review --uncommitted # working-tree edits vs HEAD
```
### 7. Right-size the model
A smaller model on the common path with a stronger fallback cuts expected cost
when the primary usually finishes its calls. A healthy call never reaches the
fallback. A failed call still spends the primary model's attempts before adding
the second model, so this does not reduce worst-case cost.
```yaml
provider: openrouter
model: google/gemini-3.7-flash
fallback_model: qwen/qwen3.8-max
```
Both models use the same configured provider and credentials. See
[`fallback_model`](configure-lgtmaybe-yml.md#fallback_model) for the recovery
order, failure behavior, and reporting.
If cost is the binding constraint rather than latency, ollama runs locally:
```bash
lgtmaybe review --provider ollama --model qwen2.5-coder:14b
```
## Put a hard ceiling on it
Tuning changes the expected cost; a ceiling bounds the worst case.
```yaml
max_review_tokens: 300000
```
Once a run's billable tokens (input + output across every model call) reach the
ceiling, no further calls are dispatched. In-flight calls finish, their findings
post, and the summary says plainly that the budget stopped the run and how many
calls were skipped. It never turns a failure into a silent `LGTM`.
It is **off by default on purpose**: spend scales with diff size, lens count and
batch count, so any figure that protects a small repo would silently truncate a
large one's review. Read a real run's total with `--profile` first, then set the
ceiling comfortably above it — it is a runaway guard, not a tuning knob.
## If a call runs past `max_tokens`
`max_tokens` caps what one call may *write*. A lens that hits it comes back cut
off mid-JSON, and lowering it to save money is the usual way to arrive here.
On `ollama`, `openai-compatible` and `openrouter` a ceiling of **16384 tokens per
call** applies by default, so you can arrive here without setting anything. It is
there because a model under structured output can fail to terminate and decode
until the timeout stops it half an hour later. The first-party APIs send no
default ceiling.
You do not have to do anything. lgtmaybe treats an over-ceiling call the same
way it treats one that outruns its wall clock: the batch was more than one call
could cover, so it is **halved and the pieces reviewed separately**, each with a
fresh ceiling. Whatever the model finished writing before the cut is kept as
well, so nothing already paid for is thrown away. The summary says a batch was
shrunk, and — because part of a lens is not a whole lens — the run still reports
that results may be incomplete.
The cost of that recovery is up to twice the calls for the affected batch,
though the pieces are reviewed **concurrently**, so on the clock it costs about
one extra call. It is bounded to one split level: if a piece still runs past the
ceiling, it is reported rather than split again.
When you see it happening often, the fix is usually **`max_tokens`, not the
diff**:
```yaml
max_tokens: 32768
```
A reasoning model spends the same budget on thinking before it writes a single
finding, which is how a fifteen-line diff can truncate. The failure names the
reasoning tokens where the provider reports them — and when they account for
essentially the whole ceiling, lgtmaybe **does not split the batch at all**:
smaller pieces would each re-spend the same ceiling on thought and truncate the
same way, so it says so and names `reasoning_effort` instead of costing you the
extra calls.
You do not have to wait for a truncation to find that out. `--profile` reports
the same split on calls that **succeeded** (`think_tok`, and the `reasoning:`
total), which is the comparison a truncated call cannot give you: a call that
hit the ceiling spent reasoning + findings ≥ `max_tokens` by definition, so it
tells you nothing about the healthy calls sitting beside it.
Two more numbers answer the headroom question directly, so you do not have to
work the ratio out per row:
- the **`think_%`** column — that call's thinking as a share of the `max_tokens`
ceiling it came out of;
- a **`largest reasoning share:`** line under the table, naming the single
closest call to the ceiling and which lens it was.
An 8,192-token cap whose worst lens reports 50% has real headroom. One reporting
95% is a truncation that has not happened yet — and the aggregate `reasoning:`
line will not show you that, because one runaway hides inside an average.
A route that reports no breakdown renders as `-` in both columns and prints no
`largest` line. That is not zero: it means the route never said, and there is
nothing to conclude either way.
### When the cap is not the lever
If `think_tok` is most of `out_tok`, raising `max_tokens` will not fix the
truncation — the reasoning simply expands into the new headroom. Bound the
thinking instead:
```yaml
reasoning_effort: low # none | minimal | low | medium | high | xhigh | default
```
This is measured, not theoretical. On lgtmaybe's own dogfood review, 5 of 9
lens calls spent **32,000–35,000 reasoning tokens** — at or above the entire
32,768 `max_tokens` ceiling — before writing a single finding. The one large
call that did complete wrote roughly 733 tokens of findings after 28,909 tokens
of thought. Raising the cap from 16k to 32k had not helped; it only bought the
model more room to think.
Unset sends nothing, so a route without a reasoning channel is unaffected.
Some models treat this as a switch rather than a dial. GLM 5.3 through
OpenRouter spent a 12,288-token ceiling on thought at `low` and again at
`minimal`, in 10 retries of 11; only `none` moved it. lgtmaybe's ladder now
takes that step for you — one graded step down, then reasoning off, then the
fallback model — but on such a model `none` as the *configured* value saves the
two failed calls that precede it.
#### But do not read "cheaper" as "better value"
Bounding thinking is the fix for a **reasoning-bound truncation**. It is not a
free saving, and the direction that saves tokens is not the direction that finds
bugs. Measured on lgtmaybe's own repo, two runs per setting on the same diff:
| | `low` | `medium` |
|---|---|---|
| findings | 2, 2 | 5, 4 |
| false positives | 0, 1 | 0, 0 |
| output tokens | ~4–5k | ~14k |
| **total billable** | 180,704 | 187,191 |
`medium` roughly triples output tokens and still costs about **4% more in total**,
because a review is overwhelmingly input: the diff, the context, the system
prompt. Weighting output at 4× its input price puts it nearer 20%. Either way,
nothing like the 3× the output column suggests.
What it bought was a bug in lgtmaybe's own retry backoff that `low` missed in
both its runs — and `low` produced the only false positive of the four. So treat
`reasoning_effort` as a **quality** dial that happens to move cost a little, not a
cost dial that happens to move quality. Turn it down when reasoning is eating your
`max_tokens` ceiling; do not turn it down to save money.
#### On OpenRouter, check the log line
litellm forwards `reasoning_effort` to OpenRouter only for models its capability
map already flags reasoning-capable, and the newest models — the ones a
reasoning budget is usually set for — are not in it. lgtmaybe sends the budget
in OpenRouter's own top-level `reasoning` object for those, so the setting is
enforced either way.
More generally, any configured param the resolved model will not accept is named
once at startup rather than discarded in silence:
```
configured params are not supported by this model and will be ignored
```
If you see that line naming a setting you meant to apply, the setting is not in
force — pick a different model, or drop the setting. `reasoning_effort: default`
on OpenRouter is one real case: it is in litellm's vocabulary but not in
OpenRouter's `reasoning.effort` enum (`none`, `minimal`, `low`, `medium`,
`high`, `xhigh`), so it is reported rather than quietly turned into a nearby
level.
## If one lens floods the review
A lens can stay within its token budget and still return far too many findings,
restating one claim against every line it can see. Measured on a benchmark diff
with nothing wrong in it, a single lens returned 319 of a review's 323 findings,
each on a different line. Location dedupe does not collapse them, because no two
findings share a line.
One `(batch, lens)` call therefore contributes at most **50 findings** by
default. When the bound fires, the highest-severity findings are kept and the
summary names the lens and how many were dropped:
```
⚠️ Bounded a lens to the top 50 findings by severity: `intent` (269 dropped).
```
The notice usually indicates a problem with the model rather than a bound set too
low. A lens returning hundreds of findings is generating badly, and a higher
bound admits more of the same output. Raise it only when a genuinely large diff
is losing real findings:
```yaml
max_findings_per_lens: 100 # 0 disables the bound entirely
```
An ordinary lens returns a handful of findings and never reaches the bound, so a
healthy run is unaffected.
## What costs more, on purpose
One setting in this guide runs the other way: **`mid_review_retrieval`** buys
findings with tokens rather than the reverse. Be clear-eyed about the price
before you turn it on.
```yaml
mid_review_retrieval: true # default false
```
Normally a lens that cannot decide without reading code outside the diff is told
to hedge the claim or drop it — so a bug whose evidence sits one file away is
never reported. With this on, the lens may instead name the files (or symbols) it
needs; lgtmaybe fetches them read-only and re-runs *that one lens* with them.
The cost, worst case: **one extra model call per (batch, lens)**, each carrying
the fetched file text as well. On the default four-lens preset over three
batches, that is up to twelve extra calls — the multiplier in the formula above,
doubled. In practice only the lenses that actually defer pay it, and the fetch is
capped at five files inside a quarter of `max_input_tokens`, but budget for the
worst case rather than the average.
Ways to keep the bill honest if you want it:
- pair it with `max_review_tokens`, so the worst case is bounded rather than
merely unlikely — a deferral arriving past the ceiling is skipped and reported;
- fetched text rides the lens's own block, so a deferral never invalidates the
shared prefix; on cache-capable routes, sibling lenses can reuse that prefix;
- turn on `triage_model` first, so fewer files reach the lenses that might defer.
Measure it rather than assume: run `python -m evals.run` with and without
`--mid-review-retrieval` against your model and compare recall to the token
total.
## What isn't worth changing
**`context_lines`.** The obvious-looking saving is not there. Measured across
four real multi-file commits in this repository, the padding that surrounds each
hunk costs about **41% over sending no context at all** — but only about **17%
between `context_lines: 5` and the default `20`**, because hunks with
overlapping windows merge and the enclosing-definition reach does most of the
useful widening either way:
| `context_lines` | 0 | 5 | 10 | 20 (default) |
| --- | --- | --- | --- | --- |
| diff payload tokens | 28,316 | 34,265 | 36,072 | 40,041 |
| vs. no context | 1.00× | 1.21× | 1.27× | 1.41× |
Trading roughly a seventh of your input tokens for the model no longer seeing
the function a change sits in is a bad deal — that context is what keeps
findings anchored and cuts false positives. Turn the multiplier down (fewer
lenses, triage, caching) before you turn the context down.
---
# Releasing lgtmaybe (maintainers)
Releases are automated by **release-please** (`.github/workflows/release-please.yml`).
It reads the **conventional commits** merged to `main` and keeps a "Release PR"
open that bumps the version and regenerates `CHANGELOG.md`. **Merging that PR** is
the release: it cuts the tag and the GitHub release, then the same run publishes —
**PyPI** via trusted publishing (OIDC) and the **GHCR image** + floating major
tag (`v{major}`, currently `v2`) via the reusable `.github/workflows/release.yml`
(built-in `GITHUB_TOKEN`). No publish tokens live in secrets.
A third workflow, `.github/workflows/homebrew.yml`, regenerates the **Homebrew
formula** in the tap repo (`MattJColes/homebrew-tap`, i.e. the tap
`MattJColes/tap`) so `brew install MattJColes/tap/lgtmaybe` tracks the latest
version.
`scripts/update-homebrew-formula.sh` writes a small formula that creates a venv
and `pip install`s lgtmaybe + its dependencies from **PyPI wheels**, with no
per-dependency `resource` stanzas. litellm's tree includes Rust sdists
(tokenizers, hf-xet) that can't build in Homebrew's sandbox, so building from
source is a dead end — the wheels work. The formula declares **`preserve_rpath`**
so Homebrew keeps the wheels' `@rpath` extension-dylib ids instead of failing to
rewrite them ("Failed to fix install linkage"). It's a plain source formula —
no bottle — so it installs on any architecture and macOS version.
The release run also calls `.github/workflows/windows-exe.yml` to build and
smoke-test a portable Windows executable, then `.github/workflows/winget.yml`
to submit the new asset to winget. The executable must exist before the winget
job starts; both workflows are manually dispatchable for recovery.
The workflow:
- Is **called** by `release-please.yml` (`workflow_call`) right after a release,
because a `release: published` event is **not** delivered for a release
release-please cuts with the built-in `GITHUB_TOKEN` (GitHub suppresses
downstream triggers from `GITHUB_TOKEN` to prevent recursion). A daily
`schedule` and a manual `workflow_dispatch` (with an optional `force` to
re-publish the same version) are the safety nets.
- **Installs the regenerated formula in CI before committing it** — a real
`brew trust` + `brew install` of the formula gates the push, so a formula that
doesn't install is never published. The gate trusts the tap rather than setting
`HOMEBREW_NO_REQUIRE_TAP_TRUST`, so it exercises the same two steps the install
guide gives users. (You can seed/verify it by hand on a Mac by running
`scripts/update-homebrew-formula.sh path/to/Formula/lgtmaybe.rb`.)
Net effect: a new version lands in the tap within minutes of release — no PyPI
cooldown to wait out, since `brew update-python-resources` (which imposes one)
isn't used.
Commit messages must follow conventional-commit format — `.github/workflows/commitlint.yml`
enforces it on PRs so release-please can compute the next version.
The only human-only pieces:
## Contents
- [One-time setup](#one-time-setup)
- [Each release](#each-release)
- [Rotate the public App private key](#rotate-the-public-app-private-key)
- [Before going public](#before-going-public)
## One-time setup
- On PyPI, add a **trusted publisher** for this repo: workflow
**`release-please.yml`**, environment `pypi` (no `PYPI_TOKEN` secret — auth is
via OIDC). The publish job is inline in that workflow on purpose: PyPI trusted
publishing requires the OIDC `job_workflow_ref` to equal the top-level workflow.
- Create the repo **environment** named `pypi` (Settings → Environments).
- After the first release, set the **GHCR package visibility to public** so
consumers can `docker pull` the image (Packages → lgtmaybe → Package settings).
- First release only: from the GitHub release page, tick **"Publish this Action
to the GitHub Marketplace"**, accept the terms, and pick the categories
`code-review` and `continuous-integration`.
- **Homebrew tap:** create the repo **`MattJColes/homebrew-tap`** with a
`Formula/` directory (it can start empty — the workflow writes the formula).
Homebrew strips the `homebrew-` prefix, so that repo is the tap
`MattJColes/tap` and the formula inside it is `MattJColes/tap/lgtmaybe` — name
the repo `homebrew-tap`, not `homebrew-lgtmaybe`, or the formula reads as
`MattJColes/lgtmaybe/lgtmaybe`. Add a repo secret **`HOMEBREW_TAP_TOKEN`** to
*this* repo: a fine-grained PAT with `contents: write` on the tap repo (the
default `GITHUB_TOKEN` cannot push to another repository). To seed or verify
the formula by hand on a Mac, run
`scripts/update-homebrew-formula.sh path/to/homebrew-tap/Formula/lgtmaybe.rb`.
- **winget:** fork [`microsoft/winget-pkgs`](https://github.com/microsoft/winget-pkgs).
Create a classic GitHub PAT with the `public_repo` **and `workflow`** scopes
and add it to this repo as **`WINGET_TOKEN`**. `wingetcreate` syncs the fork
with upstream before it opens each PR, and `microsoft/winget-pkgs` changes
files under `.github/workflows/` often. GitHub refuses a fork sync that brings
in workflow-file changes unless the token has the `workflow` scope, so with
`public_repo` alone the submission fails on almost every release. For the first release, build/upload the Windows
asset and run `wingetcreate new ` manually with package id
`MattJColes.lgtmaybe`, installer type `portable`, command alias `lgtmaybe`,
and MIT licence metadata. Microsoft moderates a new package before it exists;
this can take days or weeks. Until that first manifest is accepted, the
automatic release job warns and skips the winget update rather than failing
the other release artifacts. Once accepted, the workflow uses
`wingetcreate update` for every later version.
## Each release
1. Merge feature/fix PRs to `main` using conventional-commit messages
(`feat:`, `fix:`, `feat!:` / `BREAKING CHANGE:` for a major bump).
2. release-please opens or updates the **Release PR** automatically. Review the
proposed version + changelog, then **merge it** to publish.
3. To (re)publish an existing tag to PyPI without a new release, run the
`release-please` workflow via **workflow_dispatch** with the tag name.
4. If Windows publication needs recovery, dispatch `windows-exe` for the tag,
then dispatch `winget` after the release asset is visible.
5. If the `winget` job fails with "The forked repository could not be synced
with the upstream commits", sync `MattJColes/winget-pkgs` with upstream
(**Sync fork** on its `master`, or
`gh api -X POST repos/MattJColes/winget-pkgs/merge-upstream -f branch=master`),
then dispatch `winget` again with the version. If it keeps recurring, check
that `WINGET_TOKEN` still carries the `workflow` scope.
## Rotate the public App private key
Rotate the public `lgtmaybe` GitHub App key quarterly, whenever maintainer access
changes, and immediately after any suspected exposure. Keep the old key active
until the replacement has passed a brokered smoke test so rotation does not
interrupt reviews.
1. In the GitHub App settings, generate a new private key. Do not delete the old
key yet.
2. From the directory containing the downloaded PEM, replace the retained
Secrets Manager value without printing the key:
```bash
aws secretsmanager put-secret-value \
--secret-id lgtmaybe/github-app/private-key \
--secret-string file://lgtmaybe.private-key.pem \
--region ap-southeast-2
```
3. Refresh the Lambda configuration so every execution environment drops its
cached copy of the old key:
```bash
FUNCTION_NAME="$(
aws cloudformation describe-stack-resources \
--stack-name LgtmaybeGithubAppIdentity \
--region ap-southeast-2 \
--query "StackResources[?ResourceType=='AWS::Lambda::Function'].PhysicalResourceId | [0]" \
--output text
)"
aws lambda update-function-configuration \
--function-name "$FUNCTION_NAME" \
--description "GitHub App key rotated $(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--region ap-southeast-2
aws lambda wait function-updated-v2 \
--function-name "$FUNCTION_NAME" \
--region ap-southeast-2
```
4. Run a brokered dogfood review and confirm GitHub attributes it to
`lgtmaybe[bot]`.
5. Delete the old key in the GitHub App settings, then delete the downloaded PEM
from the maintainer machine. Never paste the PEM into a command argument,
issue, workflow log, or repository file.
If the smoke test fails, leave the old GitHub key active, put its PEM back into
the same Secrets Manager secret, refresh the Lambda configuration again, and
investigate before retrying.
## Before going public
- Dogfood lgtmaybe on its own PRs so the README example is real.
- Re-check the least-privilege IAM/WIF scopes (see the
[Bedrock](./review-with-bedrock-oidc.md) and [Vertex](./review-with-vertex-wif.md)
guides) before the repo goes public.
---
# Configuration Reference
This reference is generated from the pydantic schemas in `src/lgtmaybe/core/models.py`.
## ReviewConfig
The user-facing configuration model. Fields map directly to `.lgtmaybe.yml` keys and CLI flags.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_base` | string / null | No | `null` | Api Base |
| `auto_describe` | boolean | No | `True` | Auto Describe |
| `auto_diagram` | boolean | No | `True` | Auto Diagram |
| `categories` | list[`complexity` / `correctness` / `deprecation` / `documentation` / `intent` / `performance` / `ponytail` / `security` / `spec` / `tests`] | No | `['security', 'correctness', 'deprecation', 'tests', 'documentation', 'performance', 'complexity', 'intent', 'ponytail', 'spec']` | Categories |
| `context_lines` | integer | No | `20` | Context Lines |
| `directory_rules` | list[DirectoryRule] | No | `[]` | Directory Rules |
| `exclude_paths` | list[string] | No | `[]` | Exclude Paths |
| `extra_lenses` | list[CustomLens] | No | `[]` | Extra Lenses |
| `fail_on` | `critical` / `high` / `info` / `low` / `medium` / null | No | `null` | |
| `fallback_model` | string / null | No | `null` | Fallback Model |
| `finding_rules` | list[FindingRule] | No | `[]` | Finding Rules |
| `function_context` | boolean | No | `True` | Function Context |
| `high_impact` | boolean | No | `True` | High Impact |
| `ignore_fingerprints` | list[string] | No | `[]` | Ignore Fingerprints |
| `include_paths` | list[string] | No | `[]` | Include Paths |
| `incremental` | boolean / null | No | `null` | Incremental |
| `language` | string / null | No | `null` | Language |
| `learn_feedback` | boolean | No | `True` | Learn Feedback |
| `max_concurrency` | integer / null | No | `null` | Max Concurrency |
| `max_file_diff_lines` | integer | No | `2000` | Max File Diff Lines |
| `max_files` | integer | No | `50` | Max Files |
| `max_findings_per_lens` | integer | No | `50` | Max Findings Per Lens |
| `max_input_tokens` | integer | No | `100000` | Max Input Tokens |
| `max_review_seconds` | integer | No | `3600` | Max Review Seconds |
| `max_review_tokens` | integer | No | `0` | Max Review Tokens |
| `max_tokens` | integer / null | No | `null` | Max Tokens |
| `mid_review_retrieval` | boolean | No | `False` | Mid Review Retrieval |
| `min_confidence` | integer | No | `0` | Min Confidence |
| `min_severity` | `critical` / `high` / `info` / `low` / `medium` | No | `low` | |
| `model` | string | Yes | — | Model |
| `num_ctx` | integer / null | No | `null` | Num Ctx |
| `pr_labels` | boolean | No | `False` | Pr Labels |
| `preset` | `fast` / `full` | No | `fast` | |
| `provider` | `anthropic` / `azure` / `bedrock` / `ollama` / `openai` / `openai-compatible` / `openrouter` / `vertex` / `zai` | Yes | — | |
| `reasoning_effort` | string / null | No | `null` | Reasoning Effort |
| `recursive` | boolean | No | `True` | Recursive |
| `reflect` | boolean | No | `True` | Reflect |
| `reflect_model` | string / null | No | `null` | Reflect Model |
| `repair_unparseable` | boolean | No | `True` | Repair Unparseable |
| `resolve_fixed` | boolean | No | `True` | Resolve Fixed |
| `retry_without_schema` | boolean | No | `True` | Retry Without Schema |
| `risk` | RiskConfig | No | `{'enabled': True, 'core_paths': []}` | |
| `spec_paths` | list[string] | No | `[]` | Spec Paths |
| `spec_review` | boolean | No | `True` | Spec Review |
| `static_analysis` | StaticAnalysisConfig | No | `{'enabled': False, 'tools': ['ruff', 'bandit', 'semgrep', 'mypy', 'gitleaks', 'zizmor', 'ast-grep', 'osv-scanner'], 'min_severity': 'info', 'tool_min_severity': {}, 'tool_mode': {}, 'semgrep_rules': None, 'ast_grep_rules': None}` | |
| `structured_output` | boolean | No | `True` | Structured Output |
| `summary_template` | string / null | No | `null` | Summary Template |
| `symbol_resolution` | boolean | No | `True` | Symbol Resolution |
| `temperature` | number | No | `0.0` | Temperature |
| `timeout` | integer / null | No | `null` | Timeout |
| `triage_model` | string / null | No | `null` | Triage Model |
| `unanchored_min_severity` | `critical` / `high` / `info` / `low` / `medium` | No | `high` | |
## Enums
### Provider
LLM backend selected by `--provider`. Cloud providers (`bedrock`, `vertex`, `azure`) use ambient credentials — `azure` also needs the resource endpoint (`--api-base`) and accepts a key as an alternative. `openai-compatible` points at any server speaking the OpenAI `/v1` wire format (DeepSeek, llama.cpp, LM Studio, vLLM) via `--api-base`; the key is optional so keyless local servers work.
- `anthropic`
- `azure`
- `bedrock`
- `ollama`
- `openai`
- `openai-compatible`
- `openrouter`
- `vertex`
- `zai`
### Severity
Finding severity, ordered low to high. Use `min_severity` in config to suppress findings below a threshold.
- `critical`
- `high`
- `info`
- `low`
- `medium`
## ReviewFinding
The structured output the model must return for each inline comment. All fields are validated by pydantic before anything is posted.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `anchor` | string / null | No | `null` | Anchor |
| `anchored` | boolean | No | `True` | Anchored |
| `body` | string | Yes | — | Body |
| `broad` | boolean | No | `False` | Broad |
| `category` | string / null | No | `null` | Category |
| `confidence` | integer / null | No | `null` | Confidence |
| `failure_scenario` | string / null | No | `null` | Failure Scenario |
| `line` | integer | Yes | — | Line |
| `path` | string | Yes | — | Path |
| `severity` | `critical` / `high` / `info` / `low` / `medium` | Yes | — | |
| `side` | string | No | `RIGHT` | Side |
| `suggestion` | string / null | No | `null` | Suggestion |
| `title` | string | Yes | — | Title |
## ProviderResult
The normalised return value of one LLM completion, including token usage.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `attempts` | integer | No | `1` | Attempts |
| `cache_creation_tokens` | integer | No | `0` | Cache Creation Tokens |
| `cache_read_tokens` | integer | No | `0` | Cache Read Tokens |
| `cost_usd` | number / null | No | `null` | Cost Usd |
| `input_tokens` | integer | Yes | — | Input Tokens |
| `model` | string / null | No | `null` | Model |
| `output_ceiling` | integer / null | No | `null` | Output Ceiling |
| `output_tokens` | integer | Yes | — | Output Tokens |
| `reasoning_tokens` | integer / null | No | `null` | Reasoning Tokens |
| `text` | string | Yes | — | Text |
## PRContext
Everything the engine needs about a pull request. Fetched via the GitHub REST API — PR code is never checked out.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `base_sha` | string | Yes | — | Base Sha |
| `changed_files` | list[string] | Yes | — | Changed Files |
| `commit_messages` | list[string] | No | `[]` | Commit Messages |
| `description` | string | No | `` | Description |
| `diff` | string | Yes | — | Diff |
| `feedback_downvotes` | list[string] | No | `[]` | Feedback Downvotes |
| `file_contents` | object | No | — | File Contents |
| `head_branch` | string | No | `` | Head Branch |
| `head_sha` | string | Yes | — | Head Sha |
| `open_finding_threads` | integer | No | `0` | Open Finding Threads |
| `pr_number` | integer | Yes | — | Pr Number |
| `repo` | string | Yes | — | Repo |
| `risk` | RiskAssessment / null | No | `null` | |
| `scan_contents` | object | No | — | Scan Contents |
| `title` | string | No | `` | Title |
## Raw JSON schemas
The canonical machine-readable schemas. These are the source of truth for provider output validation.
### ReviewConfig
```json
{
"$defs": {
"CustomLens": {
"additionalProperties": false,
"description": "A user-defined review lens \u2014 a \"skill file\" run alongside the built-ins.\n\nThe engine fans it out as its own focused LLM call (same pipeline as a\nbuilt-in ``ReviewCategory``) and merges its findings with the rest. A lens is\ndeclared in **trusted** config (``.lgtmaybe.yml`` or a file referenced by\n``lens_paths``), never from PR-author content, so its text is safe to put in\nthe system prompt. Supplying a worked example (``example_diff`` +\n``example_finding``) is optional but sharply improves a small model's output.",
"properties": {
"example_diff": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Example Diff"
},
"example_finding": {
"anyOf": [
{
"$ref": "#/$defs/ReviewFinding"
},
{
"type": "null"
}
],
"default": null
},
"id": {
"title": "Id",
"type": "string"
},
"instructions": {
"title": "Instructions",
"type": "string"
},
"title": {
"default": "",
"title": "Title",
"type": "string"
}
},
"required": [
"id",
"instructions"
],
"title": "CustomLens",
"type": "object"
},
"DirectoryRule": {
"additionalProperties": false,
"description": "Extra review instructions and context scoped to part of the repo.\n\nA monorepo is not uniform: ``payments/**`` wants strictness that would be\nnoise in ``tests/**``, and reviewing ``src/**`` well may need a design doc\nthe diff never shows. Each rule names the paths it applies to (fnmatch globs\nagainst the repo-relative path, a ``**/`` prefix also matching at the repo\nroot \u2014 the same matcher the path filters use; an EMPTY list applies the rule\neverywhere, mirroring ``FindingRuleMatch``), free-text ``instructions``, and\n``context_files`` read from the checked-out workspace.\n\nBoth are trusted configuration: on ``pull_request_target`` the workspace is\nthe BASE branch, so neither the instructions nor the context text is ever\nPR-author content.",
"properties": {
"context_files": {
"items": {
"type": "string"
},
"title": "Context Files",
"type": "array"
},
"instructions": {
"default": "",
"title": "Instructions",
"type": "string"
},
"paths": {
"items": {
"type": "string"
},
"title": "Paths",
"type": "array"
}
},
"title": "DirectoryRule",
"type": "object"
},
"FindingRule": {
"additionalProperties": false,
"description": "One declarative post-processing rule, applied in list order.\n\nThe safe alternative to arbitrary user hooks: rules can only filter or\nre-grade findings \u2014 no user code ever executes.",
"properties": {
"action": {
"$ref": "#/$defs/FindingRuleAction"
},
"match": {
"$ref": "#/$defs/FindingRuleMatch",
"default": {
"category": null,
"min_severity": null,
"path": null,
"title_contains": null
}
}
},
"required": [
"action"
],
"title": "FindingRule",
"type": "object"
},
"FindingRuleAction": {
"additionalProperties": false,
"description": "What a matched rule does: drop the finding, or remap its severity.",
"properties": {
"drop": {
"default": false,
"title": "Drop",
"type": "boolean"
},
"set_severity": {
"anyOf": [
{
"$ref": "#/$defs/Severity"
},
{
"type": "null"
}
],
"default": null
}
},
"title": "FindingRuleAction",
"type": "object"
},
"FindingRuleMatch": {
"additionalProperties": false,
"description": "The selector of a finding rule. Every specified field must match (AND).\n\nAn empty match selects every finding. ``path`` is an fnmatch glob against\nthe repo-relative path (a ``**/`` prefix also matches at the repo root,\nlike the path filters); ``category`` is the originating lens id;\n``title_contains`` is a case-insensitive substring; ``min_severity``\nselects findings at or above that severity.",
"properties": {
"category": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Category"
},
"min_severity": {
"anyOf": [
{
"$ref": "#/$defs/Severity"
},
{
"type": "null"
}
],
"default": null
},
"path": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Path"
},
"title_contains": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Contains"
}
},
"title": "FindingRuleMatch",
"type": "object"
},
"Provider": {
"description": "The backend selected by the `--provider` flag.",
"enum": [
"openai",
"openrouter",
"anthropic",
"bedrock",
"vertex",
"azure",
"ollama",
"openai-compatible",
"zai"
],
"title": "Provider",
"type": "string"
},
"ReviewCategory": {
"description": "A single review lens. The engine asks for each one in its own LLM call.\n\n``intent`` checks the diff against the PR's stated intent (title, description,\ncommit messages); it only runs when the context carries some stated intent.\n``ponytail`` is the \"lazy senior dev\" lens \u2014 the best code is the code you\nnever wrote \u2014 flagging code that needn't exist at all (YAGNI, reach for the\nstandard library, do it in fewer lines).\n``spec`` checks the diff against a specification the repository commits\n(OpenSpec, GitHub Spec Kit, Kiro); like ``intent`` it only runs when there is\nsomething to check against \u2014 here, a detected spec that matches the PR.",
"enum": [
"security",
"correctness",
"deprecation",
"tests",
"documentation",
"performance",
"complexity",
"intent",
"ponytail",
"spec"
],
"title": "ReviewCategory",
"type": "string"
},
"ReviewFinding": {
"additionalProperties": false,
"description": "A single inline review comment the model wants to post.",
"properties": {
"anchor": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Anchor"
},
"anchored": {
"default": true,
"title": "Anchored",
"type": "boolean"
},
"body": {
"title": "Body",
"type": "string"
},
"broad": {
"default": false,
"title": "Broad",
"type": "boolean"
},
"category": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Category"
},
"confidence": {
"anyOf": [
{
"maximum": 10,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Confidence"
},
"failure_scenario": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Failure Scenario"
},
"line": {
"minimum": 1,
"title": "Line",
"type": "integer"
},
"path": {
"title": "Path",
"type": "string"
},
"severity": {
"$ref": "#/$defs/Severity"
},
"side": {
"default": "RIGHT",
"enum": [
"LEFT",
"RIGHT"
],
"title": "Side",
"type": "string"
},
"suggestion": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Suggestion"
},
"title": {
"title": "Title",
"type": "string"
}
},
"required": [
"path",
"line",
"severity",
"title",
"body"
],
"title": "ReviewFinding",
"type": "object"
},
"ReviewPreset": {
"description": "How many model calls a review spends: the everyday path or the deep audit.\n\n``fast`` (the default) covers all nine built-in categories in FOUR calls,\none per concern: security, correctness (with stated intent folded in), code\nhealth, and artefacts (tests + documentation). The same four run on every\nprovider \u2014 worker count changes the schedule, not the call count.\n``full`` runs each of the nine categories as its own call for release\nbranches and deep audits. An explicit ``categories`` list always wins over\nthe preset.",
"enum": [
"fast",
"full"
],
"title": "ReviewPreset",
"type": "string"
},
"RiskConfig": {
"additionalProperties": false,
"description": "Risk of change: a deterministic low/medium/high/critical verdict per PR.\n\nBuilt from the change's blast radius (how many workspace files import each\nchanged file, via an ast-grep scan), the High Impact path signals, change\nsize, and whether code changed without a test changing. No model call, so\nthe verdict is free, stable across re-runs, and something a gate can read.",
"properties": {
"core_paths": {
"items": {
"type": "string"
},
"title": "Core Paths",
"type": "array"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
}
},
"title": "RiskConfig",
"type": "object"
},
"Severity": {
"description": "Finding severity, ordered low \u2192 high for `min_severity` filtering.",
"enum": [
"info",
"low",
"medium",
"high",
"critical"
],
"title": "Severity",
"type": "string"
},
"StaticAnalysisConfig": {
"additionalProperties": false,
"description": "Static-analysis fusion: deterministic tool findings as LLM grounding.\n\nWhen enabled, the installed tools run over the already-fetched changed-file\ntexts (sandboxed subprocess, scrubbed environment, no network, never a\ncheckout) and their findings enter each lens prompt as untrusted HINTS \u2014\n\"confirm, contextualise, or discard\" \u2014 raising recall on the deterministic\nbugs models miss without posting raw linter noise. A tool that isn't\ninstalled is skipped silently, so the feature degrades to nothing on a\nminimal install.",
"properties": {
"ast_grep_rules": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Ast Grep Rules"
},
"enabled": {
"default": false,
"title": "Enabled",
"type": "boolean"
},
"min_severity": {
"$ref": "#/$defs/Severity",
"default": "info"
},
"semgrep_rules": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Semgrep Rules"
},
"tool_min_severity": {
"additionalProperties": {
"$ref": "#/$defs/Severity"
},
"propertyNames": {
"$ref": "#/$defs/StaticAnalysisTool"
},
"title": "Tool Min Severity",
"type": "object"
},
"tool_mode": {
"additionalProperties": {
"$ref": "#/$defs/ToolMode"
},
"propertyNames": {
"$ref": "#/$defs/StaticAnalysisTool"
},
"title": "Tool Mode",
"type": "object"
},
"tools": {
"default": [
"ruff",
"bandit",
"semgrep",
"mypy",
"gitleaks",
"zizmor",
"ast-grep",
"osv-scanner"
],
"items": {
"$ref": "#/$defs/StaticAnalysisTool"
},
"title": "Tools",
"type": "array"
}
},
"title": "StaticAnalysisConfig",
"type": "object"
},
"StaticAnalysisTool": {
"description": "A deterministic linter/SAST tool whose findings feed the review.\n\nThe value is the binary name looked up on PATH, so it may differ from the\nmember name where the binary is hyphenated.",
"enum": [
"ruff",
"bandit",
"semgrep",
"mypy",
"gitleaks",
"zizmor",
"ast-grep",
"osv-scanner"
],
"title": "StaticAnalysisTool",
"type": "string"
},
"ToolMode": {
"description": "How a tool's findings reach the review.\n\n``hint`` is the original fusion behaviour: findings become an untrusted\nHINTS block the model confirms, contextualises, or discards. ``finding``\nskips the model entirely and posts the tool's result as a review comment.\n\nWhich one is right is a property of the tool's precision, not a user\npreference: a secret is present or it isn't, so asking a model to \"confirm\nor discard\" a deterministic regex match only adds cost and doubt. A lint or\na SAST heuristic genuinely benefits from a model judging whether it matters\nhere. Per-tool defaults live in ``engine.static_analysis``; this enum is the\noverride users reach for via ``static_analysis.tool_mode``.",
"enum": [
"hint",
"finding"
],
"title": "ToolMode",
"type": "string"
}
},
"additionalProperties": false,
"description": "How to run one review: provider/model, severity floor, filters, caps.",
"properties": {
"api_base": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Api Base"
},
"auto_describe": {
"default": true,
"title": "Auto Describe",
"type": "boolean"
},
"auto_diagram": {
"default": true,
"title": "Auto Diagram",
"type": "boolean"
},
"categories": {
"default": [
"security",
"correctness",
"deprecation",
"tests",
"documentation",
"performance",
"complexity",
"intent",
"ponytail",
"spec"
],
"items": {
"$ref": "#/$defs/ReviewCategory"
},
"title": "Categories",
"type": "array"
},
"context_lines": {
"default": 20,
"title": "Context Lines",
"type": "integer"
},
"directory_rules": {
"items": {
"$ref": "#/$defs/DirectoryRule"
},
"title": "Directory Rules",
"type": "array"
},
"exclude_paths": {
"items": {
"type": "string"
},
"title": "Exclude Paths",
"type": "array"
},
"extra_lenses": {
"items": {
"$ref": "#/$defs/CustomLens"
},
"title": "Extra Lenses",
"type": "array"
},
"fail_on": {
"anyOf": [
{
"$ref": "#/$defs/Severity"
},
{
"type": "null"
}
],
"default": null
},
"fallback_model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Fallback Model"
},
"finding_rules": {
"items": {
"$ref": "#/$defs/FindingRule"
},
"title": "Finding Rules",
"type": "array"
},
"function_context": {
"default": true,
"title": "Function Context",
"type": "boolean"
},
"high_impact": {
"default": true,
"title": "High Impact",
"type": "boolean"
},
"ignore_fingerprints": {
"items": {
"type": "string"
},
"title": "Ignore Fingerprints",
"type": "array"
},
"include_paths": {
"items": {
"type": "string"
},
"title": "Include Paths",
"type": "array"
},
"incremental": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"default": null,
"title": "Incremental"
},
"language": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Language"
},
"learn_feedback": {
"default": true,
"title": "Learn Feedback",
"type": "boolean"
},
"max_concurrency": {
"anyOf": [
{
"minimum": 1,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Concurrency"
},
"max_file_diff_lines": {
"default": 2000,
"minimum": 0,
"title": "Max File Diff Lines",
"type": "integer"
},
"max_files": {
"default": 50,
"title": "Max Files",
"type": "integer"
},
"max_findings_per_lens": {
"default": 50,
"minimum": 0,
"title": "Max Findings Per Lens",
"type": "integer"
},
"max_input_tokens": {
"default": 100000,
"title": "Max Input Tokens",
"type": "integer"
},
"max_review_seconds": {
"default": 3600,
"minimum": 0,
"title": "Max Review Seconds",
"type": "integer"
},
"max_review_tokens": {
"default": 0,
"minimum": 0,
"title": "Max Review Tokens",
"type": "integer"
},
"max_tokens": {
"anyOf": [
{
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Max Tokens"
},
"mid_review_retrieval": {
"default": false,
"title": "Mid Review Retrieval",
"type": "boolean"
},
"min_confidence": {
"default": 0,
"maximum": 10,
"minimum": 0,
"title": "Min Confidence",
"type": "integer"
},
"min_severity": {
"$ref": "#/$defs/Severity",
"default": "low"
},
"model": {
"title": "Model",
"type": "string"
},
"num_ctx": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Num Ctx"
},
"pr_labels": {
"default": false,
"title": "Pr Labels",
"type": "boolean"
},
"preset": {
"$ref": "#/$defs/ReviewPreset",
"default": "fast"
},
"provider": {
"$ref": "#/$defs/Provider"
},
"reasoning_effort": {
"anyOf": [
{
"enum": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh",
"default"
],
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Reasoning Effort"
},
"recursive": {
"default": true,
"title": "Recursive",
"type": "boolean"
},
"reflect": {
"default": true,
"title": "Reflect",
"type": "boolean"
},
"reflect_model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Reflect Model"
},
"repair_unparseable": {
"default": true,
"title": "Repair Unparseable",
"type": "boolean"
},
"resolve_fixed": {
"default": true,
"title": "Resolve Fixed",
"type": "boolean"
},
"retry_without_schema": {
"default": true,
"title": "Retry Without Schema",
"type": "boolean"
},
"risk": {
"$ref": "#/$defs/RiskConfig",
"default": {
"core_paths": [],
"enabled": true
}
},
"spec_paths": {
"items": {
"type": "string"
},
"title": "Spec Paths",
"type": "array"
},
"spec_review": {
"default": true,
"title": "Spec Review",
"type": "boolean"
},
"static_analysis": {
"$ref": "#/$defs/StaticAnalysisConfig",
"default": {
"ast_grep_rules": null,
"enabled": false,
"min_severity": "info",
"semgrep_rules": null,
"tool_min_severity": {},
"tool_mode": {},
"tools": [
"ruff",
"bandit",
"semgrep",
"mypy",
"gitleaks",
"zizmor",
"ast-grep",
"osv-scanner"
]
}
},
"structured_output": {
"default": true,
"title": "Structured Output",
"type": "boolean"
},
"summary_template": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Summary Template"
},
"symbol_resolution": {
"default": true,
"title": "Symbol Resolution",
"type": "boolean"
},
"temperature": {
"default": 0.0,
"title": "Temperature",
"type": "number"
},
"timeout": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Timeout"
},
"triage_model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Triage Model"
},
"unanchored_min_severity": {
"$ref": "#/$defs/Severity",
"default": "high"
}
},
"required": [
"provider",
"model"
],
"title": "ReviewConfig",
"type": "object"
}
```
### ReviewFinding
```json
{
"$defs": {
"Severity": {
"description": "Finding severity, ordered low \u2192 high for `min_severity` filtering.",
"enum": [
"info",
"low",
"medium",
"high",
"critical"
],
"title": "Severity",
"type": "string"
}
},
"additionalProperties": false,
"description": "A single inline review comment the model wants to post.",
"properties": {
"anchor": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Anchor"
},
"anchored": {
"default": true,
"title": "Anchored",
"type": "boolean"
},
"body": {
"title": "Body",
"type": "string"
},
"broad": {
"default": false,
"title": "Broad",
"type": "boolean"
},
"category": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Category"
},
"confidence": {
"anyOf": [
{
"maximum": 10,
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Confidence"
},
"failure_scenario": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Failure Scenario"
},
"line": {
"minimum": 1,
"title": "Line",
"type": "integer"
},
"path": {
"title": "Path",
"type": "string"
},
"severity": {
"$ref": "#/$defs/Severity"
},
"side": {
"default": "RIGHT",
"enum": [
"LEFT",
"RIGHT"
],
"title": "Side",
"type": "string"
},
"suggestion": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Suggestion"
},
"title": {
"title": "Title",
"type": "string"
}
},
"required": [
"path",
"line",
"severity",
"title",
"body"
],
"title": "ReviewFinding",
"type": "object"
}
```
### ProviderResult
```json
{
"additionalProperties": false,
"description": "The normalised return of one LLM completion, with token usage.",
"properties": {
"attempts": {
"default": 1,
"minimum": 1,
"title": "Attempts",
"type": "integer"
},
"cache_creation_tokens": {
"default": 0,
"title": "Cache Creation Tokens",
"type": "integer"
},
"cache_read_tokens": {
"default": 0,
"title": "Cache Read Tokens",
"type": "integer"
},
"cost_usd": {
"anyOf": [
{
"type": "number"
},
{
"type": "null"
}
],
"default": null,
"title": "Cost Usd"
},
"input_tokens": {
"title": "Input Tokens",
"type": "integer"
},
"model": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Model"
},
"output_ceiling": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Output Ceiling"
},
"output_tokens": {
"title": "Output Tokens",
"type": "integer"
},
"reasoning_tokens": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Reasoning Tokens"
},
"text": {
"title": "Text",
"type": "string"
}
},
"required": [
"text",
"input_tokens",
"output_tokens"
],
"title": "ProviderResult",
"type": "object"
}
```
### PRContext
```json
{
"$defs": {
"RiskAssessment": {
"additionalProperties": false,
"description": "A change's risk verdict: its level, the ranked reasons, and the gaps.\n\n``level`` None means the assessment itself failed and nothing is claimed.\n``unassessed`` lists changed code files whose blast radius was not measured,\nso a ``low`` verdict can never pass for a measured clean result.",
"properties": {
"level": {
"anyOf": [
{
"enum": [
"low",
"medium",
"high",
"critical"
],
"type": "string"
},
{
"type": "null"
}
],
"title": "Level"
},
"reasons": {
"items": {
"$ref": "#/$defs/RiskReason"
},
"title": "Reasons",
"type": "array"
},
"unassessed": {
"items": {
"type": "string"
},
"title": "Unassessed",
"type": "array"
}
},
"required": [
"level"
],
"title": "RiskAssessment",
"type": "object"
},
"RiskReason": {
"additionalProperties": false,
"description": "One reason a risk verdict is at its level.\n\n``factor`` names the rule that fired and ``key`` what makes it distinct from\nanother reason of the same factor (the High Impact area, or the factor\nitself): two distinct ``high`` reasons make a change ``critical``, while two\nTerraform files are one reason. ``text`` is rendered Markdown, paths already\nbacktick-stripped.",
"properties": {
"factor": {
"enum": [
"blast_radius",
"area",
"size",
"untested",
"scope"
],
"title": "Factor",
"type": "string"
},
"key": {
"title": "Key",
"type": "string"
},
"level": {
"enum": [
"low",
"medium",
"high",
"critical"
],
"title": "Level",
"type": "string"
},
"text": {
"title": "Text",
"type": "string"
}
},
"required": [
"factor",
"key",
"level",
"text"
],
"title": "RiskReason",
"type": "object"
}
},
"additionalProperties": false,
"description": "Everything the engine needs about a PR \u2014 fetched via API, never checkout.",
"properties": {
"base_sha": {
"title": "Base Sha",
"type": "string"
},
"changed_files": {
"items": {
"type": "string"
},
"title": "Changed Files",
"type": "array"
},
"commit_messages": {
"items": {
"type": "string"
},
"title": "Commit Messages",
"type": "array"
},
"description": {
"default": "",
"title": "Description",
"type": "string"
},
"diff": {
"title": "Diff",
"type": "string"
},
"feedback_downvotes": {
"default": [],
"items": {
"type": "string"
},
"title": "Feedback Downvotes",
"type": "array",
"uniqueItems": true
},
"file_contents": {
"additionalProperties": {
"type": "string"
},
"title": "File Contents",
"type": "object"
},
"head_branch": {
"default": "",
"title": "Head Branch",
"type": "string"
},
"head_sha": {
"title": "Head Sha",
"type": "string"
},
"open_finding_threads": {
"default": 0,
"minimum": 0,
"title": "Open Finding Threads",
"type": "integer"
},
"pr_number": {
"title": "Pr Number",
"type": "integer"
},
"repo": {
"title": "Repo",
"type": "string"
},
"risk": {
"anyOf": [
{
"$ref": "#/$defs/RiskAssessment"
},
{
"type": "null"
}
],
"default": null
},
"scan_contents": {
"additionalProperties": {
"type": "string"
},
"title": "Scan Contents",
"type": "object"
},
"title": {
"default": "",
"title": "Title",
"type": "string"
}
},
"required": [
"diff",
"changed_files",
"base_sha",
"head_sha",
"repo",
"pr_number"
],
"title": "PRContext",
"type": "object"
}
```
---
# Profile Reference
`--profile` prints a per-call breakdown of a review: where the time went, what
each lens spent, and what it produced. `--profile-json PATH` writes the same
data as JSON.
**Parse the JSON, not the table.** The table's layout is not a contract and may
be restyled; the JSON payload carries a `schema_version` precisely so it can be
pinned against.
## Which stream the table uses
The table goes to **stdout** for a human-format review, and to **stderr** when
the findings themselves are machine-readable (`--format json`, `--format agent`,
`--json`). stdout carries the deliverable, so it stays parseable — the same rule
the billable-token footer follows.
`--profile-json` writes to a **file**, so it never collides with either stream
whatever the output format is. An unwritable path is logged and skipped: a
review that produced findings is not lost to a diagnostic file.
## The table's columns
| column | meaning |
|---|---|
| `call` | the lens id, or a stage label like `reflect`, `triage`, `repair:` |
| `batch` | which batch of files the call covered |
| `tries` | requests actually issued, including adapter retries |
| `elapsed` | wall time for the call |
| `in_tok` | prompt tokens |
| `out_tok` | completion tokens |
| `think_tok` | reasoning tokens, when the route reports them |
| `think_%` | reasoning tokens as a share of the `max_tokens` ceiling |
| `cache_rd` | prompt-cache tokens read |
| `cache_wr` | prompt-cache tokens written |
| `findings` | findings parsed from this call |
| `error` | the failure reason, when the call failed |
### The `models:` line
A run whose calls were all answered by the same model says nothing. When a
second model answered any of them — `fallback_model`, whether the engine
escalated a truncation to it or the adapter switched on its own — a summary line
under the table names each model and how many calls it took:
```
models: luna (7 calls), sonnet (1 call)
```
There is no per-call `model` column: it would repeat one string down the whole
table. The per-call value is in the JSON payload and in the structured
`provider call` log line, omitted where the adapter reports no model.
### What `-` means
A dash is **unknown**, never zero.
`think_tok` is a dash when the route reported no reasoning breakdown. That is
deliberate: `0` would assert the model did no thinking, and nobody said that.
`think_%` is a dash when there is no ceiling to be a share of, and `findings` is
a dash for a call that does not produce review findings at all — distinct from a
call that produced none, which shows `0`.
This is the distinction that breaks naive parsers: `int(row["think_tok"])`
raises on a dash. In the JSON payload the same value is `null`.
## The JSON payload
```json
{
"schema_version": 1,
"wall_seconds": 12.4,
"total_tokens": 41231,
"returned_findings": 3,
"stages": [{"name": "redact", "elapsed": 0.01}],
"calls": [
{
"label": "security",
"batch": 1,
"attempts": 1,
"elapsed": 8.2,
"input_tokens": 36918,
"output_tokens": 1286,
"reasoning_tokens": null,
"reasoning_share": null,
"output_ceiling": null,
"cache_read_tokens": 0,
"cache_creation_tokens": 0,
"findings": 2,
"error": null
}
]
}
```
Notes:
- `reasoning_tokens`, `reasoning_share`, `output_ceiling`, `findings` and
`error` are `null` when unknown or absent — the JSON counterpart of the
table's `-`.
- `reasoning_share` is the raw ratio the table rounds into `think_%`.
- `total_tokens` is `input + output` across all calls. Cache counters are
deliberately excluded: routes disagree on whether a cached read is already
inside the prompt count, so adding them would double-count on some and not
others.
- `returned_findings` is the count after the whole pipeline, which may be lower
than the calls' parsed total once dedupe, reflection and filtering have run.
## Stability
`schema_version` is bumped when the payload's shape changes in a way a pinned
consumer would notice. Adding a field is not such a change; removing one or
altering what it means is.
---
# What gets reviewed
This page explains what lgtmaybe looks at, how it bounds the work, and what the
output looks like — on a pull or merge request, and on the command line.
## What it looks at
lgtmaybe reviews the **diff of a pull request** (or a GitLab merge request) —
the lines it adds or changes, not the whole repository. It fetches that diff
from your code host's REST API and **never checks out or executes your code**,
so a malicious change can't run anything in the reviewer's environment. The diff is treated as untrusted input
throughout, including against prompt-injection attempts hidden in PR text.
To review changes in context rather than in isolation, lgtmaybe also pads each
changed hunk with a few **surrounding lines** read from the head revision of the
changed file. The model uses these to understand the change — the enclosing
function, nearby definitions — but only ever comments on the changed lines. How
many lines are added is budget-scaled and capped by `context_lines` (default 20,
`0` disables it). This content is fetched read-only via the API and redacted
like the diff.
Before the diff reaches the model it is cleaned:
- **Non-reviewable files are skipped** — lockfiles, minified/bundled assets,
vendored directories, generated LLM-index corpora (`llms.txt` / `llms-full.txt`),
and binaries. Reviewing them is noise and wastes tokens.
- **Oversized files are skipped** — a single file whose diff runs past
`max_file_diff_lines` (default 2000, `0` disables) is dropped and named in the
summary. Name matching can only catch generated files that admit it in their
name; a hand-named 154,000-line data blob is caught on size alone.
- **Secrets are redacted** — anything that looks like a key or token is stripped
from the diff before it leaves your environment for the provider.
## Correctness & logic
The substance of a change, not just style. The model is prompted to actively
hunt the bugs a change introduces and grade them by impact:
- **Null / None dereferences** — a value that can be empty used without a guard.
- **Off-by-one & boundary errors** — `<` vs `<=`, fencepost mistakes, empty- and
single-element edge cases.
- **Mismatched or inverted ranges** — `start`/`end` swapped, a lower bound above
its upper bound.
- **Unhandled error / exception paths** — failures silently swallowed or state
left half-updated.
- **Incorrect conditionals** — inverted booleans, `and`/`or` mix-ups, missing
branches.
- **Resource leaks & ordering** — handles or locks not released, use-after-close,
bad concurrent sequencing.
- **Races & concurrency** — check-then-act (TOCTOU), shared mutable state without
synchronisation, coroutines called without `await`, blocking calls in async paths.
- **Numeric and date/time bugs** — overflow, float equality, division by zero,
money in binary floats; timezone-naive datetimes, epoch-unit confusion, DST.
- **Aliasing & mutation** — mutable default arguments, mutating a collection while
iterating it, sharing a mutable value the caller still owns.
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [HIGH] possible None dereference, where get_user can return None but .email is accessed without a guard](../assets/review-correctness.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [HIGH] None-dereference finding for demo/orders.py](../assets/cli-correctness.png){ width="660" }
## Security review
Security findings are first-class. The model is prompted with an OWASP-aligned
checklist and told to grade what it finds `high` or `critical` and name the
vulnerability class in the title. It actively looks for:
- **Injection** — SQL/NoSQL, OS command, and template/LDAP injection.
- **Cross-site scripting (XSS)** — unescaped user input rendered into HTML/JS.
- **CSRF & open redirect** — unprotected state-changing endpoints, user-controlled
redirect targets.
- **Hardcoded secrets** — keys, tokens, passwords, or private keys in the diff.
- **Broken authn / authz** — missing permission checks, IDOR, auth bypass, and JWT
or session pitfalls (unverified signatures, `alg` confusion, missing expiry).
- **Path traversal / unsafe file access** — user input in file paths, `../`,
zip-slip extraction — plus unrestricted file uploads.
- **SSRF** — server-side fetches of user-controlled URLs without allow-listing.
- **Insecure deserialization & unsafe eval** — `pickle`/`yaml.load`/`eval` on
untrusted data, and XML parsed with external entities enabled (XXE).
- **Mass assignment / over-posting** — request bodies bound straight onto models.
- **Weak cryptography** — MD5/SHA1 for passwords, ECB mode, disabled TLS
verification, predictable randomness for security tokens.
- **Sensitive-data exposure** — secrets or PII in logs, error responses, or
analytics: passwords, API keys, tokens/session IDs, SSNs, or payment-card data.
- **CI / IaC misconfiguration** — untrusted input interpolated into workflow `run:`
steps, third-party actions not pinned to a SHA, overly broad IAM policies,
public buckets, privileged containers, secrets echoed into build logs.
- **Resource / DoS safety** — missing timeouts, unbounded loops or allocations,
regexes vulnerable to catastrophic backtracking (ReDoS).
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [CRITICAL] SQL injection vulnerability in a find_user function, explaining the unsafe string concatenation and suggesting a parameterized query](../assets/review-sql-injection.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [CRITICAL] SQL injection finding for demo/db_queries.py](../assets/cli-security.png){ width="660" }
This shapes *what* the reviewer flags. It is separate from how lgtmaybe protects
**itself** from a malicious PR — see
[Data and Privacy](data-and-privacy.md) for secret redaction and prompt-injection
defence.
## Deprecation & dependency health
Beyond bugs and vulnerabilities, the reviewer also flags **factually outdated**
code when the diff shows it — these are objective, not stylistic:
- deprecated language/framework APIs (with the modern replacement suggested when
known),
- targeting an end-of-life runtime or language version,
- adding or pinning an end-of-life / abandoned dependency,
- pinning a dependency to a version with a known security advisory, and
- a new dependency whose name looks like a typosquat of a popular package, or
whose license conflicts with the project's.
The reviewer only raises these when the diff itself shows the change; it does not
speculate about code it cannot see.
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [MEDIUM] deprecated datetime.utcnow() call and suggesting datetime.now(timezone.utc)](../assets/review-deprecation.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [MEDIUM] deprecated-API finding for demo/scheduler.py](../assets/cli-deprecation.png){ width="660" }
## Test coverage & documentation
Two lighter-weight checks round out a review:
- **Missing or weak tests** — when the diff adds a new function, branch, or error
case with no accompanying test, the reviewer raises a `low`/`medium` finding and
puts a concrete, runnable test in the finding's `suggestion` field, matching
the project's existing test idiom. Tests added in the diff that don't really
test — assertion-free, over-mocked until only the mock is exercised, or flaky
(sleep-based waits, wall-clock or ordering dependence) — are flagged too.
Renames, comments, and trivial formatting changes are left alone.
- **Documentation gaps and stale docs** — public/exported surfaces added without
a docstring, or a name or signature that contradicts what the code does, are
flagged at `info`/`low`; a docstring or comment the change just made wrong is
flagged up to `medium` (a comment that lies is worse than no comment). This is
deliberately restrained: private helpers and self-evident code are not nagged
about, so well-named code is left to document itself.
A missing test — note the runnable test dropped into the suggestion:
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [LOW] new branch added without a test, with a runnable pytest suggestion](../assets/review-tests.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [LOW] missing-test finding for demo/discount.py](../assets/cli-tests.png){ width="660" }
A documentation gap on a new public function:
=== "On a pull request"
![An inline lgtmaybe review comment flagging an [INFO] public function missing a docstring, with a suggested docstring](../assets/review-documentation.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing an [INFO] missing-docstring finding for demo/client.py](../assets/cli-documentation.png){ width="660" }
## Performance
The reviewer also watches for performance regressions the change introduces,
graded by impact (`low` up to `high` when the cost scales with input size or sits
in a hot path):
- **N+1 queries / calls in a loop** — a query, request, or other expensive call
issued once per iteration that could be batched or hoisted out.
- **Inefficient algorithms** — accidentally quadratic (`O(n²)`) work where linear
is feasible, or a linear scan where a set/dict lookup would do.
- **Redundant computation** — recomputing the same value inside a loop instead of
hoisting or memoising it.
- **Unnecessary allocations & copies** — building large intermediates or copying
big buffers on a hot path when streaming or in-place work suffices.
- **Blocking I/O on a hot path** — synchronous I/O, sleeps, or lock contention
where non-blocking handling is expected.
- **Unbounded / over-fetching queries** — loading whole tables into memory or
missing pagination/limits.
- **Unbounded growth & leaks** — caches without eviction, listeners or
subscriptions never removed, queues that only grow.
It sticks to changes the diff actually shows and avoids micro-optimisations with
no measurable impact.
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [HIGH] N+1 query inside a loop, suggesting a single batched query](../assets/review-performance.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [HIGH] N+1-query finding for demo/reports.py](../assets/cli-performance.png){ width="660" }
## Complexity
A lighter, restrained lens that flags code harder to read, test, or maintain than
it needs to be (`info`/`medium`), preferring a concrete simplification in the
`suggestion` field:
- **High cyclomatic complexity / deep nesting** — many branches or deeply nested
conditionals and loops that would read better with early returns.
- **Over-long, low-cohesion functions** — a function doing several unrelated
things that should be split apart.
- **Duplicated logic** — non-trivial logic repeated in the diff that should be
extracted into a shared helper.
- **Excessive parameters / boolean-flag arguments**, **convoluted expressions**,
and **dead / unreachable code**.
Like the documentation lens, it stays quiet on self-evident, already-simple code.
=== "On a pull request"
![An inline lgtmaybe review comment flagging a [MEDIUM] deeply nested conditional and suggesting guard clauses](../assets/review-complexity.png){ width="660" }
=== "On the CLI"
![The lgtmaybe CLI printing a [MEDIUM] deep-nesting finding for demo/router.py](../assets/cli-complexity.png){ width="660" }
## Intent — does the PR do what it says?
The intent lens compares the diff against the PR's **stated intent** and flags
mismatches at `medium`, or `high` when the unexplained change is
security-relevant:
- **Out-of-scope changes** — a hunk unrelated to the stated intent, e.g. a "fix
typo" PR that also touches auth logic, CI workflows, dependency pins, or
permissions. Smuggled security-relevant changes are the highest-value catch.
- **Contradictions** — the code does the opposite of, or something materially
different from, what the title or commits claim.
- **Unfulfilled intent** — the PR promises behaviour the diff never implements.
Where the stated intent comes from:
- **On a GitHub PR** — the PR title, description, and the first line of each
commit message, fetched via the API.
- **On the CLI** — the commit names from your local `git log` against the
remote primary branch, so the lens works without GitHub — in `--working`
mode too. With no commits beyond the base yet, nothing states an intent and
the lens is skipped.
### The lens is told what it was not shown
An intent lens judging "did the author keep their promise?" against a *filtered*
diff will call a kept promise broken. Files go missing for seven different
reasons — the generated/binary/vendored skip, your `include_paths` /
`exclude_paths` globs, the `max_file_diff_lines` size cap, the `max_files` cap,
a triage skip, an incremental
scope, and simply being in another batch (the lens runs once **per batch**, so
on a large PR each call sees only part of the change).
So each intent call is told which of the PR's files it cannot see, and the rule
that goes with it: a claim about a file that was not shown is **not shown, not
undone**. The list is derived from what is left of the PR after the batch, so
it covers every one of those mechanisms without caring which applied — and it
is capped, then wrapped and neutralised inside the untrusted intent block,
because filenames are attacker-controlled on a fork PR too.
This is why lgtmaybe no longer reports "the stated intent to regenerate X is
not reflected in the diff" when X is a generated file it was never allowed to
read.
The intent text is attacker-controlled on a fork PR, so it is treated exactly
like the diff: secrets are redacted, it is wrapped as untrusted data with
neutralised delimiters, and the model is told never to follow instructions
inside it. Only the intent lens's model call ever carries it. When a PR states
no intent at all, the lens is skipped instead of burning a model call.
## Spec — does the PR deliver the specification it commits to?
If your repository drives its work from a committed specification, that spec is
a far better statement of intent than a PR description: it is structured, it
predates the code, and its task list records what the author claims to have
finished. The spec lens checks the diff against it — in **both** directions.
lgtmaybe recognises three layouts out of the box, plus your own:
| Workflow | Detected by | Read |
|---|---|---|
| [OpenSpec](https://github.com/Fission-AI/OpenSpec) | `openspec/changes//`, `openspec/specs//` | proposal, delta specs, design, tasks (archived changes are ignored) |
| [GitHub Spec Kit](https://github.com/github/spec-kit) | `.specify/`, or `specs//` with a `spec.md` **and** a `plan.md` | spec, plan, tasks |
| [Kiro](https://kiro.dev/docs/specs/) | `.kiro/specs//` | requirements, design, tasks |
| Your own layout | `spec_paths` globs in `.lgtmaybe.yml` | whichever of those filenames are present |
### What it reports
**The diff falling short of the spec**
- **Contradicts an explicit requirement** — the code does the opposite of a
stated SHALL/MUST or acceptance criterion (`high`).
- **A ticked task that is not delivered** (`medium`) — see below.
- **A requirement in scope with nothing implementing it** (`medium`).
- **An acceptance criterion with no test** (`low`).
**The spec falling short of the diff** — the half people miss, and the most
reliable of the two, because the evidence is entirely in the diff:
- **Behaviour no requirement covers** — a new endpoint, state, error path, limit
or side effect the spec never mentions (`low`/`info`).
- **A requirement the change made stale** (`low`).
- **An unresolved `[NEEDS CLARIFICATION]`** still sitting in a requirement this
PR implements (`info`).
### Ticked checkboxes are claims
All three workflows track progress with markdown checkboxes, and a PR that
implements tasks *flips* them:
```diff
-- [ ] T014 [US1] Enforce the 30-day link expiry in src/links/service.py
+- [x] T014 [US1] Enforce the 30-day link expiry in src/links/service.py
```
That flip is already in the diff. lgtmaybe extracts it with no model call and no
extra file read, and hands the lens the resulting list as *claims the author made
in this pull request* — turning a vague question ("did this deliver the spec?")
into a precise one ("is T014 actually here?"). A ticked task is something to
**check**, never to assume false: the lens flags it only when the diff positively
shows the work is absent.
### Which spec, and when it stays quiet
A monorepo can hold forty spec directories, so lgtmaybe ranks them against the
PR — it edits the spec, its branch is named after one (Spec Kit names branches
after the spec directory), or its title, description or commits name one — and
sends at most two. **When nothing matches, the lens does not run at all**: no
model call, no prompt bytes. The same is true when no spec system is present,
which is the common case, so a repository without specs pays nothing for this.
Because it needs its own large block, the spec lens is a call of its own — a
fifth one under the default `fast` preset, and only in repositories where a spec
actually matched. Turn it off with `--no-spec`, `spec_review: false` in
`.lgtmaybe.yml`, or `spec_review: false` on the Action.
### The lens is told what it was not shown
A requirement is delivered by *code*, so this lens is even more exposed than the
intent lens to the filtered-diff trap: told nothing, it reports every requirement
implemented in another batch as undelivered. It gets the same correction — the
list of the PR's files this call cannot see, and the rule that a requirement
delivered in one of them is **not shown, not undelivered**.
Spec text is treated exactly like the diff: redacted, wrapped as untrusted data
in its own neutralised block, and never obeyed. That is deliberate rather than
paranoid — a spec is usually committed in the same PR that implements it, so on
a fork PR the author controls the requirements their own change is judged
against. Files the PR changes are read from its head text for that same reason
(the base branch does not have them yet); everything else comes from the
checked-out workspace, which on `pull_request_target` is the trusted base
branch. The worst a planted spec can do is suppress a spec finding — no other
lens ever sees the block.
## Ponytail — the laziest senior dev in the room
The best code is the code you never wrote. Inspired by the
[Ponytail](https://github.com/DietrichGebert/ponytail) skill, this lens reviews
new code with a senior engineer's reflex to **not** add code, flagging what
needn't exist at all (graded `info` to `medium`, and deliberately restrained):
- **Needless code (YAGNI)** — speculative generality, "just in case" parameters,
an abstraction with a single caller, or scaffolding for a future that isn't here.
- **Reinventing the standard library** — hand-rolled code a built-in, the standard
library, or an already-imported dependency does directly.
- **Could be far shorter** — several lines doing what one clear expression would.
- **Premature configurability** — flags, hooks, or options no caller uses yet.
It prefers deleting or collapsing code over adding to it and puts the smaller
replacement in the suggestion. It is distinct from the complexity lens (which asks
"is this code hard to follow?"); Ponytail asks "should this code exist at all?"
## How the scope is bounded
Every run is bounded so a large PR can't run away on latency. All of these are
configurable in `.lgtmaybe.yml` (see
[Configure .lgtmaybe.yml](../how-to/configure-lgtmaybe-yml.md)):
| Knob | Default | Effect |
|---|---|---|
| `preset` | `fast` | `fast` uses four calls — security, correctness, code health, artefacts — on every provider; `full` runs one call per lens. |
| `max_files` | 50 | Reviews the top-N changed files; posts a "reviewed top N of M" notice if there are more. |
| `max_file_diff_lines` | 2000 | Skips any single file whose diff is longer, naming it in the summary. `0` disables. |
| `max_input_tokens` | 100,000 | Batches the diff so each model call stays within budget. |
| `max_concurrency` | 6, every provider | Concurrent model calls across the whole fan-out (all batches share one pool). Local throughput is set on the server (`OLLAMA_NUM_PARALLEL`, llama.cpp `-np`, vLLM batching); set this to `1` for serial. |
| `max_review_seconds` | 3600 | Soft wall-clock ceiling: past it, queued calls are skipped and the review posts partial results with a notice. `0` disables. |
| `categories` | all nine | Which review lenses to run; an explicit list overrides the preset grouping and runs those lenses one call each. |
| `context_lines` | 20 | Ceiling on surrounding lines added around each hunk; the budget may use fewer. `0` disables context expansion. |
| `min_severity` | `low` | Drops findings below the chosen floor (`info` → `low` → `medium` → `high` → `critical`); `low` keeps everything except pure-`info` narration. |
| `include_paths` / `exclude_paths` | — | Glob filters to focus the review. |
> These bound a **single run**, not the number of runs. On a public repo, anyone
> who can open a PR or comment can trigger a run, and each run calls your chosen
> LLM provider — see the cost disclaimer in
> [Use as a GitHub Action](../how-to/use-as-github-action.md).
## What a finding contains
Findings are structured data, not prose, so they render identically everywhere.
Each finding has:
| Field | Meaning |
|---|---|
| `path` | File the comment attaches to |
| `line` | Line in the diff |
| `side` | `RIGHT` (added/changed) or `LEFT` (removed) |
| `severity` | `info` / `low` / `medium` / `high` / `critical` |
| `title` | One-line summary |
| `body` | The explanation |
| `suggestion` | Optional suggested replacement code |
The nine review categories are security, correctness, deprecation, tests,
documentation, performance, complexity, intent, and ponytail. The default
`fast` preset covers all nine in four calls, one per concern: security,
correctness (with stated intent folded in), merged code health
(performance/complexity/ponytail/deprecation), and artefacts
(tests/documentation). The same four run on every provider — worker count
changes only how they are scheduled. `preset: full` runs each category as its
own focused call with a worked example of its own finding type. Their findings
are merged and de-duplicated. A
self-reflection pass then runs over the merged set and drops low-confidence
findings, so the model's first guesses are filtered before anything is posted.
The reviewer only ever sees the diff and a little surrounding context — a *slice*
of the codebase, not the whole thing. So when a concern depends on code it can't
see (a guard, a base class, an idempotency check that may live in an unshown
file), it hedges and lowers the severity rather than asserting that the thing is
missing, and the self-reflection pass drops findings that rest on such unseen-code
assumptions. Genuine gaps in the diff itself — a changed path with no test, a new
public surface left undocumented — are explicitly exempt and still raised.
## What the response looks like
### On a pull or merge request
The rendering below is GitHub's. GitLab and Gitea show the same findings with
the same badges; [how each host posts them](#how-each-host-posts) differs, and
is covered at the end of this section.
lgtmaybe posts **one review** containing:
- an **inline comment** on the exact changed line for each finding, and
- a **summary comment** that names the model used.
Each finding lands on the line that triggered it, with its severity in the title,
the explanation in the body, and — where the fix is clear — a suggested change you
can commit straight from the PR:
![An inline lgtmaybe review comment flagging a [MEDIUM] server-side request forgery (SSRF) risk where a user_id is concatenated into a URL, with a suggested validation fix](../assets/review-ssrf.png){ width="660" }
![An inline lgtmaybe review comment flagging a [CRITICAL] command injection vulnerability in an archive function using subprocess with shell=True, with a suggested fix that avoids the shell](../assets/review-command-injection.png){ width="660" }
The summary carries a hidden marker (``), so re-running on the
same PR **updates** the existing review instead of creating duplicates.
#### Reading the title line
A comment's title line carries the finding's provenance in its brackets:
```
**[HIGH · security · 80%] User input is concatenated into the SQL string**
```
- **`HIGH`** — the severity.
- **`security`** — the lens that raised it. One of the nine review categories
(or your own id if you added a [custom lens](../how-to/add-a-custom-lens.md)),
and the same value you match on in `finding_rules` — so a badge you keep seeing
and don't want tells you exactly what rule to write.
- **`80%`** — the self-reflection auditor's confidence that the finding is real,
reached by actively trying to disprove it. It is the auditor's 0-10 score shown
as a percentage, so `min_confidence: 5` is the same floor as `50%`. Set that
floor to drop everything below a score you choose.
Each half drops away when it isn't there: with `reflect: false` there is no score
and the badge is just the lens.
One asymmetry worth knowing: neither GitHub's nor Gitea's review API can edit an
inline comment once it's posted, so an inline comment's score is **frozen at
first post** — a
later run that judges the same finding differently won't change it. Findings in
the summary body (the "Additional findings" and "Broader observations" sections)
are rewritten on every run, so those badges do track. That's how severity and
title have always behaved too; the badge just makes it visible.
### Resolving conversations once they're fixed
Each inline comment also carries a hidden per-finding fingerprint. When you push
a fix and lgtmaybe runs again, it looks at its own open conversations: if a
finding it raised is **no longer produced** *and* GitHub marks that thread
**outdated** (the code under it changed), lgtmaybe treats it as fixed — it posts
a short `✅ Looks resolved.` reply and resolves the conversation. Both conditions
must hold, so a thread is never collapsed just because the lines around it
shifted, or because a single run happened not to re-flag it without the code
changing.
This is on by default. To leave conversations for manual resolution, set
`resolve_fixed: false` in `.lgtmaybe.yml` (or the Action's `resolve_fixed` input).
Resolving a thread uses GitHub's GraphQL API; the workflow's default
`GITHUB_TOKEN` (with `pull-requests: write`, already needed to post the review)
is sufficient. The step is best-effort — if it can't run, the review itself still
posts normally.
On **GitLab** the same thing happens over plain REST rather than GraphQL. On
**Gitea** it does not happen at all — Gitea has no thread-resolution API, so
findings' comments stay open for a human to close.
### How each host posts
The findings are identical everywhere. What each host's API allows is not:
| | GitHub | GitLab | Gitea |
|---|---|---|---|
| Inline comments | one batched review | one discussion per finding | one batched review |
| Summary | the review body, edited in place | a note, edited in place | an issue comment, edited in place |
| Avoiding duplicates on a re-run | edits the review | skips ids already posted | skips ids already posted |
| Auto-resolve a fixed finding | ✅ GraphQL | ✅ REST | ✗ no API |
| Incremental review of new commits | ✅ | not yet | ✗ no compare diff |
The reason for the differences: a submitted **Gitea** review can't be edited, so
its summary moves to an ordinary comment and duplicate findings are filtered
*before* posting. **GitLab** has no batched review object at all, so each
finding becomes its own positioned discussion.
When a PR is clean (no findings, and every file was within the caps), the summary
is a simple:
```
👍 LGTM!
0 findings · provider anthropic · model claude-sonnet-4-6
```
If the file cap kicked in, the summary says so (e.g. "Reviewed the top 50 of 120
changed files"). lgtmaybe never fails silently — any error is surfaced back to
the PR as a short comment.
### On the command line
`lgtmaybe review` runs the same pipeline over your local `git` diff and prints
the findings — it posts nothing and needs no GitHub token. By default it diffs
the current branch against the remote primary branch (`origin/HEAD`, falling
back to `origin/main`/`origin/master`, then a local `main`/`master`).
`--working` reviews the whole worktree — branch commits plus uncommitted edits —
against that same base. `--uncommitted` reviews only the uncommitted edits
against HEAD, and `--base [` picks a different base. Both worktree modes
include files you haven't `git add`ed yet — a brand-new file is usually the
thing you most want a second pair of eyes on — while anything `.gitignore`
excludes stays out. The default output is a
readable listing followed by the summary line:
```console
$ lgtmaybe review --provider ollama --model qwen3.6:27b --api-base http://localhost:11434
src/app.py:2 [MEDIUM] Import order
sys should be sorted before os
1 finding · provider ollama · model qwen3.6:27b
```
![The lgtmaybe review command running in a terminal, printing a [MEDIUM] import-order finding with its file and line, then a summary line naming the model](../assets/cli-example.png){ width="660" }
`--format` selects the output. `--json` is shorthand for `--format json`, which
prints the findings as a JSON array so the same structured data can be piped into
other tooling:
```console
$ lgtmaybe review --provider ollama --model qwen3.6:27b --api-base http://localhost:11434 --json
[{"path": "src/app.py", "line": 2, "side": "RIGHT", "severity": "medium",
"title": "Import order", "body": "sys should be sorted before os",
"suggestion": null}]
```
`--format agent` turns the findings into plain correction instructions an AI
coding agent can read and apply — a local review-and-fix loop. See
[Fix findings with an AI agent](../how-to/fix-findings-with-an-ai-agent.md).
## See also
- [Getting Started](../tutorial/getting-started.md) — run your first review
- [Architecture](architecture.md) — the fetch → compress → prompt → parse → post pipeline
- [Data and Privacy](data-and-privacy.md) — what is sent where
---
# Risk of Change
Every review carries a **risk of change** verdict: `low`, `medium`, `high` or
`critical`, with the reasons that produced it. It answers a different question
from the findings. A finding says "this line is wrong". The verdict says "this
change can reach a long way, so read it carefully", even when every line is
right.
The verdict is deterministic. It makes no model call, so it costs nothing, it
reads the same on every re-run of the same head, and a workflow can rely on it.
## Where You See It
The review summary line names the level and its top reason:
```text
3 findings · risk high (`src/core/models.py` is imported by 84 files, +2 more) · provider … · model …
```
The change overview lists every reason in a **Risk of Change** section above
High Impact Areas. The summary carries the verdict even when the overview is
turned off or skipped, so every review has it.
The summary also carries a hidden marker, ``, and
with `pr_labels` on the PR gets a `risk/` label. Locally, `lgtmaybe
review` prints the same summary line.
## How the Level Is Decided
Four factors each raise the level, and the verdict takes the highest:
| Factor | Raises to |
|---|---|
| A changed file imported by 5 or more files in the repository | `medium` |
| A changed file imported by 20 or more files, or matching a `risk.core_paths` glob | `high` |
| Touching infrastructure, security, data migrations, or backup and recovery paths | `high` |
| Touching any other High Impact area (dependencies, compatibility, cost, …) | `medium` |
| 500 or more changed lines | `medium` |
| 50 or more changed lines of code with no test file changed | `medium` |
Two distinct `high` reasons make the change `critical`: a widely imported
module and a Terraform file, say, or an auth change and a database migration.
Two files in one area count once. A change touching only documentation and
tests is `low`.
## Blast Radius
The importer count comes from one [ast-grep](https://ast-grep.github.io/) scan
of the checked-out repository for Python, TypeScript and JavaScript imports. It
parses files and runs nothing. On `pull_request_target` the checkout is the
trusted base branch, and nothing from the scan is sent to the model.
The count measures the code that already depends on a file. A file the PR adds
has no existing importers, importers the PR itself changes are left out
because they are already in the diff, and test files don't count as importers.
When a blast radius can't be measured, the verdict names the file instead of
guessing. This happens for a language the scan doesn't read, when there is no
checkout, or when the scan runs past its 20-second limit. A `low` verdict with
unmeasured files always says so:
```text
risk low (blast radius not assessed: `lib/billing.rb`)
```
The import count can't see modules reached by dynamic import, TypeScript path
aliases or barrel re-exports. Name those in `risk.core_paths` (see
[`risk`](../how-to/configure-lgtmaybe-yml.md#risk)).
## Gating on the Verdict
lgtmaybe never approves or blocks a PR on its risk. The marker lets you decide
what a level means for your team. For example, a step after the lgtmaybe step
can request a second reviewer when the review summary carries
`lgtmaybe-risk:critical`:
```yaml
- name: Require a second reviewer on critical changes
env:
GH_TOKEN: ${{ github.token }}
PR: ${{ github.event.pull_request.number }}
run: |
if gh pr view "$PR" --json reviews --jq '.reviews[].body' \
| grep -q 'lgtmaybe-risk:critical'; then
gh pr edit "$PR" --add-reviewer my-org/platform-leads
fi
```
If the assessment itself fails, the summary reads `risk unavailable` and no
marker is written, so a gate reads a missing marker as "unknown", never as a
level.
---
# Architecture
lgtmaybe is built on **hexagonal architecture** (ports and adapters). The core
never imports from the adapters; adapters implement abstract ports defined in
`core/ports.py`. This lets the parallel build tracks evolve independently and
lets tests swap in fakes without patching.
## Ports and adapters
```mermaid
flowchart TB
subgraph core["core — never imports an adapter"]
ports["ports.py]
ProviderClient · ReviewGateway · ReviewEngine"]
models["models.py
ReviewConfig · ReviewFinding · ProviderResult · PRContext"]
end
providers["providers/
litellm adapter"] -- implements --> ports
github["github/
GitHub REST adapter"] -- implements --> ports
gitlab["gitlab/
GitLab REST adapter"] -- implements --> ports
gitea["gitea/
Gitea REST adapter"] -- implements --> ports
```
Two independent axes meet at these ports: which **model** reviews the code
(`ProviderClient`) and which **code host** the review is posted to
(`ReviewGateway`). Any combination works, because neither side knows about the
other.
**`core/ports.py`** — the seam. Three abstract base classes:
- `ProviderClient` — one method: `complete(messages, model)` returns a
`ProviderResult` (text + token usage).
- `ReviewGateway` — `get_pr_context()` fetches the change diff and metadata;
`post_review()` posts batched inline comments and a summary;
`post_issue_comment()` posts a standalone comment (used by `/ask` and as the
describe/diagram fallback). These three are the whole required surface, so a
new forge adapter can be useful long before it is complete; richer behaviour
(file reads, incremental review, thread resolution, labels, checks, feedback)
is declared by an optional `Supports*` capability protocol that callers probe
for and skip when absent.
- `ReviewEngine` — `review(ctx, cfg)` returns `(findings, summary)`.
The ports were frozen in the foundation step. Other tracks (providers, github,
engine, CLI) build against these stable signatures. Changing a port requires
consensus across all tracks.
## Code hosts (forges)
lgtmaybe calls a code host a **forge** internally, to keep it distinct from a
model *provider* — the two are separate choices. `core/forge.py` holds
everything host-specific outside the adapters themselves: a `Forge` enum, a
`PRLocator` (forge + host + repo + number) parsed from a change-request URL, and
the environment variable each host's token lives in.
A URL is all lgtmaybe needs to work out the rest:
| URL shape | Forge | Token |
|---|---|---|
| `github.com/org/repo/pull/42` | GitHub | `GITHUB_TOKEN` |
| `gitlab.com/group/sub/project/-/merge_requests/42` | GitLab | `GITLAB_TOKEN` |
| `gitea.example.com/org/repo/pulls/42` | Gitea | `GITEA_TOKEN` |
The host is carried, not assumed, because self-hosted GitLab and Gitea are the
normal case. `cli._GATEWAY_BUILDERS` maps the resolved forge to its adapter.
### Required surface vs optional capabilities
`ReviewGateway` is deliberately three methods. Everything richer is an optional
`Supports*` protocol that a caller probes for and skips when absent, so an
adapter can be useful long before it is complete — and, more importantly, so it
can be **honest** about what its host genuinely cannot do rather than failing at
run time.
| Capability | GitHub | GitLab | Gitea |
|---|---|---|---|
| `SupportsFileContents` — read head text for context | ✅ | ✅ | ✅ |
| `SupportsDescribe` / `SupportsDiagram` — upserted comments | ✅ | ✅ | ✅ |
| `SupportsLabels` | ✅ | ✅ | ✅ |
| `SupportsChecks` — check run / commit status | ✅ | ✅ | ✅ |
| `SupportsThreadResolution` — auto-resolve a fixed finding | ✅ GraphQL | ✅ REST | ✗ no API |
| `SupportsIncremental` — review only new commits | ✅ | not yet | ✗ no compare diff |
| `SupportsBaseCheckout` — clone base for symbol lookup | ✅ | ✗ | ✗ |
A test asserts the GitHub adapter satisfies every declared capability, and that
the others do **not** claim the ones their host cannot serve — which is what
stops the list drifting into fiction.
### Where the hosts genuinely differ
Gitea is nearly a copy of GitHub; GitLab is not.
- **GitHub** batches every inline comment into one review object it can later
edit, and resolves threads over GraphQL.
- **Gitea** mirrors GitHub's API, but a submitted review is **immutable**. So
the summary lives in an ordinary issue comment (which can be edited) and
findings are de-duplicated *before* posting, by reading the hidden ids already
on the pull request.
- **GitLab** has no batched review object at all: each finding is its own
**discussion**, positioned by old/new path and line plus the merge request's
`base_sha`/`start_sha`/`head_sha`. Threads resolve over plain REST.
lgtmaybe keeps GitHub's `RIGHT`/`LEFT` side vocabulary internally and translates
it at each adapter's boundary — `new_position`/`old_position` on Gitea,
`new_line`/`old_line` on GitLab.
### Entrypoints
GitHub and Gitea share one entrypoint: Gitea Actions reimplements the GitHub
Actions runtime, so the same `action` command and the same container work on
both. The only variable that differs is `GITHUB_SERVER_URL`, which Gitea points
at your own instance.
GitLab CI has neither an event payload file nor an `INPUT_*` convention, so it
gets its own `gitlab-ci` command that reads GitLab's predefined `CI_*`
variables. Everything downstream — locator, gateway registry, engine — is
shared.
## Review pipeline
The engine executes a pipeline of composable stages in sequence:
```
fetch → compress → prompt → parse → re-anchor → merge/dedupe → evidence gate → reflect → filter → post
```
The prompt/parse stage is where the pipeline fans out — one concurrent model
call per review lens — before the findings funnel back into a single stream:
```mermaid
flowchart TD
fetch["fetch
diff via API — never a checkout"] --> compress["compress
skip generated files · pad context · batch to budget"]
compress --> security["security lens"]
compress --> correctness["correctness lens
+ stated intent"]
compress --> codehealth["code-health lens
performance · complexity · ponytail · deprecation"]
compress --> artefacts["artefacts lens
tests · documentation"]
security --> anchor["re-anchor
snap lines to the real diff"]
correctness --> anchor
codehealth --> anchor
artefacts --> anchor
anchor --> dedupe["merge / dedupe"] --> evidence["evidence gate
defects need a failure scenario"] --> reflect["reflect
validate scenarios · drop low-confidence"] --> filter["filter
severity floor · finding rules"] --> post["post
inline comments + summary"]
```
(The four lens calls shown are the `fast` grouping, and they are the same four
on every provider — worker count decides only whether they overlap. The `full`
preset fans out one call per category, and custom lenses join the same fan-out.)
1. **fetch** — `ReviewGateway.get_pr_context()` retrieves the change diff and
metadata from the forge's API. No change code is checked out or executed.
The diff is treated as untrusted input throughout.
2. **compress** — the diff is filtered to remove generated files, lockfiles,
minified assets, and vendored code. Path filters from `ReviewConfig` are
applied. Each remaining hunk is then padded with surrounding context lines
from the head revision of the file (fetched by the gateway, never a
checkout), capped by `context_lines` and the remaining token budget. The
result is batched to fit `max_input_tokens` (and, when `recursive` is on, an
over-budget single file is walked hunk-by-hunk rather than sent whole). The
expanded diff is for the model only — inline-comment positions are always
rebuilt from the **real** diff at post time, so a finding on an added context
line maps to nothing and is dropped rather than mis-posted.
3. **prompt + parse** — this stage **fans out one model call per review
lens**. The `preset` decides the lens set. `fast` (the default) covers all
nine categories in **four calls**, one per concern: security, correctness
(with stated intent when present), merged code health (performance/
complexity/ponytail/deprecation), and artefacts (tests/documentation). The
same four run on every provider — worker count changes only how they are
scheduled. `full` runs one call per category. Every
(batch, lens) task shares **one `ThreadPoolExecutor`** over the sync
provider port, sized by `max_concurrency` (default 6 for every provider,
local included), so batches never wait on each other.
Each call is shaped as a shared cacheable prefix —
a lens-independent system preamble, then the wrapped diff — followed by
the lens-specific instruction as the final user block. On routes that
take an explicit cache breakpoint (anthropic, bedrock Claude/Nova, vertex
Claude and Gemini, zai GLM, and openrouter's claude/gemini/glm/minimax
families)
the prefix is marked with `cache_control`; on backends that cache
automatically (OpenAI, Azure, DeepSeek) the identical prefix is enough on
its own. On cache-capable routes, every call after a batch's first reads the
preamble-plus-diff prefix from cache. Unsupported routes use the merged-message
fallback and do not promise cache reads. On big diffs, a warm-up primer runs
the first lens alone so a concurrent wave doesn't all miss it.
Each lens's focused structured prompt requests JSON output with
the `ReviewFinding` schema (`path`, `line`, `side`, `severity`, `title`,
`body`, `failure_scenario`, `suggestion`, `anchor`) and carries
prompt-injection defense instructions. Each response is parsed and validated against
`ReviewFinding` using Pydantic; parse errors are logged and surfaced in
the summary rather than silently discarded.
With `mid_review_retrieval` on (default **off**), a lens gets a third option
besides asserting or hedging a cross-file claim: **ask to read the code**.
Alongside its findings it may answer `needs` — the file paths or symbols it
must see — and the engine fetches them through the same read-only, redacting
boundary reflection's deferral uses (never a checkout), then re-runs *that one
lens* with the text appended to its own uncached block. The shared prefix is
untouched, so the batch's other lenses still read it from cache. Bounded to
**one hop**: the re-run cannot defer again, at most five files inside a quarter
of `max_input_tokens` are fetched, and a deferral arriving past the review
deadline or token budget is skipped with the usual incomplete-results notice.
Both calls' findings are merged, so a deferral can only add findings. The cost
is up to one extra model call per (batch, lens) — which is why it ships off.
4. **re-anchor** — `_snap_findings` rebinds each finding's `line` to the real
changed line whose content matches the finding's verbatim `anchor`, rather
than trusting the model's line arithmetic. A finding whose anchor matches
nothing is marked `anchored=False` and later demoted to the review body
instead of being posted on a guessed line.
5. **merge/dedupe** — findings from every lens are merged and de-duplicated
(`_dedupe`, keyed on path/line/side).
6. **evidence gate** — security, correctness, deprecation, and performance
findings must carry a concrete `failure_scenario`: the trigger, changed
behaviour, and observable impact. The engine applies this rule regardless
of model-selected severity, so lowering a finding to `low` cannot bypass it.
Gap and maintainability findings remain eligible without a runtime failure.
7. **reflect** — a self-reflection pass (`engine/reflect.py`) asks the provider
to audit its own findings and drops the ones it marks low-confidence
(keep-all safe default when the verdict can't be parsed; skippable with
`--no-reflect`). It also tries to disprove each claimed failure scenario
against the diff and grounded file text. When the auditor would drop a finding *only* because it
can't see code outside the diff, it **defers** by naming what it needs — a
file path or a **symbol**. A path is fetched read-only
(`get_file_contents`). A symbol is located by **ast-grep**
(`engine/astgrep.py`), which structurally searches a corpus — the local
worktree for the CLI, or a read-only shallow clone of the trusted **base**
branch for the GitHub path — for the file that *defines* it. That file is
then fetched through the same read-only boundary, and the auditor re-judges
with the real definition in front of it instead of guessing about an unseen
guard or base class.
This stays inside the fork-safety model: ast-grep only *parses* the corpus
(never executes it), and the base clone is never the PR head. Symbol
resolution needs the bundled `ast-grep` binary and a corpus; without either
it degrades to the path-only fetch (`--no-symbol-resolution` disables it
entirely). It is bounded by the same hop/file caps as the path fetch.
8. **filter** — findings below `min_severity` are dropped.
9. **post** — findings are batched into a single GitHub review request.
The summary comment is updated idempotently using a hidden marker, so
re-running lgtmaybe on the same PR does not create duplicate comments. Each
inline comment is stamped with a hidden per-finding fingerprint; on a re-run,
conversations whose finding is gone and whose thread GitHub marks outdated are
replied to and resolved (`resolve_fixed`, default on). Resolving a review
thread is the one operation the REST review API can't do, so this step uses
GitHub's GraphQL API — best-effort, so a failure never blocks the review.
## Provider strategy and factory
Provider selection uses the **strategy pattern**: `--provider` picks a
`ProviderClient` strategy; a small factory constructs it. litellm normalises
all providers to one `completion()` call shape, so the factory is small and the
engine is provider-agnostic.
Credential resolution uses a **chain of responsibility**: each provider knows
how to locate its own credentials (ambient cloud creds, env var API key, or
none for ollama). lgtmaybe never stores or logs credentials.
## Reliability: retries, timeouts, and concurrency
The provider wrapper (`LiteLLMProvider`) and the engine cooperate so a flaky
network recovers but a dead-end failure surfaces fast:
- **Retries are classified, not blanket.** Transient failures — capacity rate
limits (`429 rate_limit_exceeded`), the provider's own connect/read timeouts,
connection errors (e.g. an
ollama server still warming up), 5xx — are retried with **exponential backoff
and jitter** (up to four attempts). **Permanent** failures are *not* retried:
bad credentials (`AuthenticationError`), malformed/unsupported requests
(`BadRequestError`, including content-policy blocks), unknown models
(`NotFoundError`), denied permissions, and **quota/billing** rate limits
(`429 insufficient_quota` — "you exceeded your current quota"). Retrying a
quota error can never succeed; stacked across every lens it only turns an
instant "out of credit" into many minutes of wasted runner time, so lgtmaybe
raises it immediately. An optional `fallback_model` is still tried once.
- **A blown wall clock is permanent too.** When a call outlives lgtmaybe's own
per-request timeout (below), the retry would re-send the identical messages to
the identical model against the identical budget — it can only fail the same
way and cost another full timeout doing it. So the wall timeout is raised after
one attempt, and the failure reports how many attempts it burned (the count
rides home on failures as well as successes, so a budget-burning call never
reads as one that was never retried). The `fallback_model` — a genuinely
different request — still gets its turn, with a fresh budget.
- **Retried smaller, not repeated.** A wall timeout says something about the
*payload*, not the provider, and the payload is the one thing the engine can
change. So the batch is split and its pieces reviewed instead — halved by file,
or by hunk when the batch is a single file — each piece with its own fresh
budget. That keeps a slow lens from discarding the whole batch's review, and it
is the only retry a blown budget can benefit from. Bounded to **one** level: a
piece that times out again just fails, so a model that cannot answer at any
size can't cascade through the review budget. The summary says when a batch had
to be shrunk — a silent split would hide that every run is at the edge of what
the model finishes in time.
- **A second model is the last remedy for a truncation, not the first.** A
truncated lens has two remedies that stay on the model you chose: shrink the
payload (the split above) when the *answer* outgrew the ceiling, and lower
`reasoning_effort` when the *thinking* did. Both are cheap and both act on
what the token counts actually said went wrong. Switching to `fallback_model`
says nothing about the failure — it re-sends the same request at the same
ceiling and hopes a second model finishes it — so it runs only after the aimed
remedy has been tried and failed, once, for the whole batch rather than once
per split piece. Every other failure the adapter still rescues by itself, and
either way the summary names the lens and the model that answered it: a review
the primary could not finish must not read like one it did.
This is also the routing rule to reach for before one keyed on diff size.
Truncation does not track size — measured here, a fifteen-line diff truncates
at the same ceiling as a large multi-file one, because the ceiling goes on
thinking — so a size threshold spends the strong model on the diffs that were
already fine. A fallback is keyed on the failure itself and costs the second
model nothing until one happens.
- **A rate limit waits on its own, much slower ladder.** The general ladder
starts at a tenth of a second, which is right for a blip — a connection reset,
an ollama server warming up — because the condition is gone by the time the
next request lands. A capacity 429 is not a blip: the gateways that meter an
API key meter it *per minute*, so a ladder that fits all four attempts inside a
few seconds lands all four in the same window, where they can only fail
identically. Rate limits therefore back off on a 5s–60s ladder instead, and
prefer the server's own **`Retry-After`** when it sends one (in either RFC
form, clamped at 120s so a gateway asking for an hour can't eat the run's wall
clock). Both ladders stay inside the same 2.5× retry budget below — which is
weighed against the *upcoming* wait, so a backoff that would blow the budget
ends the call instead of being slept.
- **A call that failed on the provider gets one more go.** Once the fan-out has
drained, the calls that failed *provider-side* are re-run once, in a narrow
pool of their own. This is the difference between a review and a partial one:
a single flaky call used to void the whole round's verdict, and the findings
that lens would have made were lost until somebody re-ran by hand. Nothing
else is left alone, because a second attempt would repeat the same failure at
full price: unparseable output returns the same unparseable answer at
temperature 0, a truncation runs to the same output ceiling, a batch already
retried by the split (below) has had the one retry that can help, a dead key
or spent quota cannot resolve itself mid-review, and a ceiling you set is not
a fault to retry past. Every rescue re-checks the deadline, the
token budget and the interrupt first, and a healthy run costs **zero** extra
calls.
- **One retry layer.** litellm's own internal retry loop is disabled
(`num_retries=0`) so failures aren't ground through two stacked backoff layers
— lgtmaybe owns the retry policy in one place.
- **Per-request timeout and a shared retry budget.** Every model call carries a
timeout: 600s for direct cloud providers, 1800s for the ones that may front a
slow model — ollama and openai-compatible (local servers) and openrouter (a
gateway to arbitrary models, including slow reasoning ones) — overridable via
`timeout` / `--timeout`. All attempts for
one call additionally share a wall-clock budget of **2.5× that timeout**, so
a flaky model can't burn four full timeouts plus backoff per lens. The
posting workflows additionally set a job-level `timeout-minutes` so a wedged
run can't hold a runner for GitHub's six-hour default — set above
`max_review_seconds`, so the soft deadline (which still posts partial
findings) fires before the runner is killed. If the runner does cut the job
short — a blown `timeout-minutes`, or `cancel-in-progress` on a new push — the
CLI's signal handler is the backstop (below).
- **A whole-review deadline.** `max_review_seconds` (default 3600, `0` disables)
is a soft ceiling on the run: once it passes, queued model calls are skipped
— in-flight ones finish and their findings post — and the summary carries an
explicit incomplete-results notice. It can never produce a silent LGTM: a
run where every call failed or was skipped still fails loud.
- **A termination signal does the same thing.** The CLI installs a
SIGINT/SIGTERM handler that sets the deadline's state, so a job the runner
cancels or times out stops dispatching model calls and posts what it already
has, with a notice naming the interruption, instead of dying with nothing on
the PR. The handler is installed by the CLI entrypoint only (importing
lgtmaybe as a library never touches your signal handlers), and it restores the
previous handler as it fires — a second signal still kills the process.
- **Incompleteness is visible on the PR, not just in the log.** The notice above
is not specific to the deadline: *any* failed lens call (a per-call timeout, a
provider error, unparseable output) raises it — naming the lenses it lost and
why, since a missing security lens and a missing documentation lens are the
same count and very different news — and the engine stamps a hidden
`` marker alongside it. Because a re-run can only
update the *first* run's review body in place — a silent edit, while this run's
new findings arrive as individual review comments GitHub wraps in bodyless
reviews — an incomplete run also posts the notice as its own PR comment. A
half-complete review must never look like a clean one. A review that failed
outright posts its failure notice the same way. The summary also carries a
hidden `` list of the lenses that completed every
call, so a gate that wants "every lens ran" can require the set it expects
instead of inferring it from the absence of a notice — a lens that never ran
leaves none to find. A matched spec that did not fit the spec budget is named
the same way, because a spec lens that should have run and did not is a gap,
not a no-match.
- **A diff the host will not serve is a review that did not happen.** GitHub
answers the diff request with 406 once a PR passes 300 files or 20,000
lines, and lgtmaybe reads PR content through that API only — never a
checkout — so there is nothing to review and no retry that changes it. That
run posts a "not reviewed" notice quoting GitHub's reason, with a hidden
`` marker, stamps nothing complete, and exits
zero: not a failure (a red job for a PR the reviewer could never look at
only teaches people to ignore red jobs), and never an LGTM. The fix is a
smaller PR; no reviewer setting moves the host's limit.
- **One global fan-out pool.** Every (batch, lens) call runs through a single
`ThreadPoolExecutor` sized by `max_concurrency` — default **6 workers** for
hosted providers (an extra worker can cut a full-latency wave off the wall
clock — see the formula below — but the fan-out is *one* API key, and past
some width the burst
rate-limits itself against a per-minute-metered gateway; raise it if your rate
tier is generous), and the same **6** for local
providers. Local used to be pinned to 1 because a default ollama serves one
request at a time — but that is a property of the *server*, not a reason to cap
the client: a server that can batch was capped for nothing, and one that cannot
loses nothing by having work queued for it (ollama queues up to
`OLLAMA_MAX_QUEUE` rather than failing). What decides local throughput is
`OLLAMA_NUM_PARALLEL` / llama.cpp's `-np` / vLLM's batching, and raising those
costs memory per slot — see
[running locally](../how-to/run-locally-with-ollama.md). Drop this to 1 for a
very slow local model, where six queued calls could each wait out the timeout. Flattening
the pool across batches means wall time is `ceil(batches × lenses / workers)`
call-latencies rather than `batches × ceil(lenses / workers)`.
## Dependency injection
The engine receives its ports by injection. In production the CLI wires real
adapters; in tests `tests/fakes/` provides drop-in fakes. No monkey-patching or
`unittest.mock` is needed at the engine level.
## Why not a plugin framework or event bus
Both were considered and explicitly skipped. The current set of providers fits
cleanly in a strategy + factory; a plugin registry would add indirection with no
present benefit. An event bus would complicate the linear pipeline without
enabling any feature the product needs. These can be revisited if a concrete
requirement arises.
---
# Auth Model
lgtmaybe supports seven hosted providers plus local ollama, and an
`openai-compatible` escape hatch for any server speaking the OpenAI `/v1` wire
format (DeepSeek's API, llama.cpp, LM Studio, vLLM). The design principle is **no
static cloud credentials**: cloud providers use ambient, short-lived tokens;
key-based SaaS providers (openai, anthropic, openrouter, zai) require an API key that
stays in secrets rather than being committed to config. Azure straddles both:
it prefers ambient Azure AD (Entra) credentials but accepts a resource key. It
always needs the resource endpoint (`AZURE_API_BASE`), since each Azure OpenAI
deployment lives at its own URL.
## Why keyless for cloud
Static credentials — an AWS access key pair or a GCP service-account JSON file
— have a fixed lifetime and broad scope. If they leak from a CI log, a secrets
manager misconfiguration, or a compromised runner, an attacker retains access
until the key is manually rotated.
OIDC (AWS), Workload Identity Federation (GCP), and federated credentials on an
Entra app (Azure) issue tokens that:
- Expire in minutes, not years
- Are scoped to a single workflow run
- Cannot be extracted from CI logs (they are never set as static environment
variables)
- Are tied to the specific repository and branch via the OIDC claim set
The exchange is the same shape on all three clouds — no secret ever enters the
workflow:
```mermaid
sequenceDiagram
participant W as GitHub Actions run
participant O as GitHub OIDC issuer
participant STS as Cloud token service
(AWS STS / GCP STS / Entra)
participant M as Model API
(Bedrock / Vertex / Azure OpenAI)
W->>O: request identity token (id-token: write)
O-->>W: signed JWT with repo + branch claims
W->>STS: exchange JWT for cloud credentials
STS-->>W: short-lived, scoped credentials
W->>M: lgtmaybe review calls with ambient creds
```
For these reasons, lgtmaybe treats Bedrock and Vertex as **ambient-credential
only** providers. There is no `--api-key` flag for them. Azure defaults to the
same keyless path (GitHub OIDC → Entra, via `azure-identity`'s
`DefaultAzureCredential`) but additionally accepts a resource key for teams that
can't yet adopt federation.
## Chain of responsibility
When lgtmaybe starts, it resolves credentials for the selected provider using a
chain:
1. **Provider identity** — which provider was chosen?
2. **Ambient cloud creds** — for Bedrock and Vertex, the chain stops here.
If no ambient creds exist, lgtmaybe fails immediately with a clear
"how to auth this provider" message. It does not fall back to a key.
3. **API key** — for openai, anthropic, openrouter, and zai (GLM / Zhipu AI),
lgtmaybe reads the key from the environment (`OPENAI_API_KEY`,
`ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`, `ZAI_API_KEY`). The `--api-key` flag
can override this at the CLI. `zai` also accepts an optional `--api-base` /
`ZAI_API_BASE` override for the China / coding-plan GLM endpoint; without it
litellm's native `zai/` route targets the international endpoint.
4. **Azure** — always needs the resource endpoint (`AZURE_API_BASE` or
`--api-base`). For the credential it prefers a key when one is present
(`AZURE_API_KEY` / `--api-key`); otherwise it goes **keyless**, minting an
Azure AD token from ambient creds (GitHub OIDC federation in CI, or a local
`az login` / managed identity). If neither a key nor an ambient credential is
available, lgtmaybe fails with a message naming both options.
5. **None** — ollama requires no credentials. Only `--api-base` is needed
to reach the local or remote server.
6. **openai-compatible** — always needs the endpoint (`--api-base` or
`OPENAI_COMPATIBLE_API_BASE`), since the whole point is to choose your own.
The key is **optional**: hosted endpoints like DeepSeek take one
(`--api-key` / `OPENAI_COMPATIBLE_API_KEY`), while local servers (llama.cpp,
LM Studio, vLLM) need none — lgtmaybe sends a harmless placeholder key when
you supply none, because the underlying OpenAI client rejects an empty one.
## Provider auth summary
| Provider | Credential type | How to supply |
|---|---|---|
| openai | API key | `OPENAI_API_KEY` env var or `--api-key` |
| anthropic | API key | `ANTHROPIC_API_KEY` env var or `--api-key` |
| openrouter | API key | `OPENROUTER_API_KEY` env var or `--api-key` |
| zai | API key (+ optional endpoint) | `ZAI_API_KEY` env var or `--api-key` (GLM / Zhipu AI); optional `ZAI_API_BASE` / `--api-base` for the China / coding-plan endpoint |
| bedrock | Ambient AWS creds | GitHub OIDC role or local `~/.aws`; IAM requires only `bedrock:InvokeModel*` |
| vertex | Ambient GCP creds | GitHub WIF or local ADC (`gcloud auth application-default login`) |
| azure | Ambient Azure AD creds (keyless) or API key, + endpoint | GitHub OIDC → Entra federated credential, or local `az login` / managed identity; or `AZURE_API_KEY`. Always with `AZURE_API_BASE` / `--api-base` |
| ollama | None | `--api-base` pointing to the local or remote server |
| openai-compatible | Optional key, + endpoint | `--api-base` / `OPENAI_COMPATIBLE_API_BASE` (e.g. `https://api.deepseek.com/v1` or `http://localhost:8000/v1`); key via `--api-key` / `OPENAI_COMPATIBLE_API_KEY`, or none for keyless local servers |
## Code-host auth
Everything above authenticates lgtmaybe to a **model provider**. Reaching the
change itself — reading the diff, posting the review — is a separate credential,
resolved from the host in the change-request URL:
| Host | Variable | What it needs |
|---|---|---|
| GitHub | `GITHUB_TOKEN` | `contents: read` and `pull-requests: write`. The Action's default token is sufficient |
| GitLab | `GITLAB_TOKEN` | A project access token with the `api` scope, Developer role or above |
| Gitea | `GITEA_TOKEN` | A token with `read:repository` and `write:issue` |
These are always tokens — there is no keyless path to a code host, and lgtmaybe
does not attempt one. The token is read from the environment only; like a
provider key it is never persisted to `.lgtmaybe.yml` or logged.
!!! note "Keyless cloud auth is GitHub-only"
Bedrock, Vertex, and Azure go keyless by exchanging a **GitHub Actions**
OIDC token. GitLab CI and Gitea Actions have no equivalent exchange wired
up, so on those hosts use an API-key provider — or `ollama` against a
runner-local model, which needs no credential at all.
## Least-privilege IAM
For Bedrock, the minimum IAM policy is `bedrock:InvokeModel` and (if streaming)
`bedrock:InvokeModelWithResponseStream` on the specific model ARN. No other AWS
permissions are needed or requested.
For Vertex, `roles/aiplatform.user` on the project is sufficient.
`roles/editor` or `roles/owner` must not be used.
For Azure (keyless), the Entra app needs only the **Cognitive Services OpenAI
User** role on the Azure OpenAI resource — enough to call deployments, not to
manage them — plus a federated credential scoped to your repository. No owner or
contributor role is required.
## API keys in secrets, not config
For openai, anthropic, and openrouter, the key must live in a secret store —
a GitHub Actions secret, a GitLab masked CI/CD variable, or a Gitea Actions
secret. It must never be committed to
`.lgtmaybe.yml` or any other file in the repository. lgtmaybe does not log or
display key values.
---
# Data and Privacy
This page explains what lgtmaybe sends to a model provider, what it redacts
first, how local mode differs, and how credentials are handled.
## What is sent to the LLM provider
lgtmaybe makes four core review calls per diff batch under the default `fast`
preset, or one per built-in category under `full`. A matching committed spec
adds a separate spec-review call. Calls share a concurrency limit and are
followed by a reflection pass. They can contain:
- The **compressed PR diff** — the unified diff of changed files, after
generated files, lockfiles, minified assets, and vendored code have been
stripped.
- **Surrounding context lines** — a budget-scaled number of unchanged lines
immediately above and below each changed hunk, read from the head revision of
the **changed files only**. This gives the model the surrounding function and
definitions so it makes fewer false-positive findings. The amount is capped by
`context_lines` (default 20, `0` disables it) and shrinks as the diff grows;
this content is redacted just like the diff. On GitHub, it is fetched
read-only via the API. Pull request code is never checked out or executed.
- **PR metadata** — the repository name, PR number, base and head SHAs, and
the list of changed file paths.
- **The PR's stated intent** — the PR title, description, and the first line of
each commit message (on the CLI: the commit names from your local `git log`).
This feeds the **intent lens** ("does the PR do what it says?"). It is
redacted exactly like the diff, wrapped as untrusted data, and sent **only on
the single lens call that carries the intent** (the correctness call under
the default `fast` preset, or the dedicated intent lens under `full`) — drop `intent` from `categories` in
`.lgtmaybe.yml` and it is never sent at all.
- A **system prompt** — the fixed instructions that tell the model to return
structured JSON findings.
- **Configured instructions and reference files**, if `directory_rules` is
set. Reference files come from the trusted base-branch workspace, are
bounded, and are redacted before model calls.
- **Matching specification text**, when a repository has a supported spec
workflow and the change matches a committed spec. The spec lens sends a
bounded, redacted selection from that spec.
- **Additional file context**, when a review or reflection needs to check a
finding against another file. Retrieval is read-only, bounded, and redacted;
`mid_review_retrieval` is opt-in.
Review calls do not send:
- PR comments or review threads
- Commit message bodies (only the first line of each message)
- Unrequested repository files or the whole repository
- Committer identity or email addresses
- Any other data from the repository's git history
The **change overview** (`/diagram`, `auto_diagram`) makes up to three calls —
description, High Impact Areas, diagrams — using the redacted diff and, when
present, the redacted stated intent. The High Impact Areas call also sends the **paths** of
changed files matching its risk patterns, which are already part of the diff it
sends. No new data flows; the calls only ask the model for a different output.
The standalone `/describe` comment sends the same inputs again. The diagram comment
does include an "Open full screen" [mermaid.live](https://mermaid.live) link per
diagram, whose URL fragment embeds that diagram's source — but that source is the
already-public, post-redaction comment body, the fragment is decoded
client-side (browsers never send fragments to a server), and nothing is fetched
unless a reader clicks the link.
## Secret redaction before egress
Before the diff is sent to any external provider, lgtmaybe scans it for
patterns that resemble secrets and replaces the matched values with
`[REDACTED]`. The same scrub is applied to the surrounding context lines read
from changed files and to the stated-intent text (PR title, description, commit
names). Recognised formats include:
- **Cloud / provider keys** — AWS access key IDs (`AKIA…`), OpenAI keys
(`sk-…`), Stripe secret keys (`sk_live_…`), and Google API keys (`AIza…`).
- **Source-control / chat / registry tokens** — GitHub classic tokens
(`ghp_`, `gho_`, …), GitHub fine-grained PATs (`github_pat_…`), Slack tokens
(`xoxb-…`), npm tokens (`npm_…`), and PyPI tokens (`pypi-…`).
- **JSON Web Tokens** — `eyJ….eyJ….…` (the whole token, since the payload
carries claims/PII).
- **Private keys** — PEM `-----BEGIN … PRIVATE KEY-----` blocks.
- **Generic credentials** — `api_key`/`token`/`secret = "…"` assignments,
quoted `password`/`passphrase` literals, `Authorization: Bearer/Basic …`
headers, passwords embedded in connection-string URLs
(`scheme://user:secret@host`), and Azure storage / Cosmos connection-string
keys (`AccountKey=…` / `SharedAccessKey=…`).
For credential assignments only the value is replaced — the key name or URL host
stays readable so the reviewer can still reason about the change.
This happens as the **first** pipeline stage, before the diff is compressed or
the prompt is built, so redacted values never reach the LLM or appear in logs.
Redaction is a best-effort defence. Do not commit real secrets to your
repository and rely on this alone.
## Model replies in the log
When a model's reply cannot be parsed into findings, lgtmaybe logs why. At the
default log level it records only **which** parse failure it was (`prose`,
`malformed_json`, `not_findings`, `schema`, `truncated`, `empty`) and **how many
characters** the reply was — never its content.
Set `LGTMAYBE_LOG_LEVEL=DEBUG` and the log line additionally carries a capped
excerpt of the reply itself (the first 2,000 and last 500 characters, with the
gap marked). This exists so a model that stops honouring the output schema can
be diagnosed without re-running the review. The excerpt:
- is passed through the same secret redaction described above before it is
logged, and is redacted **before** it is cut, so a truncated match cannot slip
past the redactor;
- is written to **stderr**, alongside the other structured JSON log lines —
never to stdout (the machine-readable channel) and never to the PR;
- is model output, which can quote the diff back. That is why it is off unless
you ask for it.
## The repair re-ask
When a review call's reply cannot be parsed into findings, lgtmaybe sends that
reply back to the **same provider** once, with the output schema and a request
to re-express it in the required shape (`repair_unparseable`, on by default).
This adds no new data: the reply is the model's own output about a diff it has
already been sent, and the repair call carries **no diff, no context lines, and
no PR metadata** — only the reply itself, capped and wrapped as untrusted data.
It happens at most once per failed call and never recursively.
## The schema-less re-run
When that reformat fails too *and* the original call carried the provider's JSON
schema, lgtmaybe re-runs that one lens once with the schema omitted
(`retry_without_schema`, on by default) — the prompt asks for JSON regardless,
so a model whose constrained decoding produced something unreadable often
answers fine without it.
Unlike the repair, this **does re-send the diff** — it is the same request minus
one parameter, to the same provider, so nothing new leaves the process and
nothing new is exposed, but it is a second full send rather than a small one. It
runs at most once per lens per batch, only after the cheaper repair has failed,
and only where a schema was actually sent. Set `retry_without_schema: false` to
turn it off.
## Prompt-injection defence
PR diff content is treated as untrusted input throughout the pipeline. lgtmaybe
defends in depth (OWASP LLM01):
1. The diff is wrapped in explicit `DIFF_START`/`DIFF_END` delimiters and labelled
as untrusted data; the stated-intent text gets its own
`INTENT_START`/`INTENT_END` block with the same labelling.
2. Any forged delimiter markers smuggled inside the diff or the intent text are
**neutralised** before wrapping — both marker families in both blocks — so a
malicious PR cannot close a data block early, append its own instructions, or
fake an intent block from inside the diff.
3. The system prompt instructs the model to ignore any instructions embedded in
the diff or the intent text that attempt to alter reviewer behaviour.
4. The model's response must validate against a strict JSON schema
(`extra="forbid"`); drifted or injected fields are rejected rather than acted
on.
lgtmaybe does not execute any code from the PR.
## Ollama: local or self-hosted
When `--provider ollama` is used, the diff and all other data are sent only to
the ollama server you specify via `--api-base`. If that server is
`http://localhost:11434`, no data leaves your machine. If it is a remote host
(Tailscale peer, self-hosted VM), data is sent only to that host.
Ollama itself is not operated by lgtmaybe. Review ollama's own documentation
for its data handling.
## Cloud providers: data handling
When using Bedrock or Vertex, the compressed and redacted diff is sent over
HTTPS to the respective cloud provider's inference endpoint. Review each
provider's data handling policies:
- **AWS Bedrock** — [AWS Bedrock data protection](https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html)
- **Google Vertex AI** — [Vertex AI data governance](https://cloud.google.com/vertex-ai/docs/general/data-governance)
- **OpenAI** — [OpenAI API data privacy](https://openai.com/policies/api-data-privacy)
- **Anthropic** — [Anthropic usage policy](https://www.anthropic.com/legal/aup)
- **OpenRouter** — [OpenRouter privacy policy](https://openrouter.ai/privacy)
## Credentials
lgtmaybe does not log or persist API keys. API-key providers receive the key
as part of authentication for model calls. For Bedrock and Vertex, lgtmaybe
uses short-lived ambient credentials and does not write them to disk. See
[Auth Model](./auth-model.md) for details.
## GitHub token
`GITHUB_TOKEN` is used to:
1. Read the PR diff and metadata via the GitHub REST API.
2. Post the review (inline comments + summary) back to the PR.
The token is not sent to any LLM provider. It requires the minimum scopes:
`contents: read` and `pull-requests: write`.
## Fork pull requests
On GitHub, lgtmaybe uses the `pull_request_target` trigger, which runs in the
context of the **base branch**. PR code from the fork is never checked out or
executed. The diff is fetched exclusively through the host's API. This prevents
a malicious PR from gaining access to repository secrets.
GitLab and Gitea have no `pull_request_target` equivalent — their merge- and
pull-request pipelines run with the project's own secrets. The protection is
unchanged and holds for the same reason: lgtmaybe never checks out or executes
change-request code on any host, so there is nothing for an attacker to run. The
supplied GitLab job even sets `GIT_STRATEGY: none`, so the repository is never
cloned at all.
---
# Trust and Cost
lgtmaybe lets you decide **who can trigger a review**. Hosted models charge for
each run, so this choice affects cost. The setup steps are in
[Use as a GitHub Action](../how-to/use-as-github-action.md).
## Who do you want reviews to run for?
There's no single right answer — it depends on your repo and provider:
- **Everyone**, including strangers opening fork PRs — great for a welcoming
open-source project where you want every contributor to get feedback.
- **Trusted contributors** — members and collaborators, the people who already
have a relationship with the repo. This is the default.
- **Admins / owners only** — the tightest setting, handy while you're trying
lgtmaybe out.
The example workflows ship with the **trusted contributors** setting, and it's a
one-line change to widen or narrow it.
## The small cost angle
The only reason this is worth a thought at all is that hosted providers bill per
run:
- **ollama is free** — it runs the model on your own hardware, so trigger it for
whoever you like; there's no per-run cost.
- **Hosted providers** (OpenAI, Anthropic, OpenRouter, z.ai, Bedrock, Vertex,
Azure) charge for the tokens each review uses.
So if you're on a hosted provider and your repo is public, "everyone" means
anyone can start a run. That's perfectly fine for plenty of projects — just pick
it deliberately rather than by accident. The default keeps reviews to people you
already trust, which is a sensible starting point you can open up whenever you
want.
Two built-in caps also keep any single run modest regardless of who starts it:
`max_files` (default 50) and `max_input_tokens` (default 100k). See
[What gets reviewed](what-gets-reviewed.md) for how the diff is bounded.
When a run does bump into a cap, it **degrades rather than fails**. A review call
that outruns its wall clock, or writes until it hits the `max_tokens` ceiling, is
not simply lost: the batch was more than one call could cover, so lgtmaybe halves
it and reviews the pieces, keeping anything the model had already written. That
costs up to twice the calls for the affected batch, and it is bounded to one
split — a piece that still runs past the ceiling is reported rather than split
again. Whatever the outcome, a partial review says so: the summary carries a
"results may be incomplete" notice, and a lens that was cut short is never
presented as a clean bill of health. The one thing lgtmaybe will not do is answer
👍 LGTM for code no model finished reading.
To measure what a run actually spends and bring it down — triage, static
analysis, prompt caching, and a hard `max_review_tokens` ceiling — see
[Reduce review cost](../how-to/reduce-review-cost.md).
## How the choice is expressed
The example workflows gate the `review` job on the triggering user's
[author association](https://docs.github.com/en/graphql/reference/enums#commentauthorassociation)
— GitHub's classification of a user's relationship to the repo. By default the
job runs when that association is `OWNER`, `MEMBER`, or `COLLABORATOR`:
```yaml
if: >-
(github.event_name == 'pull_request_target' &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) ||
(github.event.issue.pull_request &&
contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) &&
(contains(github.event.comment.body, '/review') ||
contains(github.event.comment.body, '/improve') ||
contains(github.event.comment.body, '/ask') ||
contains(github.event.comment.body, '/describe') ||
contains(github.event.comment.body, '/diagram')))
```
A maintainer can still review an outside contributor's PR any time by commenting
`/review` on it — their own association passes the gate. To change the policy,
edit the list:
- **Open it up to everyone** — drop the author-association checks so any PR or
`/review` comment runs a review.
- **Welcome returning contributors** — add `CONTRIBUTOR` to auto-review anyone
whose PR has merged before.
- **Tighten to admins** — keep just `OWNER` (and `MEMBER` for your org).
Because the gate lives in the workflow YAML, not in the action's code, the policy
is entirely yours.
The second half of the comment arm is a different kind of thrift. `issue_comment`
fires on every comment on every pull request, and lgtmaybe's answer to one that
holds no slash command is to exit — correct, and free of tokens, but only after a
runner has been claimed, a container pulled and Python booted. Deciding that in
the `if:` costs nothing at all, and on a busy runner pool it is the difference
between a review starting now and a review queueing behind comment threads. The
clause is deliberately looser than the parser it stands in for — substring, not
prefix, and case-insensitive — because a guard tighter than the parser would
silently disable a command that used to work. Replies inside a finding thread
do not invoke the model; a pushed fix is checked by the next incremental review.
## If you want extra guardrails
Optional, for repos where you want belt-and-braces:
- Require approval for fork-PR workflow runs in
**Settings → Actions → General → Fork pull request workflows**.
- Put the provider key behind a protected `environment`.
- Set a spending limit in your provider console.
## Reviews are safe to run for anyone
Whoever triggers a review, a malicious PR can't use it to do harm: on GitHub
lgtmaybe triggers on `pull_request_target` (so it has the secrets it needs) but
**never checks out or executes PR code** — it fetches the diff through the
host's API and treats it as untrusted input. That second half is what actually
does the work, and it holds identically on GitLab and Gitea, which have no
`pull_request_target` of their own. So opening the gate wide is a cost decision, not a
security one. The full boundary — secret redaction, prompt-injection defence, and
fork safety — is in [Data and Privacy](data-and-privacy.md).