Limits & skips
Most of the engineering in a code reviewer goes into deciding what not to say. This page is the complete list of what we refuse to look at, what we cap, and how each decision is reported.
The principle
A skip is never silent. Every rule below either posts a comment saying it applied, or is visible in @superdiffs config. A reviewer that goes quiet without explanation is indistinguishable from a broken integration, and teams debug it by uninstalling.
Files we never comment on
These are decided before any model is called, so a skipped file costs nothing and — more to the point — can never produce a comment.
| Rule | What matches | Why | Override |
|---|---|---|---|
| Lockfiles | package-lock.json, pnpm-lock.yaml, go.sum, Cargo.lock, Gemfile.lock and about twenty more, by exact filename wherever they sit in the tree. | A model asked to review four thousand lines of resolved dependency graph will always find "something". None of it is ever a defect you would fix by editing the lockfile. | Not overridable. Dependency risk is covered by the advisory scanners instead, which read the lockfile without commenting on it. |
| Generated code | Paths that only generators write (*.pb.go, *.gen.ts, *.freezed.dart, dist/, __generated__/) and any file carrying a generator banner — @generated, Code generated by …, DO NOT EDIT — in its first 30 lines. | Generated code is reviewed by reviewing the generator input. A comment on the output is a comment nobody can act on. | Re-include a specific path with a non-! pattern in paths.filters; the marker check still applies to file content. |
| Vendored trees | Directory segments named vendor, node_modules, third_party, Pods, bower_components and similar. Only directory segments count — a file called vendor.ts is yours. | It is somebody else’s code, checked in. Reviewing it produces findings you cannot fix in this repository. | Re-include with an explicit paths.filters pattern if you genuinely maintain a fork in-tree. |
| Minified or bundled output | Detected by shape rather than by name: a single line over 500 characters, or a mean line length over 200. | Bundles have no reviewable structure, and they are the fastest way to burn a review budget on nothing. | Not overridable. |
| Binary files | Anything Git reports as binary: images, archives, compiled artefacts, fonts. | There is no diff to read. | Not overridable. |
| Your own path filters | Whatever paths.filters excludes, in every layer that applies to the pull request. | You said so. | Edit paths.filters. Among patterns that match, the last one wins, so a later layer can re-include a path an earlier one excluded. |
| Deleted files | Files removed by the pull request. | A comment on a line that no longer exists cannot be anchored, and "you deleted this" is not a finding. Deletions still inform the review of the files that remain — a removed guard is a defect in the caller. | Not overridable. |
| Mode and metadata-only changes | Permission-bit changes, pure renames with no content edit, and empty diffs. | There is nothing to read. | Not overridable. |
Skipped files are still read where they inform the review — a lockfile tells the dependency scanners what changed, and a deleted file explains why the caller now breaks. The rule is about where a comment may land, not about what we look at.
Pull requests we skip entirely
| Condition | What happens | Setting |
|---|---|---|
| The pull request is a draft | No review. Marking it ready for review triggers one immediately. | review.drafts: true |
| The title contains an ignore keyword | No review, no summary, no check run. | review.ignore_title_keywords (default wip, [skip review]) |
| The author is on the ignore list | No review. Typically used for bots that open mechanical pull requests. | review.ignore_authors |
| The base branch is not in scope | No review. | review.base_branches (empty means every branch) |
| The description contains `@superdiffs ignore` | No review, permanently, for this pull request. | The only command honoured from a description. |
| Automatic review is off, or paused | Nothing happens on push; explicit commands still work. | review.auto: false, or @superdiffs pause |
| Several pushes with no reply from anyone | Automatic review pauses itself and says so once. A reviewer nobody is reading should stop talking. | review.auto_pause_after_commits (0 disables) |
| More changed files than the limit | Summary and a walkthrough are posted, findings are not. The comment explains how to opt in, and `@superdiffs full-review` overrides it for that pull request. | review.file_change_limit |
| Every changed file was skipped by the rules above | A short comment naming what was skipped and why. Silence with no explanation reads like a broken integration. | — |
Comment budget
Every review has a maximum number of line comments — 20 at the default profile. When findings exceed it, they are dropped from the bottom:
- Anything below
severity_thresholdnever enters the budget in the first place. - Then
nits are collapsed into a single collapsed section, which costs one comment rather than fifteen. - Then the remaining findings are ordered by severity and, within a severity, by the confidence that survived verification. The tail is cut.
- The summary comment states how many findings were suppressed, so a cut budget is a disclosed fact rather than a missing one.
Raising the budget is one line. It is usually the wrong fix: if a review is hitting the cap regularly, either the pull requests are too large or the threshold is too low, and both are better problems to solve.
Plan limits
| Plan | Reviews / hour | Max changed files | Profiles | Analysers | Spend cap |
|---|---|---|---|---|---|
| free | 4 | 100 | quiet, balanced | No | $5/month |
| oss | 12 | 400 | quiet, balanced, thorough | Yes | $60/month |
| team | 30 | 400 | quiet, balanced, thorough | Yes | None |
| enterprise | Unmetered | 2,000 | quiet, balanced, thorough | Yes | None |
These values are read from the same entitlement table the enforcement code reads, so this page cannot promise a limit the product does not apply. See pricing for what each plan costs.
Rate limits and degradation
- Over the hourly review limit, a review is queued rather than dropped, and the check run says how long the wait is.
- Sustained overuse on a metered plan degrades to the cheap lane before it degrades to refusal: a smaller model still reviews, with the summary saying so.
- The free and open-source plans carry a monthly inference ceiling. It exists so a public GitHub App cannot be used to farm our compute, and it is stated in the entitlement table above rather than discovered when reviews stop.
GitHub’s own secondary rate limits apply on top. We back off and retry rather than dropping a review; a review delayed by rate limiting still posts, with the delay visible on the check run.
How a skip is reported
- Per-file skips — counted in the summary comment’s walkthrough (“12 files reviewed, 4 skipped: 3 generated, 1 lockfile”).
- Whole-PR skips — a check run with a neutral conclusion naming the reason. No wall of text on a pull request nobody wanted reviewed.
- Budget cuts — a line in the summary with the number suppressed and the severity of the highest one cut.
- Everything at once —
@superdiffs configprints the effective configuration and which layer set each value.
What is not a limit
Some quiet is the product working. A review that posts nothing on a clean pull request has not hit a limit; neither has one that posts two comments on a 600-line refactor because only two things were wrong. The counts we publish in the benchmark methodology include comments-per-pull-request precisely because a tool that says more is not a tool that found more.