Releases
Changelog
Subscribe by RSS. Every entry that changes comment volume carries a Noise note saying which way and by how much.
Reviews now run on Anthropic Claude
Every review now runs on Anthropic Claude: Claude Haiku 5.5 triages the pull request and writes the summary, and Claude Opus 5.5 finds and verifies the findings. OpenAI is no longer used for anything and is sent nothing, so the subprocessor list now names Anthropic as the model provider in use.
- ChangedReviews run on Anthropic Claude: Haiku 5.5 for triage and the summary, Opus 5.5 to find and verify. No review content is sent to OpenAI any more.
- ChangedThe subprocessor list marks Anthropic as the model provider in use and no longer lists OpenAI.
- NoiseFewer comments, more of them right: on our 50-pull-request benchmark replay the Claude lanes posted about 2.2 inline comments per pull request, down from 3.1.
Comment volume
Quieter. On the same 50-pull-request replay and the same judge, inline comments fell from about 3.1 to 2.2 per pull request, precision rose from 37.6% to 47.5%, and recall moved from 37.3% to 36.1%.
Public beta
superdiffs is in public beta. Every new installation starts a 14-day Team trial with no card; after it, the Free plan, or Team at $19 per pull request author per month billed annually ($24 monthly). The first ten teams can join the design partner program: Team free for six months. The site now says plainly what is not built yet, and no longer describes usage-based billing, which never shipped.
- ChangedPublic beta: reviews are open to any GitHub installation, starting with a 14-day Team trial and no card.
- AddedDesign partner program: Team free for six months for the first ten teams, in exchange for real pull request traffic, honest feedback and about half an hour a month.
- ChangedReview comments now say a finding was “checked by a second pass”. They used to call that check independent; the second pass is the same model family re-reading the claim, and the label should not claim more.
- FixedPlan-limit messages on pull requests no longer quote a price per review: billing by usage is not built, and Team is priced per author only.
- ChangedThe pricing, enterprise and trust pages list SSO/SAML, audit log export, self-hosted and air-gapped deployment, EU data residency, zero-retention mode and an uptime SLA as not available yet.
- AddedComparison pages for CodeRabbit, GitHub Copilot code review and Cursor Bugbot, with every fact about them linked to the vendor’s own pages.
Comment volume
Neutral. One label in our comments changed wording; nothing changed how many comments are posted.
Legal documents published
The Terms of Service, Privacy Policy, Data Processing Addendum, Acceptable Use Policy, Refund Policy and the subprocessor list are published as version 1.0. Publishing the list as final meant correcting it first.
- AddedLegal documents published as version 1.0, with the version and date at the top of each.
- FixedThe subprocessor list marks OpenAI as the model provider in use, adds Railway (hosting) and Resend (email), and no longer lists error monitoring as in use while it is switched off.
A 14-day trial, allowances in reviews, and email you can switch off
New installations start a 14-day Team trial. Plan allowances are shown as reviews and a reset date instead of inference dollars, in the dashboard and on the pull request. Customer email goes out from an outbox, with one-click unsubscribe on every message.
- AddedEvery new installation starts a 14-day Team trial; when it ends, the organization moves to the Free plan. Admins are emailed before it ends and when it has.
- ChangedAllowances are described in reviews, not dollars of inference: the dashboard shows reviews left and the reset date, and limit messages on pull requests say the same.
- AddedAt 80% of the monthly allowance the PR summary says so, and organization admins are emailed at 80% and when it runs out. A dashboard switch turns those emails off.
- AddedCustomer email — the welcome at first sign-in, trial and usage notices — is delivered from an outbox, and every message carries a signed one-click unsubscribe link.
- FixedBilling counts only pull request authors with write access, and a new author’s seat is added to the Paddle subscription as it appears.
- ChangedOnly organization admins can change billing and review settings in the dashboard, and the dashboard shows when a canceled subscription ends.
- SecurityThe site sends a Content-Security-Policy in report-only mode.
- AddedError tracking with Sentry is built in. It stays off unless a DSN is configured.
Comment volume
No change to what a review posts. The trial runs Team limits — every depth and the deterministic checks — so a new installation can see more findings than the Free plan would show it.
Reviews explain themselves on the pull request
Every outcome now shows up on the pull request: a status comment edited in place, the reason when a review is skipped or delayed, and a note when one fails. Installing reviews your most recent open pull requests straight away. A point a human already made is not repeated, our threads resolve themselves when the code is fixed, and replies can teach the reviewer.
- AddedOne status comment per pull request, edited in place: skipped with the reason, delayed by a rate limit with the wait, failed or timed out.
- AddedInstalling the app queues reviews of up to three of the most recently updated open pull requests.
- AddedA short welcome note on a repository’s first review, saying that many pull requests get a summary and nothing else.
- AddedThreads we opened are resolved once a push fixes the code under them — and only when the same finding is not raised again on that push.
- Added`remember: …` in a review thread, and `@superdiffs resolve <reason>`, create rules for the repository. Admins edit them under Dashboard → Learnings.
- AddedDeterministic checks for committed secrets, merge conflict markers and leftover debugger statements, on Team, the trial and open source.
- ChangedReviews read the pull request title and description and the repository’s reviewer guidance files (CLAUDE.md, .cursorrules, Copilot instructions), and verification can read and search the checkout.
- FixedThe review no longer repeats a point a human reviewer already made on the pull request.
- FixedThe PR summary is written from the diff, not only the file names, and model and prompt versions no longer appear in customer comments.
- FixedReviews of pull requests touching bracketed route directories such as `app/[id]` no longer fail at checkout.
Comment volume
Lower where people have already commented, and on threads that now close themselves. Higher on plans with the deterministic checks, which can add comments for committed secrets, conflict markers and debugger statements. A new installation gets up to three extra reviews from the install backfill.
The first published numbers, and shorter comments
The seeded-bug eval of the review pipeline is published on the benchmark page: 25 fixtures we seeded ourselves, scored twice, with every miss and false positive listed. Long lists in summary and finding comments now fold away.
- AddedThe benchmark page publishes the first measured run of the pipeline on our own seeded-bug fixtures, with both matcher scores, the misses and the false positives. It is a regression signal, not a real-world catch rate.
- ChangedLong lists in summary and finding comments fold behind a toggle after the first few items.
- Changed`.superdiffs.yml` is accepted alongside `.superdiffs.yaml`, `standard` is accepted as another name for `balanced`, and `review.ignore` takes gitignore-style globs.
- NoiseRecorded late: since August 13 the profiles run one find pass on `quiet` and `balanced` and three on `thorough` (they were one, three and five). On our measurements one pass of a frontier model found more real issues than three passes of a cheaper one.
Comment volume
Neutral for this release: the same findings, in shorter comments.
All nine bot commands are live
The six commands that parsed but answered "not live yet" now do the thing they name. `summary` rewrites the walkthrough without re-running the finder, `resolve` closes our threads and records them user-resolved so they stay closed, `pause`, `resume`, and `ignore` control automatic review per pull request (an `@superdiffs ignore` in the PR description works too), and `config` posts the effective configuration for the repository.
- AddedAll six remaining bot commands are live: `summary`, `resolve`, `pause`, `resume`, `ignore`, `config`.
- Added`@superdiffs ignore` may now be placed in the pull-request description — the one surface an outside author controls, and the one directive it may carry.
- Added`@superdiffs config` posts the resolved configuration with the source of each value, from the last-known-good cache a review records.
- ChangedThe help table no longer marks any command "coming soon"; the live list is now derived from the contract itself.
- FixedThe in-memory store (the keyless dev twin) stopped wiping a repository’s cached configuration on every webhook event, matching Postgres upsert semantics.
Comment volume
Neutral by default. `pause`, `resume`, and `ignore` can only reduce what we post; `summary` rewrites one comment; `resolve` closes threads the user asked to close.
Private alpha: the review pipeline, end to end
First cut of the whole path — diff parsing, triage, multi-pass finding, evidence-grounded verification, fingerprint dedupe, ranking, and a batched GitHub review. Running against our own repositories only. No public sign-up, no published accuracy numbers yet: the benchmark methodology is published, the first run is not.
- AddedReview pipeline: triage → find (1–5 passes by profile) → vote → verify → dedupe → rank, with every stage independently replayable from a stored trace.
- AddedEvidence grounding: a finding whose quoted snippet does not appear in the diff or the file it claims to be in is dropped rather than posted with a caveat.
- AddedFingerprint matching (strong: content-addressed; weak: rule + path + line) so a resolved comment is not posted again on the next push.
- Added`.superdiffs.yaml` configuration: org → repo → directory layering, with file-count and comment caps where the lowest layer wins.
- AddedBot commands (`@superdiffs review`, `full-review`, `summary`, `resolve`, `pause`, `resume`, `ignore`, `config`, `help`), authorized by the actor’s repository permission rather than by the text of the comment.
- SecuritySandboxed execution with a default-deny egress policy. The only credential that crosses into the sandbox is a one-hour, single-repository, contents-read token that cannot post.
- SecurityTwo-way sanitization: invisible-character stripping and structural delimiting on everything a model reads; image, raw-HTML, and foreign-link stripping on everything we post.
- NoiseDefault profile is `balanced`: 3 find passes, minor-and-above posted, 20 comments maximum per review, nits collapsed into one section.
Comment volume
Baseline release — there is no previous version to compare against. Comment volume from this version is the baseline every later entry is measured against.
Versions are semantic with one addition: a change to default comment volume is treated as breaking, because it is the change most likely to surprise a team that had the tool tuned the way they wanted it.