Benchmarks
Two measurements, because they answer two questions.
A single percentage that blurs them is worth nothing to anyone deciding whether to buy. One is the number you experience; the other is the number that says which kinds of puzzle the model is good at.
- On real captchas — the client driving a real browser against the vendors' own public demo pages, end to end, until the vendor accepts.
- On static images — the model alone, answering a held-out set of real captured puzzles, with no browser involved.
The browser number is the more honest one: a correct answer still has to be clicked, in the right place, before the widget's own timeout, past whatever the vendor thinks of the mouse that moved.
Every figure here is a count over a dated run against a named model. Nothing is extrapolated and nothing is rounded up. Each table covers the vendors listed in it, and its totals are computed over the rows shown.
On real captchas
Driven in a browser, until the vendor accepted.
Each row is one puzzle type driven on the vendor's own public demo page through the hosted API, with every attempt scored. Counts, not percentages: at these sample sizes a percentage implies a precision the count does not have, and the count is the thing you can actually reason about.
A row is the whole widget, not one puzzle. Vendors ask again — reCAPTCHA keeps going until it is satisfied, and a later round is often a different shape from the one that opened. A row is named for the puzzle the vendor opened with, and the time covers every round after it. That is both the honest reading and the stronger claim.
The median is challenge-visible to verified: the span from the puzzle appearing to the vendor accepting. Page load, the checkbox and the widget's own boot are the site's latency, not ours, and are excluded.
Measured between 2026-11-12 and 2026-11-15 against CaptchaKraken Abyss, the model the hosted API serves by default, driven by CaptchaKraken dev build on Camoufox 156.0.1. Every clip on the home page comes out of these same runs; a type needs 8 in 10 to get one, and the rows marked otherwise have none. FunCaptcha is shown as work in progress, and Other's moving board is shown at 6/8, under the bar, because it is the only moving board. Other is a vendor we solve and do not name.
| Puzzle | Vendor | Solved | Median | Recorded |
|---|---|---|---|---|
| Drag and arrange | ||||
| Shuffled tiles Dingxiang · 2026-11-15 | Dingxiang | 6/6 | 5.9s | 2026-11-15 |
| Gobang GeeTest · 2026-11-15 | GeeTest | 6/6 | 6.3s | 2026-11-15 |
| Icon crush GeeTest · 2026-11-15 | GeeTest | 6/6 | 6.1s | 2026-11-15 |
| Drag the missing piece into the pattern Other · 2026-11-12 | Other | 7/7 | 16.4s | 2026-11-12 |
| Find and click | ||||
| Largest area Dingxiang · 2026-11-15 | Dingxiang | 6/6 | 5.1s | 2026-11-15 |
| Odd one out Dingxiang · 2026-11-15 | Dingxiang | 6/6 | 4.1s | 2026-11-15 |
| Font Dingxiang · 2026-11-15 | Dingxiang | 6/6 | 4.7s | 2026-11-15 |
| Icon click no clipDingxiang · 2026-11-15 | Dingxiang | 6/6 | 7.6s | 2026-11-15 |
| 3D shapes Dingxiang · 2026-11-15 | Dingxiang | 6/6 | 3.7s | 2026-11-15 |
| Text click no clipDingxiang · 2026-11-15 | Dingxiang | 6/6 | 6.7s | 2026-11-15 |
| Find it in a scattered scene Other · 2026-11-15 | Other | 37/37 | 7.3s/board | 2026-11-15 |
| Icons in order NetEase Yidun · 2026-11-15 | NetEase Yidun | 6/6 | 7.8s | 2026-11-15 |
| Characters in order NetEase Yidun · 2026-11-15 | NetEase Yidun | 6/6 | 7.7s | 2026-11-15 |
| Ordered icon click no clipGeeTest · 2026-11-15 | GeeTest | 6/7 | 7.6s | 2026-11-15 |
| Cycling line art under 8/10GeeTest · 2026-11-15 | GeeTest | 18/43 | 22.5s | 2026-11-15 |
| Moving boards | ||||
| Animated: the island the animal never visited Other · 2026-11-15 | Other | 6/8 | 10.7s/board | 2026-11-15 |
| Image grids | ||||
| 3×3 photo grid no clipGeeTest · 2026-11-15 | GeeTest | 6/6 | 6.9s | 2026-11-15 |
| Image grid by description Other · 2026-11-15 | Other | 2/2 | 7.4s/board | 2026-11-15 |
| 3×3 dynamic reCAPTCHA · 2026-11-15 | reCAPTCHA | 4/4 | 18.7s | 2026-11-15 |
| 4×4 tile grid reCAPTCHA · 2026-11-15 | reCAPTCHA | 14/14 | 6.6s | 2026-11-15 |
| 3×3 tile grid reCAPTCHA · 2026-11-15 | reCAPTCHA | 20/22 | 6.5s | 2026-11-15 |
| Sliders | ||||
| Slide jigsaw on rendered art GeeTest · 2026-11-15 | GeeTest | 6/6 | 7.7s | 2026-11-15 |
| Slide jigsaw Tencent · 2026-11-15 | Tencent | 6/6 | 6.2s | 2026-11-15 |
| Tilted piece, two notches Dingxiang · 2026-11-15 | Dingxiang | 6/7 | 12.5s | 2026-11-15 |
| Slide jigsaw no clipNetEase Yidun · 2026-11-15 | NetEase Yidun | 6/7 | 6.0s | 2026-11-15 |
| Slide jigsaw (v3) under 8/10GeeTest v3 · 2026-11-15 | GeeTest v3 | 6/8 | 9.4s | 2026-11-15 |
| Puzzle games | ||||
| Claw machine work in progressFunCaptcha · 2026-11-12 | FunCaptcha | 6/13 | 9.5s | 2026-11-12 |
Tried in the same run and not in this table: too few attempts to score a row, and only reCAPTCHA's fading grid passed at all. Tencent text 0/2 and icon click 0/2; YiDun picture click 0/3 and icon click 0/3; Dingxiang word order 0/3, rotate 0/2 and restore 0/2; reCAPTCHA fading 3x3 1/3.
Tried since, and not solved: FunCaptcha's orbit and dice games 0/16 each (2026-11-12, beside the claw machine above). On 2026-11-13, through the hosted API: MTCaptcha 0/16, BotDetect 0/16 and Lemin 0/16, and Yandex 0/1 (its demo page showed no puzzle in the other 15 attempts).
On static images
The model alone, one screenshot at a time.
The model answering 901 real captured puzzles it has never trained on, with no browser involved. Scored the way a widget scores: exact set match, pass or fail per puzzle, with the vendor's own leeway. No partial credit — a partially-correct grid answer is a rejected captcha.
Every real capture is held out; nothing hand-labelled is trained on. These measure skill rather than memorisation.
The n column is how many held-out captures of that puzzle we hold. Where it is small, read the rate loosely — a row over 11 captures moves 9 points on one puzzle, which is why the denominator is printed beside every rate rather than left out to tidy the table.
Measured 2026-09-06 against CaptchaKraken v1.2 Twilight, a v1 model the v2 lineup has since replaced: read it as where the previous generation stood, not as the model serving today. These are percentages rather than counts because the denominator is in the row beside them; the table above has no such column, which is why it prints counts.
| Vendor | Puzzle | n | Solved |
|---|---|---|---|
| BotDetect | Distorted text | 28 | 96% |
| GeeTest | Canvas puzzle | 74 | 99% |
| GeeTest | Gobang | 26 | 96% |
| GeeTest | Slide jigsaw | 23 | 91% |
| GeeTest | Ordered icon click | 25 | 88% |
| GeeTest | Icon crush | 24 | 83% |
| GeeTest | Nine-tile icon grid | 12 | 83% |
| GeeTest | Slide to fit (v3) | 11 | 73% |
| Lemin | Cropped image | 11 | 100% |
| MTCaptcha | Distorted text | 29 | 97% |
| Prosopo | 3x3 photo grid | 11 | 73% |
| Tencent | Slide jigsaw | 12 | 92% |
| Yandex | Distorted text | 23 | 65% |
| YiDun | Slide jigsaw | 12 | 100% |
| YiDun | Ordered icon click | 12 | 92% |
| YiDun | Picture click | 14 | 79% |
| reCAPTCHA | 3x3 tile grid | 327 | 71% |
| reCAPTCHA | 4x4 tile grid | 227 | 44% |
Weighted by how many captures of each puzzle we hold, across 18 puzzle types — the same arithmetic as scoring all 901 captures in one pile. A flat average of the rows would let a type we hold 11 captures of push the headline as hard as reCAPTCHA's 327.
The gap
Why the two tables disagree.
A puzzle can score well on static images and badly in a browser, and the reasons are worth naming because a self-hoster will hit them too.
An animated puzzle has to be recorded before it can be read
A still screenshot of a cycling board is a picture of one frame, and the answer may belong to a frame that has already gone. Those rows carry the longest medians in the table above.
A drag has to land
Two boxes correct out of two is a solve; one out of two is a rejection. The static score gives partial geometry credit that the widget never gives.
The vendor gets a vote
A correct answer clicked by a mouse the vendor dislikes is still a failed captcha. Nothing about the model's answer changes that.
Grids are sent with the cell numbers drawn on, and this is not cosmetic: on raw un-numbered screenshots the same model scores 0% on 4x4, because it has to invent a numbering convention for sixteen cells of one continuous photograph. If you are building your own client, draw the overlay — the performance guide has the details.
Reproducing it
The browser figures ship as an example.
It drives the vendors' own demo pages through the hosted API and prints two clocks per attempt: the solve span, which is what the medians above measure, and the total, which adds page load and the demo page's own reveal click.
# in a clone of the open-source client
cd js && npm run demoThe static-image figures are measured against a held-out corpus of real captures that is not distributed. The method is stated above in full, so its shape is reproducible against your own captures.
The same two tables are in the client repository as docs/benchmarks.md. The docs have the request shape, the prompts and the rate card.