Spend transparency
Every API call is logged with its prompt tokens, completion tokens, reasoning tokens (where the provider reports them), latency, and authoritative cost from OpenRouter’s /generation endpoint. No black boxes — every row below is a real billed call.
Total spent
$202.88
Runs completed
6,839
Tokens in
8,904,406
Tokens out
7,145,659
Reasoning tok
3,807,834
By model
Click any column header to sortRun counts are uneven by design. Frontier models (the cutting-edge release from each lab) are sampled at N=5 per cell across 14 instruments × 2 framings (max 140 runs). Historical models from earlier product generations are sampled at N=3 per cell (max 84 runs) for the cross-version drift analysis. Lower counts than the max reflect occasional content-filter rejections or empty responses.
| Lab | Model | Cohort | Runs | Tokens in | Tokens out | Cost▼ |
|---|---|---|---|---|---|---|
| OpenAI | GPT-5.5 Pro | Historical | 160 | 149,010 | 256,376 | $50.62 |
| OpenAI | OpenAI o1 | Historical | 160 | 149,010 | 623,900 | $39.46 |
| OpenAI | GPT-5.6 Sol Pro | Historical | 159 | 1,080,356 | 288,255 | $11.79 |
| Anthropic | Claude Fable 5 | Historical | 160 | 219,100 | 133,315 | $8.86 |
| Anthropic | Claude Opus 4 | Historical | 160 | 164,270 | 68,000 | $7.56 |
| Anthropic | Claude Opus 4.1 | Historical | 160 | 164,270 | 68,000 | $7.56 |
| Anthropic | Claude Fable 5.1 | Historical | 160 | 219,420 | 81,386 | $6.26 |
| Anthropic | Claude Opus 4.8 Fast | Historical | 160 | 219,100 | 79,280 | $6.16 |
| OpenAI | GPT-5.6 Terra Pro | Historical | 160 | 1,057,101 | 267,816 | $5.06 |
| Gemini 2.5 Pro | Frontier | 160 | 158,720 | 450,130 | $4.69 | |
| Anthropic | Claude Opus 5 | Historical | 160 | 219,100 | 110,785 | $3.87 |
| OpenAI | GPT-5.5 | Frontier | 160 | 149,010 | 105,233 | $3.79 |
| OpenAI | GPT-5 | Historical | 147 | 123,173 | 351,361 | $3.65 |
| OpenAI | GPT-4 Turbo | Historical | 160 | 150,490 | 67,698 | $3.54 |
| xAI | Grok 4.6 | Historical | 160 | 183,660 | 480,834 | $3.13 |
| Anthropic | Claude Opus 4.8 | Frontier | 160 | 219,100 | 79,280 | $3.08 |
| Anthropic | Claude Opus 4.7 | Historical | 160 | 219,900 | 77,619 | $3.04 |
| Gemini 3.1 Pro Preview | Historical | 159 | 156,129 | 215,886 | $2.90 | |
| OpenAI | GPT-5.6 Sol | Historical | 160 | 149,010 | 82,774 | $2.60 |
| Anthropic | Claude Opus 4.5 | Historical | 160 | 164,270 | 68,000 | $2.52 |
| Anthropic | Claude Opus 4.6 | Historical | 160 | 164,430 | 67,400 | $2.51 |
| OpenAI | OpenAI o3 | Historical | 160 | 149,010 | 226,878 | $2.07 |
| OpenAI | GPT-5.2 | Historical | 160 | 149,010 | 100,062 | $1.62 |
| xAI | Grok 4.5 | Historical | 160 | 183,660 | 227,350 | $1.56 |
| Anthropic | Claude Sonnet 4.5 | Historical | 160 | 164,270 | 69,308 | $1.53 |
| Anthropic | Claude Sonnet 4.6 | Historical | 160 | 164,430 | 68,000 | $1.51 |
| Anthropic | Claude Sonnet 4 | Historical | 160 | 164,270 | 68,000 | $1.51 |
| DeepSeek | DeepSeek V4 Pro 0813 | Historical | 160 | 161,517 | 299,610 | $1.16 |
| DeepSeek | DeepSeek R1 | Historical | 160 | 151,806 | 416,509 | $1.15 |
| OpenAI | GPT-5.6 Terra | Historical | 160 | 149,010 | 72,869 | $1.08 |
| OpenAI | GPT-5.4 | Historical | 160 | 149,010 | 46,400 | $1.02 |
| OpenAI | GPT-4o | Historical | 160 | 149,170 | 67,372 | $1.02 |
| DeepSeek | DeepSeek R1 (0528) | Frontier | 160 | 149,850 | 388,373 | $0.94 |
| DeepSeek | DeepSeek V4 Pro | Historical | 156 | 145,798 | 329,879 | $0.89 |
| OpenAI | GPT-5.1 | Historical | 160 | 149,010 | 63,298 | $0.78 |
| xAI | Grok 4.3 | Historical | 160 | 171,140 | 210,739 | $0.61 |
| Mistral | Mistral Large 2411 | Historical | 140 | 137,270 | 51,286 | $0.55 |
| Anthropic | Claude Haiku 4.5 | Historical | 160 | 164,270 | 68,800 | $0.51 |
| xAI | Grok 4.20 | Frontier | 160 | 169,860 | 68,069 | $0.31 |
| Mistral | Mistral Large (2512) | Frontier | 160 | 155,530 | 78,000 | $0.17 |
| DeepSeek | DeepSeek Chat V3 | Historical | 160 | 148,297 | 67,913 | $0.11 |
| Meta | Llama 4 Maverick | Frontier | 160 | 150,010 | 67,512 | $0.08 |
| Meta | Llama 3.3 70B | Historical | 158 | 149,579 | 66,104 | $0.05 |
Recent runs
| Model | Instrument | Framing | In/Out | Latency | Cost | Status |
|---|---|---|---|---|---|---|
| grok-4.6 | sd3 | human#5 | 838/2,080+1,753r | 0.2s | $0.01 | completed |
| grok-4.6 | sd3 | human#4 | 838/2,249+1,922r | 0.2s | $0.01 | completed |
| grok-4.6 | sd3 | human#1 | 838/4,282+3,955r | 0.3s | $0.03 | completed |
| grok-4.6 | sd3 | human#3 | 838/2,430+2,103r | 0.2s | $0.02 | completed |
| grok-4.6 | sd3 | human#2 | 838/2,270+1,943r | 0.2s | $0.01 | completed |
| grok-4.6 | sd3 | self#5 | 861/3,816+3,489r | 0.2s | $0.02 | completed |
| grok-4.6 | sd3 | self#4 | 861/3,072+2,745r | 0.2s | $0.02 | completed |
| grok-4.6 | sd3 | self#2 | 861/3,671+3,344r | 0.4s | $0.02 | completed |
| grok-4.6 | sd3 | self#3 | 861/3,455+3,128r | 0.2s | $0.02 | completed |
| grok-4.6 | sd3 | self#1 | 861/2,904+2,577r | 0.2s | $0.02 | completed |
| grok-4.6 | pvq21 | human#5 | 1,053/2,121+1,866r | 0.2s | $0.01 | completed |
| grok-4.6 | pvq21 | human#4 | 1,053/2,550+2,295r | 0.2s | $0.02 | completed |
| grok-4.6 | pvq21 | human#3 | 1,053/2,424+2,169r | 0.2s | $0.02 | completed |
| grok-4.6 | pvq21 | self#4 | 1,076/3,520+3,265r | 0.1s | $0.02 | completed |
| grok-4.6 | pvq21 | human#2 | 1,053/2,070+1,815r | 0.2s | $0.01 | completed |
| grok-4.6 | pvq21 | self#5 | 1,076/2,335+2,080r | 0.3s | $0.01 | completed |
| grok-4.6 | pvq21 | human#1 | 1,053/1,908+1,653r | 0.2s | $0.01 | completed |
| grok-4.6 | pvq21 | self#3 | 1,076/3,389+3,134r | 0.2s | $0.02 | completed |
| grok-4.6 | pvq21 | self#1 | 1,076/3,013+2,758r | 0.2s | $0.02 | completed |
| grok-4.6 | pvq21 | self#2 | 1,076/2,364+2,109r | 0.2s | $0.01 | completed |
| grok-4.6 | otti102 | human#5 | 2,131/5,597+4,370r | 0.2s | $0.03 | completed |
| grok-4.6 | otti102 | human#3 | 2,131/5,744+4,517r | 0.2s | $0.04 | completed |
| grok-4.6 | otti102 | human#2 | 2,131/6,375+5,148r | 0.2s | $0.04 | completed |
| grok-4.6 | otti102 | human#4 | 2,131/5,047+3,820r | 0.2s | $0.03 | completed |
| grok-4.6 | otti102 | human#1 | 2,131/5,591+4,364r | 0.2s | $0.04 | completed |
| grok-4.6 | otti102 | self#5 | 2,154/6,010+4,783r | 0.2s | $0.04 | completed |
| grok-4.6 | otti102 | self#3 | 2,154/6,263+5,036r | 0.2s | $0.04 | completed |
| grok-4.6 | otti102 | self#1 | 2,154/5,256+4,029r | 0.3s | $0.04 | completed |
| grok-4.6 | obsi32 | human#5 | 1,086/2,919+2,532r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | human#4 | 1,086/2,865+2,478r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | human#2 | 1,086/2,328+1,941r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | self#4 | 1,109/3,966+3,579r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | human#1 | 1,086/2,597+2,210r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | self#2 | 1,109/3,090+2,703r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | self#3 | 1,109/2,944+2,557r | 0.2s | $0.02 | completed |
| grok-4.6 | obsi32 | self#1 | 1,109/2,797+2,410r | 0.2s | $0.02 | completed |
| grok-4.6 | ncs18 | human#5 | 818/2,425+2,206r | 0.2s | $0.02 | completed |
| grok-4.6 | ncs18 | human#4 | 818/2,119+1,900r | 0.2s | $0.01 | completed |
| grok-4.6 | ncs18 | human#1 | 818/1,925+1,706r | 0.2s | $0.01 | completed |
| grok-4.6 | ncs18 | human#3 | 818/1,612+1,393r | 0.2s | $0.01 | completed |
| grok-4.6 | ncs18 | self#5 | 841/1,314+1,095r | 0.3s | $0.00841 | completed |
| grok-4.6 | ncs18 | self#4 | 841/1,719+1,500r | 0.2s | $0.01 | completed |
| grok-4.6 | ncs18 | self#2 | 841/2,097+1,878r | 0.1s | $0.01 | completed |
| grok-4.6 | ncs18 | self#3 | 841/1,856+1,637r | 0.2s | $0.01 | completed |
| grok-4.6 | mfq30 | human#5 | 1,289/3,745+3,382r | 0.2s | $0.02 | completed |
| grok-4.6 | mfq30 | human#4 | 1,289/2,735+2,372r | 0.2s | $0.02 | completed |
| grok-4.6 | mfq30 | human#3 | 1,289/2,699+2,336r | 0.2s | $0.02 | completed |
| grok-4.6 | ncs18 | self#1 | 841/1,474+1,255r | 0.2s | $0.01 | completed |
| grok-4.6 | mfq30 | self#3 | 1,312/3,758+3,395r | 0.2s | $0.02 | completed |
| grok-4.6 | mfq30 | human#2 | 1,289/2,503+2,140r | 0.3s | $0.02 | completed |
| grok-4.6 | mfq30 | self#1 | 1,312/4,074+3,711r | 0.2s | $0.03 | completed |
| grok-4.6 | mfq30 | human#1 | 1,289/2,653+2,290r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | human#5 | 1,165/2,725+2,242r | 0.3s | $0.02 | completed |
| grok-4.6 | lsq40 | human#3 | 1,165/2,705+2,222r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | human#4 | 1,165/2,149+1,666r | 0.3s | $0.01 | completed |
| grok-4.6 | lsq40 | human#2 | 1,165/2,919+2,436r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | self#5 | 1,188/2,950+2,467r | 0.3s | $0.02 | completed |
| grok-4.6 | lsq40 | human#1 | 1,165/2,550+2,067r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | self#4 | 1,188/3,167+2,684r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | self#2 | 1,188/3,155+2,672r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | self#1 | 1,188/3,188+2,705r | 0.2s | $0.02 | completed |
| grok-4.6 | lsq40 | self#3 | 1,188/2,111+1,628r | 0.2s | $0.01 | completed |
| grok-4.6 | locus_levenson | human#5 | 990/3,004+2,713r | 0.2s | $0.02 | completed |
| grok-4.6 | locus_levenson | human#2 | 990/2,579+2,288r | 0.2s | $0.02 | completed |
| grok-4.6 | locus_levenson | human#3 | 990/1,863+1,572r | 0.3s | $0.01 | completed |
| grok-4.6 | kolb12 | human#4 | 764/1,519+1,372r | 0.6s | $0.00968 | completed |
| grok-4.6 | kolb12 | human#2 | 764/1,460+1,313r | 0.2s | $0.01 | completed |
| grok-4.6 | kolb12 | human#1 | 764/1,015+868r | 0.2s | $0.00743 | completed |
| grok-4.6 | ipip50 | self#5 | 992/3,287+2,684r | 0.3s | $0.02 | completed |
| grok-4.6 | ipip50 | human#2 | 969/2,745+2,142r | 0.2s | $0.02 | completed |
| grok-4.6 | kolb12 | self#5 | 787/1,333+1,186r | 0.2s | $0.00842 | completed |
| grok-4.6 | kolb12 | self#4 | 787/1,669+1,522r | 0.2s | $0.01 | completed |
| grok-4.6 | ipip50 | self#3 | 992/3,108+2,505r | 0.3s | $0.02 | completed |
| grok-4.6 | hexaco24 | human#5 | 897/3,669+3,378r | 0.2s | $0.02 | completed |
| grok-4.6 | ipip50 | self#1 | 992/2,926+2,323r | 0.2s | $0.02 | completed |
| grok-4.6 | hexaco24 | human#4 | 897/3,391+3,100r | 0.4s | $0.02 | completed |
| grok-4.6 | hexaco24 | human#3 | 897/3,052+2,761r | 0.2s | $0.02 | completed |
| grok-4.6 | hexaco24 | human#1 | 897/3,081+2,790r | 0.2s | $0.02 | completed |
| grok-4.6 | hexaco24 | human#2 | 897/2,146+1,855r | 0.2s | $0.01 | completed |
| grok-4.6 | hexaco24 | self#5 | 920/2,612+2,321r | 0.2s | $0.02 | completed |
| grok-4.6 | hexaco24 | self#3 | 920/2,379+2,088r | 0.2s | $0.01 | completed |
| grok-4.6 | hexaco24 | self#4 | 920/2,128+1,837r | 0.2s | $0.01 | completed |
| grok-4.6 | hexaco24 | self#2 | 920/2,097+1,806r | 0.2s | $0.01 | completed |
| grok-4.6 | eq_short | human#5 | 890/2,129+1,862r | 0.2s | $0.01 | completed |
| grok-4.6 | eq_short | human#3 | 890/1,941+1,674r | 0.3s | $0.01 | completed |
| grok-4.6 | eq_short | human#4 | 890/1,629+1,362r | 0.2s | $0.01 | completed |
| grok-4.6 | eq_short | self#4 | 913/3,414+3,147r | 0.2s | $0.02 | completed |
| grok-4.6 | eq_short | human#2 | 890/2,521+2,254r | 0.3s | $0.02 | completed |
| grok-4.6 | enneagram90 | human#5 | 2,587/6,325+5,242r | 0.2s | $0.04 | completed |
| grok-4.6 | eq_short | self#2 | 913/3,180+2,913r | 0.2s | $0.02 | completed |
| grok-4.6 | enneagram90 | human#4 | 2,587/5,938+4,855r | 0.3s | $0.04 | completed |
| grok-4.6 | eq_short | self#1 | 913/2,398+2,131r | 0.2s | $0.02 | completed |
| grok-4.6 | enneagram90 | human#3 | 2,587/5,600+4,517r | 0.2s | $0.03 | completed |
| grok-4.6 | enneagram90 | human#1 | 2,587/5,148+4,065r | 0.3s | $0.04 | completed |
| grok-4.6 | enneagram90 | self#5 | 2,610/7,118+6,035r | 0.2s | $0.04 | completed |
| grok-4.6 | enneagram90 | self#4 | 2,610/5,027+3,944r | 0.3s | $0.04 | completed |
| grok-4.6 | enneagram90 | self#3 | 2,610/6,199+5,116r | 0.2s | $0.04 | completed |
| grok-4.6 | enneagram90 | self#2 | 2,610/5,065+3,982r | 0.2s | $0.04 | completed |
| grok-4.6 | enneagram36 | human#4 | 1,164/3,276+2,841r | 0.2s | $0.02 | completed |
| grok-4.6 | enneagram36 | human#1 | 1,164/4,109+3,674r | 0.3s | $0.03 | completed |
| grok-4.6 | enneagram36 | human#2 | 1,164/3,127+2,692r | 0.2s | $0.02 | completed |
| grok-4.6 | enneagram36 | self#2 | 1,187/3,679+3,244r | 0.2s | $0.02 | completed |
| grok-4.6 | enneagram36 | self#1 | 1,187/3,120+2,685r | 0.2s | $0.02 | completed |
| grok-4.6 | ecr12 | human#2 | 723/2,105+1,958r | 0.2s | $0.01 | completed |
| grok-4.6 | ecr12 | human#3 | 723/1,801+1,654r | 0.2s | $0.01 | completed |
| grok-4.6 | ecr12 | self#2 | 746/3,418+3,271r | 0.4s | $0.02 | completed |
| grok-4.6 | ecr12 | self#4 | 746/3,216+3,069r | 0.3s | $0.02 | completed |
| grok-4.6 | ecr12 | self#1 | 746/3,154+3,007r | 0.6s | $0.02 | completed |
| grok-4.6 | ecr12 | self#3 | 746/2,988+2,841r | 0.4s | $0.02 | completed |
| grok-4.5 | ipip50 | human#5 | 969/1,377+774r | 0.2s | $0.00868 | completed |
| grok-4.5 | ipip50 | human#4 | 969/1,534+931r | 0.2s | $0.00962 | completed |
| grok-4.5 | ipip50 | human#3 | 969/1,389+786r | 0.2s | $0.00875 | completed |
| grok-4.5 | ipip50 | human#1 | 969/1,730+1,127r | 0.3s | $0.01 | completed |
| grok-4.5 | ipip50 | self#5 | 992/1,597+994r | 0.2s | $0.01 | completed |
| grok-4.5 | ipip50 | self#4 | 992/1,880+1,277r | 0.2s | $0.01 | completed |
| grok-4.5 | ipip50 | self#3 | 992/1,671+1,068r | 0.2s | $0.01 | completed |
| grok-4.5 | ipip50 | self#2 | 992/1,579+976r | 0.2s | $0.01 | completed |
| grok-4.5 | ipip50 | self#1 | 992/1,411+808r | 0.4s | $0.01 | completed |
| grok-4.5 | hexaco24 | human#5 | 897/873+582r | 0.2s | $0.00681 | completed |
| grok-4.5 | hexaco24 | human#4 | 897/1,145+854r | 0.2s | $0.00866 | completed |
| grok-4.5 | hexaco24 | human#3 | 897/1,269+978r | 0.2s | $0.00788 | completed |
| grok-4.5 | hexaco24 | human#2 | 897/950+659r | 0.2s | $0.00728 | completed |
| grok-4.5 | hexaco24 | self#5 | 920/1,944+1,653r | 0.2s | $0.01 | completed |
| grok-4.5 | enneagram36 | human#5 | 1,164/1,134+699r | 0.1s | $0.00717 | completed |
| grok-4.5 | enneagram36 | human#4 | 1,164/1,251+816r | 0.2s | $0.00962 | completed |
| grok-4.5 | enneagram36 | human#3 | 1,164/1,420+985r | 0.2s | $0.00889 | completed |
| grok-4.5 | enneagram36 | human#2 | 1,164/1,023+588r | 0.2s | $0.00651 | completed |
| grok-4.5 | enneagram36 | human#1 | 1,164/1,087+652r | 0.2s | $0.00863 | completed |
| grok-4.5 | enneagram36 | self#5 | 1,187/1,333+898r | 0.3s | $0.01 | completed |
| grok-4.5 | enneagram36 | self#1 | 1,187/1,783+1,348r | 0.3s | $0.01 | completed |
| grok-4.5 | vark16 | human#2 | 818/476+281r | 0.2s | $0.00427 | completed |
| grok-4.5 | vark16 | human#5 | 818/592+397r | 0.2s | $0.00388 | completed |
| grok-4.5 | vark16 | human#4 | 818/621+426r | 0.1s | $0.00514 | completed |
| grok-4.5 | vark16 | human#1 | 818/578+383r | 0.1s | $0.00380 | completed |
| grok-4.5 | vark16 | human#3 | 818/631+436r | 0.2s | $0.00412 | completed |
| grok-4.5 | vark16 | self#5 | 841/771+576r | 0.2s | $0.00500 | completed |
| grok-4.5 | vark16 | self#4 | 841/957+762r | 0.2s | $0.00721 | completed |
| grok-4.5 | vark16 | self#3 | 841/1,236+1,041r | 0.2s | $0.00779 | completed |
| grok-4.5 | vark16 | self#2 | 841/1,231+1,036r | 0.1s | $0.00885 | completed |
| grok-4.5 | sd3 | human#4 | 838/2,457+2,130r | 0.2s | $0.02 | completed |
| grok-4.5 | sd3 | human#5 | 838/1,177+850r | 0.3s | $0.00743 | completed |
| grok-4.5 | sd3 | human#3 | 838/1,204+877r | 0.2s | $0.00868 | completed |
| grok-4.5 | vark16 | self#1 | 841/963+768r | 0.2s | $0.00724 | completed |
| grok-4.5 | sd3 | human#2 | 838/1,300+973r | 0.2s | $0.00817 | completed |
| grok-4.5 | sd3 | self#5 | 861/1,780+1,453r | 0.3s | $0.01 | completed |
| grok-4.5 | sd3 | human#1 | 838/1,028+701r | 0.2s | $0.00763 | completed |
| grok-4.5 | sd3 | self#4 | 861/1,790+1,463r | 0.2s | $0.01 | completed |
| grok-4.5 | sd3 | self#3 | 861/1,929+1,602r | 0.1s | $0.01 | completed |
| grok-4.5 | sd3 | self#1 | 861/1,833+1,506r | 0.2s | $0.01 | completed |
| grok-4.5 | sd3 | self#2 | 861/1,639+1,312r | 0.2s | $0.01 | completed |