Fastest - Higher is better
Median tokens per second while the model is generating. Higher is better.
309.1
204.4
202.8
106.4
80.0
79.5
70.8
56.4
Lambda 2xB200DedicatedNebiusCrusoeDeepSeekNovitaDeepInfraGMIDigitalOcean
STAC benchmark
DeepSeek V4 Pro - providers
Recorded workload
Every run on this leaderboard uses the workload from STAC's first agentic AI benchmark: a real quant research session supplied by a large US market maker. SwarmSim replays it faithfully across different levels of concurrency on every provider.
The recorded workload* is a long-context, long-trajectory agentic system in which a DeepSeek LLM iteratively develops, debugs, and evaluates trading models.
- Median input tokens
- 172K
- Peak input tokens
- 334K
- Turns
- 300
Lowest latency - Lower is better
Median time until the first token. Lower is better.
906
1.10
1.49
2.04
2.12
2.25
4.25
5.22
CrusoeLambda 2xB200DedicatedDigitalOceanDeepSeekDeepInfraNebiusNovitaGMI
Highest throughput - Higher is better
Aggregate output tokens per second for the whole run. Higher is better.
231.5
61.2
45.3
33.5
31.7
20.6
18.6
17.3
Lambda 2xB200DedicatedNebiusDeepSeekNovitaGMICrusoeDeepInfraDigitalOcean
Request latency - Lower is better
Median time to finish a request. Lower is better.
2.06
2.09
4.17
4.87
5.65
6.33
9.40
10.8
CrusoeLambda 2xB200DedicatedDeepInfraDigitalOceanNebiusDeepSeekNovitaGMI
Prefill speed - Higher is better
Median uncached prompt tokens per second until the first token. A run that omits cache misses counts tokens added since the previous turn. Higher is better.
3.1k
1.2k
866.9
857.8
595.5
525.8
500.2
354.5
NebiusDigitalOceanCrusoeDeepSeekLambda 2xB200DedicatedNovitaGMIDeepInfra
Inter-token latency - Lower is better
Median gap between output tokens. Lower is better.
0.06
0.07
0.42
18.5
18.6
22.7
28.6
38.4
DeepSeekGMINovitaNebiusDigitalOceanCrusoeLambda 2xB200DedicatedDeepInfra
Comparison
Click a column title to sort. Click a row to expand p90/p99 spreads. Missing values stay at the bottom.
| STAC - Lambda 2xB200 (Dedicated) | 309.1 | 1.10 | 2.09 | 231.5 | n/a |
|---|---|---|---|---|---|
| Nebius | 204.4 | 2.25 | 5.65 | 61.2 | 92.4 |
| Crusoe | 202.8 | 0.906 | 2.06 | 20.6 | 98.2 |
| DeepSeek | 106.4 | 2.04 | 6.33 | 45.3 | 97.1 |
| Novita | 80.0 | 4.25 | 9.40 | 33.5 | 97.3 |
| DeepInfra | 79.5 | 2.12 | 4.17 | 18.6 | 96.1 |
| GMI | 70.8 | 5.22 | 10.8 | 31.7 | 96.5 |
| DigitalOcean | 56.4 | 1.49 | 4.87 | 17.3 | 96.2 |
Compare two providers
Click two names. Click again to drop one. A third click replaces the older pick.
Select a second provider to see the spread.
*The recording is contributed by a large US market maker. An agentic coding system (Codex with DeepSeek V4 Pro) iteratively developed, debugged, and evaluated trading models for Bitcoin/USDT market data, producing 300 sequential requests. Input token count is not strictly monotonic across the conversation: it comprises three contiguous segments separated by context-checkpoint compactions, after which accumulated input resets.