---
title: "Report Per-Session and Aggregate Token Throughput"
date: 2026-07-21
canonical: https://solmaz.io/x/2079592187587891349/
x_url: https://x.com/onusoz/status/2079592187587891349
license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
---

My recent formulation is handy when it comes to claims like this, e.g. 50 tok/s on DGX spark with DS4 flash 🤨

Because if the formulation is correct, laws of physics do not permit > 28 tok/s in a single session, for weights comparable to antirez/deepseek-v4-gguf. Ignoring speculative decoding (rho = 1)

Then I realized that the reported number is an aggregate of 4 sessions, so per single session it is like 12~14 tok/s (great result if it holds btw!)

IMO people subjectively always assume single session speeds, so when a tok/s is claimed, we should all converge on a clear notation

For example, 4x12.3 tok/s makes it very clear:
4 sessions, 12.3 tok/s each

And when we want to emphasize aggregate, let's show them always together "4x12.3 tok/s = 50 tok/s aggregate"

Another interesting result of the formulation: If you are willing to suffer 12 tok/s, then you can theoretically have up to 8 parallel sessions --- around 100 tok/s aggregate, for the same size of weights and KV cache

You can find the specific hardware/model combo upper bounds in the hugging face space link below, and my formulation as well

Let me know if you find any errors in my upper bound formulation

*Quotes a post by @Ex0byt (https://x.com/Ex0byt/status/2078917396199280724); its text is omitted here because it is not covered by this site's license.*