Post
-
Theoretical limits can expose inference problems
You can use the calculator in localmaxxing.com/en now to predict theoretical upper bounds from solmaz.io/llm-performance-upper-bounds π A little note, pay attention to the speculative decoding coefficient rho. If an upper bound is calculated using rho=1, then dspark might push it well above that limit. For example, the formulation predicts an upper bound of 28 decode tok/s, but dspark pushes it up to 35-50 tok/s in github.com/0xSero/deepseek-v4-flash-0731-spark-β¦ by @0xSero I am curious whether we will observe a phenomenological law between real-life engine performance and the upper bound, like real-life performance maxes out at e.g 80% of the upper bound in most cases That would be very useful! Then we would be able to tell when something is wrong with an inference engine, if it doesn't reach at least 80% of the upper bound, ignoring spec. decoding -