Entries for July 31, 2026

The 3-bit one leave just enough space for me to browse twitter on the dgx spark, perfect

Then it will show while visiting a model page, what decode tok/s to expect from that model. Here it is 28 tok/s for antirez/ds4

You can choose your hardware by clicking the button on the top right

Sharing a preview into a work I've been doing for a while, to get feedback. Still work in progress ourmodels.cc --> A database of sorts for open weight/local models, hardware, and other stuff, in a way I haven't seen elsewhere yet If you remember my earlier theoretical throughput limits work, I had created a calculator to show what throughput a certain @huggingface model can reach on e.g. DGX Spark You can now select your own hardware (e.g. DGX Spark), and then the website shows you the theoretical limits for each model you see I have improved the UX on that website, and combined it with a knowledge graph of tweets from this site, so that you can see all the commentary, benchmarks etc. about that model in one place! If you are the publisher or an enthusiast of a model, you will be able to go there and see what feedback people have posted about, it in one place It extracts topics as well, so you can see all llama.cpp related posts for example All from a curated set of creators (curated by me, that is) It currently has posts from the last 5-10 days, but I will work to expand that to months, based on the feedback I get here Let me know what you think! Whether you would like to see a certain feature! (fyi, every number you see on this website is a theoretical upper bound, not a submitted benchmark) Site: ourmodels.cc

me recently: just look into how codex implements X, and build an extension for that on our end

Whatever is happening to sol recently, it is definitely not the same quality for the price

I am not sure how accurate this is since 0731 weights should perform substantially better, increasing the denominator in (cost / intelligence), hence beating the preview weights However, ds4 flash and gpt 5.6 luna all being clustered at the frontier makes sense

Quick comparison of reported DeepSeek-V4-Flash-0731 benchmark results vs Claude vs GPT Looks like we will have GPT-5.6-Luna-ish at home (which just had a huge price cut) Likely to be priced competitively from other inference providers in the long run as well Compare @ArtificialAnlys Cost per Intelligence Index Task below

I had Sonnet 4 level experience today with gpt 5.6 sol high today in pi Is anyone else feeling a drop in response quality, or is it my harness?

And the vLLM blog post for those who are interested in the internals: vllm.ai/blog/2026-06-1…

Parameter sweep by localperf: github.com/osolmaz/localp…

Ever wondered what token generation looks like in a Diffusion LLM (dLLM)? @googlegemma DiffusionGemma by generates text in 256 token blocks Each block starts with random tokens On every pass, the model proposes tokens for all 256 slots in parallel. It keeps the more confident proposals and replaces the uncertain ones with new random tokens This is called denoising. Denoising repeats until the block becomes stable and confident The model then commits the finished block and starts the next one I had to patch @vllm_project to be able to visualize the canvas, and had to develop yet another custom bundle of pi, osolmaz/diffusionpi You can see all that in action in 4 parallel sessions below in glorious 4K (DGX Spark can do 16 sessions in parallel as well, but then it is not so easy to see what is going on) But the video below is not the most objective benchmark of throughput. To measure tok/s more objectively (i.e. repeatable and fixed length), I made osolmaz/localperf ignore EOS token to keep request lengths equal. Result: - nvidia/diffusiongemma-26B-A4B-it-NVFP4 - NVIDIA DGX Spark, vLLM - c4x66 ~ 264 aggregate tok/s at - 32k context config each, 1k input tokens, fixed 512-token outputs, thinking off You can see the full profiling sweep in the post below 👇