Entries for August 27, 2026

me: I want to publish in arXiv mom: we have arXiv at home arXiv at home: jokes aside, my astro blog can now render markdown/astro posts as if they were latex papers because latex unfortunately has unreasonable effectiveness in convincing people that an idea is important, even though it may not be my random shower thought got 200k views earlier, whereas what I really thought was a big deal dwindled so maybe this will help it a second time useful for literally everyone who works on optimizing inference Theoretical Upper Bounds for LLM Throughput (wip, shoot corrections in the replies):

I remember being excited about Fable that we finally have a chungus model like GPT 4.5 that can write great prose Oh how naive I was. RLVR or something else in posttraining introduced since ruins models' ability to speak in an understandable tone I have just STRUGGLED trying to make Fable write an abstract, and had to write it myself at the end. We have AGI, but it cannot write an abstract :( I wonder whether the labs are already planning to fix this? Like checking for unreadable writing can easily be made deterministic, with a readability score and such. Though adding a reward over that would probably make things worse, so I'm not sure Like some of these issues must be easy to fix. Consider "sentence parade", where each sentence in a paragraph is completely detached from each other. Like "A does B. C is D. E does F" and so on. For example: > The run reaches 30% of the ceiling. Decode reached 77% to 86% of its bound. The gap says software is the limit. Sustained FLOP/s sits below the bracket. Here is a vibeslopped script which detects such cases with spacy, with surprisingly high precision: github.com/osolmaz/ai-smell/blob/main/analyze_c… I was reading @ben_burtenshaw's preview of his Post-training book---might be a fun weekend project, training a small model for increasing readability

glm 5.3 flash at an antirez-style asymmetric q2 quant could plausibly fit on a single dgx spark at 100~110 GB but with 18b active params vs ds4 flash’s 13b, theoretical decode throughput is only ~70% of that of ds4 flash on the same hardware that is 28 tok/s ds4 vs 19~20 tok/s glm, without speculative decoding, at 32k token filled context afaik it hasn't been quantized in that style yet, but if it were, we would expect such throughput ratio between those models