Entries for September 7, 2026

Can we show my wife @qrlow some love 🤗🤗🤗 She's just started dipping her toes in evaluating LLMs in her own field I helped her out just with running the benchmark and she solved some of the issues pretty creatively, like how to come up with synthetic tasks that match what usually happens inside a trading house She is on Hugging Face too 🤗 No better place than to publish a benchmark I have been encouraging all my close circle to go towards benchmarking models as well Literally every company that uses AI in any way will have to measure their performance, on internal metrics Especially if they start leveraging their data, training their own models and such So every employee will benefit from such know-how The most important one that comes after knowing how to use agents is knowing how to measure them If you are worried of becoming obsolete, one of the highest leverage things you could learn to do would be to try out running some benchmarks, learn what @harborframework does, converting your own companies' mundane tasks into evals, and so on

close enough

Struggling to create the level of quality of 3D modeling some of the OAI employees and early accessers are posting with GPT 6 Astra It is OK, but one-shot performance is not mind-blowing like some people were claiming Do the better runs use Blender MCP? Or sth else? I prompted it to put it on a website the first time. On the second time, I prompted it to use purely blender, in a new session. It still ended up putting it on a website, with similar performance Space: huggingface.co/spaces/osolmaz/iss-atlas === prompt: use blender to create a realistic exposition of the internation space station, orbiting earth, in an as realistic way as possible should give a tour of the space station, show insides, cross sections, different modules, what everything does, including humans that work on stuff, maybe docking shuttles and stuff, its phases through time, in a nice and cinematic way, that also feels like sims or some game should eventually be put on a static hugging face space, a website in typescript inspiration should be one of those books from star wars the phantom menace where you could see to the inside of the spaceship and such should be interactive and playable and fun not sure whether blender is needed, as long as you can put it on a website, anything works

Medicine for the vibe-coded brain: Reverse-centaur mode AI plans things, but it doesn't execute for you It tells you what commands to run and what files to write, line by line *You* execute them. You run into errors. You question. You learn It slows you down a LOT. But you learn a lot more than just prompting Use it when you feel like you are not learning enough Install the skill, then just say "Activate reverse-centaur mode" Prepare to remember how annoying but educative the whole process used to be

"we deliberately designed the product to hide the chain of thought⁠, to protect it from supervision pressure in the long term^2" *Moves over to the footnote* "A secondary reason for this design was preventing distillation. However, maintaining CoT monitorability has explicitly been the bigger priority for us throughout development." I was about to tweet about dishonesty in this post. Then I decided to Ctrl+F and search for distillation Might be the first time this was said publicly through an official channel. I am not so sure about maintaining competitive edge being lower priority than safety. Distillation is an existential threat to frontier labs I wish @merettm posted more. It really increases trust in OpenAI, compared to some other people Now, to pay off the honesty debt fully, they should clear up whether they knew about collusion.wiki already, or any other rogue action for that matter