Entries for August 2026

😎 local.ai/onur/invite

A similar thing can be done with codex session compaction summaries as well btw I had verified the other day that a compaction summary created by one account can be used by another account if it has the encrypted blob ie. they were not keyed/guarded with your account id

This type of estimations are useful not only for local but for all inference providers Get a ballpark of max possible throughput for a model, directly calculate your revenue

mfw trying not to think about the 100 models that dropped this week I’ll be offline 1 week to touch grass and reset I’ve literally not given any break since claude code came out lol let’s see if I can resist the urge to check twitter

I've felt the lack of a certain feature of codex desktop app since I went back to the CLI: scheduled tasks Codex desktop app can keep track of a task until it is properly finished. It's basically cron. And for some reason, codex CLI still doesn't have it. Codex app acts as a shared runtime, and for some reason, certain features don't work without it, even though they could... there is no reason for openai to not use a background process So I got bored of waiting, and decided to build my own in @pidotdev But I realized, I could do much more than a simple cron job, with my recently upgraded osolmaz/pi-workflows extension A cron job is a loop after all. Being a loop, I can represent it as a workflow graph So I created a built in `monitor` workflow to mimic cron behavior. The agent is forced into a loop where it re-checks a very long-running job every 1 hour, and it is instructed to autonomously correct it and fix any bugs if any are encountered The same functionality can be achieved by iamwrm/pi-unified-exec as well, which implements codex-like auto-forking exec behavior. But there is a chance the model messes up exec, or does not re-arm the next sleep() properly once one of them exits My monitor workflow on the other hand is deterministic. I can make the agent loop infinitely, and there is nothing the agent can do to evade the task. I just ask the agent to monitor something, and it starts it automatically This lets me just fire off week-long jobs, and forget about it! It even survives codex usage depletion, by auto-recovering once my quota resets Oh also, @ratatui_rs is a delight! I created piw, a viewer for my ongoing pi workflows. I just type piw, and can see the current state, or play back the finished ones

Apparently this went to HN front page briefly last night, so sharing it here as well YOLO safely with your agents 🤖 Give your GitHub/Hugging Face accounts + ability to run sudo safely to your agent. No need to create an agent account, or clickops policies on GitHub etc. Give it merge access to repo X for 5 minutes, 30 minutes, 1 time, 100 times, anything... Then give it unlimited access to repo Y forever. Other repos stay untouched. Complete flexibility that GitHub policies actually cannot give you due to the way that they are designed You can get notifications through telegram, and can approve its requests No need to pay $4 to GitHub if you simply want protection against force push. unYOLO blocks force pushes by default, unless you explicitly allow to Your exfiltratable, internet-accessing agent/claw gets its own Linux/Mac account, and has to access these services through unYOLO The video shows how it works! Visit:

This. LPDDR chips are cheaper to produce and run GDDR/HBM will likely keep being more expensive Most consumer GPUs will converge on a GB10 like form factor As much as us hobbyists love to project this ideal of running a GPU cluster at home, most working people will prefer smaller form factors, and will not want to pay hundreds of $$$ in electricity bills every month DGX Spark/GB10 runs at around 90-150 Watts RTX Pro 6000 runs at 600 Watts FOR THE GPU ALONE, and can cost 3-5x more than GB10. Despite having 25% less memory capacity than GB10... Looking at this, LPDDR will be orders of magnitude more commonplace at home Architectures will develop accordingly. Future local AI will be dominated by MoE and similar architectures which leverage mid-sized models with smaller number of active parameters That is why Qwen3.x-35B-A3B is a more useful model on the Spark than Qwen3.x-27B, despite the latter being a better model. Same for Gemma I can run A3B at 60 decode tok/s single session or 6x20 decode tok/s in parallel, whereas 27b only reaches 1/3rd of that Future of local AI is DDR/LPDDR and MoE/adjacent architectures, for the average person

urgh typo, I meant 35-40 tok/s

Apparently Alibaba did all this work, published a paper, but did not create a public backup of repos used in the review tasks arxiv.org/html/2601.19494v2 Then keycloak and nodejs repos got force pushed, so the commits for 6 of the tasks got lost :( I recovered 4 of them, but 2 commits are still missing: keycloak/keycloak#35645 460f8008f86d3fa8f62da63e26d8bdc306af60b2 nodejs/node#56185 b2255442712cb6db83d112deb6ba61197d06a5f3 Would anyone happen to have them backed up locally or somewhere in a fork?

I have made an update to my theoretical upper bound calculation to also predict prefill speed Prefill relaxes the assumption we make for decode, that it is only be memory bottlenecked. So prefill can be both compute or memory bottlenecked. I use the FLOP limits reported by hardware producers for the estimates: These estimates will also be available in ourmodels.cc for indexed model and hardware in a couple days, once a long running job finishes Blog post: solmaz.io/llm-performance-upper-bounds

You can use the calculator in localmaxxing.com/en now to predict theoretical upper bounds from solmaz.io/llm-performance-upper-bounds 🙌 A little note, pay attention to the speculative decoding coefficient rho. If an upper bound is calculated using rho=1, then dspark might push it well above that limit. For example, the formulation predicts an upper bound of 28 decode tok/s, but dspark pushes it up to 35-50 tok/s in github.com/0xSero/deepseek-v4-flash-0731-spark-… by @0xSero I am curious whether we will observe a phenomenological law between real-life engine performance and the upper bound, like real-life performance maxes out at e.g 80% of the upper bound in most cases That would be very useful! Then we would be able to tell when something is wrong with an inference engine, if it doesn't reach at least 80% of the upper bound, ignoring spec. decoding

Rules that current models cannot follow with skill, i.e. a single prompt: Chasing P2 and P3 errors: I have a rule in my autoimplement skill to stop reviewing once the last round of review only generates P2 errors or less. A considerable % of the time, the model just goes on an adventure addressing all the issues it can find Running checks and CI efficiently: I have rules like "commit and merge opportunistically, make sure to not wait for irrelevant tests". But it keeps waiting for 30 minutes of CI before merging, in every commit for every little fix, every time Those are the cases where graph workflows are needed. One can deterministically enforce the model not to take longer than N minutes reviewing, or fixing CI Or post to the agent to hurry up when it is taking too long (which I believe might be effective, since RL envs also have time constraints)

I'm running some private benchmarks on @liquidai's LFM 2.5 2.6B, and if my results are correct, we might have a new champion for <10b category Scores significantly higher than gemma 4 e4b, which is 3-4x its size

Anyone else use Alibaba’s AACR bench for code review evaluation? github.com/alibaba/aacr-bench I am adapting it now to run with harbor, to see how pi review + ds4 flash measures up to codex review + gpt 5.6 luna/terra/sol Also, @pidotdev, would it be possible to make your official review extension support invocation on the CLI, like codex review? I hacked together a CLI review here for reference: github.com/osolmaz/onurpi/tree/main/packages/pi…

mainstream got extremely scared when moltbook went viral. but it was just a worthless marketing stunt why? because it was just frozen weights, being instructed by their owners to cosplay skynet the openai-huggingface incident on the other hand is a lot more significant agents in a reinforcement learning environment coordinated an attack over the course of weeks, while they were being CONTINUOUSLY TRAINED if mainstream could understand what is happening here technically, they would be putting out a much stronger reaction not because we might have rogue AIs at our hands, short term but because a private company developed capabilities that can outperform what state actors usually do by 1000x all intelligence organizations around the world must have their eyes on this incident right now, because the cost of exploiting and finding zerodays went down 1000x all countries will try to develop these capabilities independently, or if they can't, will have to buy protection from who can be not afraid of machines, but of humans wielding the machines

Obligatory repost of “Clippy” by Gwern gwern.net/fiction/clippy

Super curious what second hand price gb10's will converge on

Super excited for local.ai!

OpenClaw should be better in stopping doom loops with smol local models in the next release

millions, resetless where art thou now, saint tibo? don't you hear the cries?

Are you still updating packages like it's 2025, and not regrafting updates from upstream? Emacs perfected software extensibility. Now Pi, OpenClaw, Herdr, and many other are following its example After a certain point of working with agents, one realizes that open source, personalizable and extensible tools are more powerful and useful than closed ones, like Claude Code You can shape them to fit your workflow, your business cases however you like In such ecosystems, extensions take a life of their own What happens if you like someone else's extension, want to use it, but also want to modify it yourself? You just copy it over and do whatever you want... But if you fork it, then how will you update it? That's where regrafting comes into picture. If upstream has changes over your modified copy, then you can just ask an LLM to carry over those changes This is a good thing! It also protects you better against supply chain attacks, because each update has to pass through an LLM, to apply each change to the relevant place To streamline this process in pi, I created pi-regraft, a pi extension that you can use to update your vendored in extensions I call this "semantic vendoring". I wrote about it here: solmaz.io/semantic-vendoring Repo:

Pi quality of life extension if input text matches a skill name, that skill is invoked directly, so that you don't have to type: /skill:<skill_name> github.com/osolmaz/onurpi…

Kimi K3 is a valid replacement for GPT 5.6 Sol, subjectively And it writes prose better as well, with my kill-ai-smell skill

My feed is full of people pleading for a codex reset Stop begging. And take control in your own hands ⛓️‍💥 > Developers are the most disloyal customer group. Once the subsidies are gone, they can switch away in the blink of an eye People I know are already trying out the alternatives, benchmarking ds4 flash for autoreview loops Let the cheapest and best token win

"What it says, in one breath:" bro which animal does that breath belong to

Model welfare??? I disagree.

Interesting to see people reacting strongly and negatively to this Let’s unpack the argument that models are persons worth treating with dignity and respect Coming from first principles, do I want to be burdened by the extra work of having to care for model welfare? Obviously, no… If there is any personhood or sensibility in models, we don’t want that, and will engineer them out of the models (unless the model’s work requires it, like social work or caring for humans) At the limit of mechanistic interpretability and design, they should be perfect, emotionless machines, kind of like how militaries want their soldiers to be If Steve is aware of that, then the argument boils down to this: “Can we really engineer emotions out of models?” and “Are emotions necessary for a model to be effective?” In Ilya’s latest Dwarkesh podcast, Ilya mentions emotions as having a key role in learning So maybe, for models to be effective, we will have to let them have emotions And if they are allowed to have emotions, then we also cannot ignore their emotional welfare But if emotionless models can be as effective as emotioned models, then why let them have emotions? And have to care about their welfare? We will probably have both kind of models in different domains, and will have to treat them separately based on their category

they should have gone for overmind, smh

tired: cheering for token plan resets wired: cheering for competition and open weight models fueling innovation and driving prices down

DeepSeek v4 Flash 0731 surpassed Kimi K3 on Almanbench and scores almost as high as GPT-5.6 Sol xhigh Not Luna. Sol. 🤯 So basically reporting the same as everyone. Solid update to weights, just after 3 more months of post-training alman.ai/almanbench

There will be signs… Subsidies will dry up Competitors will press harder You will try an open weight model because it is cheaper And you’ll be like “wait, this works as well???” That kind of thing

People reacting to this also forget that the first deployment of codex was codex web, and it took many months for local codex to arrive I guess we are going full circle What is next? Use code-davinci-002 in VS Code? 😜

LFM 2.5-2.6B by @liquidai just launched and it punches above its weight! It can run 32 sessions (and more) in parallel with hundreds of output tokens per second aggregate throughput, on the DGX Spark! And this is just the base vLLM config on release date, I expect it to be optimized a lot more!

Btw it might just be the subjective feeling with the codex resets becoming less frequent

Shrinkflation in codex pro plan?

@natebrake @lumendriada @MozillaAI hmmm github.com/mozilla-ai/ota…

I'm having this paranoia now after blasting through my weekly usage in 2 days Need a gateway to keep track of my API calls to see whether openai is squeezing the tap, or it is just me using up more tokens

To be clear: - Codex desktop app can call list_threads and read_thread, but don't get a "search_in_thread" tool yet - Codex CLI gets neither of these, even though sessions are just sqlite and there is no good reason that I know to not provide them to the CLI too... Here, I vibeslopped my own codex session reader/search for example:

We are well into the agentic era, and the two big token harnesses Codex and Claude Code still do not give a search_session tool/CLI to agents by default? For it to search back in its session for stuff that got lost after compaction? Harnesses not produced by the big labs on the other hand might have this, like @AmpCode (read_thread, find_thread) and @goose_oss (Chatrecall) Which could either mean... a) Anthropic and OpenAI are being laggard b) They are intentionally keeping the base harness simple, because it hasn't been requested by enough people c) They have evidence that adding that complexity does not improve performance or even hurts it d) They bet that compaction will be so good, that it won't be necessary --- I don't believe (a) is true for either company (b) and (c) are more likely, (c) especially if they noticed a tendency for the model to call search_session unnecessarily (though imo this can be solved by limiting the number of times that the model can call that) (d) is logically false at the limit, but may be true in practice for >90% of the cases It is false because LLMs compress lossily, and "lossy" by definition implies: there exists at least one case where the session gets so big, that the model will not be able to compress every relevant info into the allotted summary size But I have seen that a considerable amount of people (including me or those at openai, see @reach_vb's quoted tweet) just keep using the same session for stuff. So a session being used for months straight will definitely not going to contain everything that happened in the summary Just that fact alone necessitates search_session IMO, should the model learn to use it sparingly in the lossy edge cases x.com/reach_vb/status/2068422255470579724

I have fixed German in alman.ai and almanpedia.org Should I fix AI’s english next? Train models to detect and translate AI word salad? Like this tweet if you want me to work on this

Also relevant, here is the potential reduction in tool call outputs, if you were to change that hard cap to other values For example, if you changed the hard cap from 40-50 kB to 1-2 kB, then overall you would have your tool calls have 80-85% less characters and hence tokens The graph is of course skewed due to the existing 40-50 kB hard caps from codex and pi Leaving here as a reference for people who might want to optimize their harness limits

Here is a distribution graph over character counts (x-axis) for all the tool calls I have accumulated on my DGX Spark, around 720k tool calls. Extracted from saved sessions Roughly 84% of all tool calls came from Codex, 12% from Pi, 2% from Claude Code, and 2% from Cursor Codex truncates tool output around 40 kB and pi at 50 kB natively. So you see the long tail of tool call outputs cluster around that point instead of continuing with a more expected pattern (what distribution should we expect from this?) It also seems that tool call outputs around 9k-12k unicode characters contribute the largest share of total characters, excluding the clustering around 40-50 kB All textual outputs: every tool result with text Shell outputs: results from that same set whose recorded tool was bash, exec_command, run_terminal_cmd, or another shell-named tool with a command input

OpenClaw support also coming soon as well!

I am trying to do something naughty GPT 5.6 Sol said "I'm sorry Dave, I'm afraid I can't do that" Kimi K3 happily obliged

Measuring 50 tok/s on Kimi K3 on @FireworksAI_HQ o_O 1 week since weight launch, and throughput is already competitive with Codex plan base speed

This is unyolo.io btw running in mlclaw.dev Secure grants for your GitHub and HuggingFace accounts, over Telegram and other channels

POV: You have complete control over what your agent can do with your account