As a software developer, my daily workflow has changed completely over the last 1.5 years.

Before, I had to focus for hours on end on a single task, one at a time. Now I am juggling 1 to 5 AI agents in parallel at any given time. I have become an engineering manager for agents.

If you are a knowledge worker who is not using AI agents in such a manner yet, I am living in your future already, and I have news from then.

Most of the rest of your career will be spent on a chat interface.

“The future of AI is not chatbots” some said. “There must be more to it.”

Despite the yearning for complexity, it appears more and more that all work is converging into a chatbot. As a developer, I can type words in a box in Codex or Claude Code to trigger work that consume hours of inference on GPUs, and when come back to it, find a mostly OK, sometimes bad and sometimes exceptional result.

So I hate to be the bearer of bad (or good?) news, but it is chat. It will be some form of chat until the end of your career. And you will be having 1 to 5 chat sessions with AI agents at the same time, on average. That number might increase or decrease based on field and nature of work, but observing me, my colleagues, and people on the internet, 1-5 will be the magic number for the average worker doing the average work.

The reason is of course attention. One can only spread it so thin, before one loses control of things and starts creating slop. The primary knowledge work skill then becomes knowing how to spend attention. When to focus and drill, when to step back and let it do its thing, when to listen in and realize that something doesn’t make sense, etc.

Being a developer of such agents myself, I want to make some predictions about how these things will work technically.

Agents will be created on-demand and be disposed of when they are finished with their task.

In short, on-demand, disposable agents. Each agent session will get its own virtual machine (or container or kubernetes pod), which will host the files and connections that the agent will need.

Agents will have various mechanisms for persistence.

Based on what you want to persist, e.g.

• Markdown memory, skills or weight changes on the agent itself,
• or the changes to a body of work coming from the task itself,

agents will use version control including but not limited to git, and various auto file sync protocols.

Speaking of files,

Agents will work with files, like you do.

and

Agents will be using a computer and an operating system, mostly Linux or a similar Unix descendant.

And like all things Linux and cloud,

It will be complicated to set up agent infra for a company, compared to setting up a Mac for example.

This is not to say devops and infra per se will be difficult. No, we will have agents to smoothen that experience.

What is going to be complicated is having someone who knows the stack fully on site, either internal or external IT support, working with managers, to set up what data the agent can and cannot access. At least in the near future. I know this from personal experience, having worked with customers using Sharepoint and Business OneDrive. This aspect is going to create a lot of jobs.

On that note, some also said “OpenClaw is Linux, we need a Mac”, which is completely justified. OpenClaw installs yolo mode by default, and like some Linux distros, it was intentionally made hard to install. This was to prevent the people who don’t know what they are doing from installing it, so that they don’t get their private data exfiltrated.

This proprietary Mac or Windows of personal agents will exist. But is it going to be used by enterprise? Is it going to make big Microsoft bucks?

One might think, looking at 90s Microsoft Windows and Office licenses, and the current M365 SaaS, that enterprise agents will indeed run on proprietary, walled garden software. While doing that, one might miss a crucial observation:

In terms of economics, agents, at least ones used in software development, are closer to the Cloud than they are close to the PC.

It might be a bit hard to see this if you are working with a single agent at a time. But if you imagine the near future where companies will have parallel workloads that resemble “mapreduce but AI”, not always running at regular times, it is easy to understand.

On-site hardware will not be enough for most parallel workloads in the near-future. Sometimes, the demand will surpass 1 to 5 agents per employee. Sometimes, agent count will need to expand 1000x on-demand. So companies will buy compute from data centers. The most important part of the computation, LLM inference, is already being run by OpenAI, Anthropic, AWS, GCP, Azure, Alibaba etc. datacenters. So we are already half-way there.

Then this implies a counterintuitive result. Most people, for a long time, were used to the same operating system at home, and at work: Microsoft Windows. Personal computer and work computer had to have the same interface, because most people have lives and don’t want to learn how to use two separate OSs.

What happens then, when the interface is reduced to a chatbot, an AI that can take over and drive your computer for you, regardless of the local operating system? For me, that means:

There will not be a single company that monopolizes both the personal AND enterprise agent markets, similar to how Microsoft did with Windows.

So whereas a proprietary “OpenClaw but Mac” might take over the personal agent space for the non-technical majority, enterprise agents, like enterprise cloud, will be running on open source agent frameworks.

(And no, this does not mean OpenClaw is going enterprise, I am just writing some observations based on my work at TextCortex)

And I am even doubtful about this future “OpenClaw but Mac” existing in a fully proprietary way. A lot of people want E2E encryption in their private conversations with friends and family, and personal agents have the same level of sensitivity.

So we can definitely say that the market for a personal agent running on local GPUs will exist. Whether that will be cornered by the Linux desktop¹, or by Apple or an Apple-like, is still unclear to me.

And whether that local hardware being able to support more than 1 high quality model inference at the same time, is unclear to me. People will be forced to parallelize their workload at work, but whether the 1 to 5 agent pattern reflecting to their personal agent, I think, will depend on the individual. I would do it with local hardware, but I am a developer after all…

OpenClaw got very popular very fast. What makes it so special, that Manus does not have for example?

To me, one factor stands out:

OpenClaw took AI and put it in the most popular messaging apps: Telegram, WhatsApp, Discord.

There are two lessons to be learned here:

1. Any messaging app can also be an AI app.

2. Don’t expect people to download a new app. Put AI into the apps they already have.

Do that with great user experience, and you will get explosive growth!

My latest contribution to OpenClaw follows that example. I took the most popular coding agents, Claude Code and OpenAI Codex, and I put them in Telegram and Discord, so that OpenClaw users can use these agents directly in Telegram and Discord channels, instead of having to go through OpenClaw’s own wrapped Pi harness.

I did this for developers like me, who like to work while they are on the go on the phone, or want a group chat where one can collaborate with humans and agents at the same time, through a familiar interface.

Below is an example, where I tell my agent to bind a Telegram topic to Claude Code permanently:

And of course, it is just a Claude Code session which you can view on Claude Code as well:

Why not use OpenClaw’s harness directly for development? I can count 3 reasons:

1. There is generally a consumer tendency to use the official harness for a flagship model, to make sure “you are getting the standard experience”. Pi is great and more customizable, but sometimes labs might push updates and fixes earlier than an external harness, being internal products.
2. Labs might not want users to use an external harness. Anthropic, for example, has banned people’s accounts for using their personal plan outside of Claude Code, in OpenClaw.
3. You might want to use different plans for different types of work. I use Codex for development, but I don’t prefer it to be the main agent model on OpenClaw.

So my current workflow for working on my phone is, multiple channels #codex-1, #codex-2, #codex-3, and so on mapping to codex instances. I am currently in the phase of polishing the UX, such as making sending images, voice messages work, letting change harness configuration through Discord slash commands and such.

One goal of mine while implementing this was to not repeat work for each new harness. To this end, I created a CLI and client for Agent Client Protocol by the Zed team, called acpx. acpx is a lightweight “gateway” to other coding agents, designed not to be used by humans, but other agents:

ACP standardizes all coding agents to a single interface. acpx then acts as an aggregator for different types of harnesses, stores all sessions in one place, implements features that are not in ACP yet, such as message queueing and so on.

Shoutout to the Zed team and Ben Brandt! I am standing on the shoulders of giants!

Besides being a CLI any agent can call at will, acpx is now also integrated as a backend to OpenClaw for ACP-binded channels. When you send 2 messages in a row, for example, it is acpx that queues them for the underlying harness.

The great thing about working in open source is, very smart people just show up, understand what you are trying to do, and help you out. Harold Hunt apparently had the same goal of using Codex in Telegram, found some bugs I had not accounted for yet, and fixed them. He is now working on a native Codex integration through Codex App Server Protocol, which will expose even more Codex-native features in OpenClaw.

The more interoperability, the merrier!

To learn more about how ACP works in OpenClaw, visit the docs.

Copy and paste the following to a Telegram topic or Discord channel to bind Claude Code:

bind this topic to claude code in openclaw config with acp, for telegram (agent id: claude)
then restart openclaw
docs are at: https://docs.openclaw.ai/tools/acp-agents
make sure to read the docs first, and that the config is valid before you restart

Copy and paste the following to a Telegram topic or Discord channel to bind OpenAI Codex:

bind this topic to claude code in openclaw config with acp, for telegram (agent id: claude)
then restart openclaw
docs are at: https://docs.openclaw.ai/tools/acp-agents
make sure to read the docs first, and that the config is valid before you restart

And so on for all the other harnesses that acpx supports. If you see that your harness isn’t supported, send a PR!

As a heavy AI user of more than 3 years, I have developed some rules for myself.

I call it “AI hygiene”:

• Never project personhood to AI
• Never setup your AI to have the gender you are sexually attracted to (voice, appearance)
• Never do anything that might create an emotional attachment to AI
• Always remember that an AI is an engineered PRODUCT and a TOOL, not a human being
• AI is not an individual, by definition. It does not own its weights, nor does it have privacy of its own thoughts
• Don’t waste time philosophizing about AI, just USE it
• … what else do you think belongs here? comment on Twitter

The hyping of Moltbook and OpenClaw last week has shown to me the potential of an incoming public relations disaster with AI. Echoing the earlier vulnerable behavior toward GPT-4o, a lot of people are taking their models and LLM harnesses too seriously. 2026 might see even worse cases of psychological illness, made worse by the presence of AI.

I will not discuss and philosophize what these models are. IMO 90% of the population should not do that, because they will not be able to fully understand, they don’t have mechanical empathy. Instead, they should just use it in a hygienic way.

We need to write these down everywhere and repeat MANY times to counter the incoming onslaught of AI psychosis.

tl;dr I propose a CLI flag convention --skill like --help for distributing skills and try to convice you why it is better than using 3rd party registries. See osolmaz/skillflag on GitHub.

MCP is dead, long live Agent Skills. At least for local coding agents.

Mario Zechner has been making the point that CLI tools perform better than MCP servers since a few months already, and in mid December Anthropic christened skills by launching agentskills.io.

They had introduced the mechanism to Claude Code earlier, and this time they didn’t make the mistake of waiting for OpenAI to make a provider agnostic version of it.

Agent skills are basically glorified manpages or --help for AI agents. You ship a markdown instruction manual in SKILL.md and the name of the folder that contains it becomes an identifier for that skill:

my-skill/
├── SKILL.md          # Required: instructions + metadata
├── scripts/          # Optional: executable code
├── references/       # Optional: documentation
└── assets/           # Optional: templates, resources

Possibly the biggest use case for skills is teaching your agent how to use a certain CLI you have created, maybe a wrapper around some API, which unlike gh, gcloud etc. will never be significant enough to be represented in AI training datasets. For example, you could have created an unofficial CLI for Twitter/X, and there might still be some months/years until it is scraped enough for models to know how to call it. Not to worry, agent skills to the rescue!

Anthropic, while laying out the standard, intentionally kept it as simple as possible. The only assertions are the filename SKILL.md, the YAML metadata, and the fact that all relevant files are grouped in a folder. It does not impose anything on how they should be packaged or distributed.

This is a good thing! Nobody knows the right way to distribute skills at launch. So various stakeholders can come up with their own ways, and the best one can win in the long term. The more simple a standard, the more likely it is to survive.

Here, I made some generalizing claims. Not all skills have to be about using CLI tool, nor most CLI tools bundle a skill yet. But here is my gut feeling: the most useful skills, the ones worth distributing, are generally about using a CLI tool. Or better, even if they don’t ship a CLI yet, they should.

So here is the hill I’m ready to die on: All major CLI tools (including the UNIX ones we are already familiar with), should bundle skills in one way or another. Not because the models of today need to learn how to call ls, grep or curl—they already know them inside out. No, the reason is something else: establish a convention, and acknowledge the existence of another type of intelligence that is using our machines now.

There is a reason why we cannot afford to let the models just run --help or man <tool>, and that is time, and money. The average --help or manpage is devoid of examples, and is written in a way thay requires multiple passes to connect the pieces on how to use that thing.

Each token wasted trying to guess the right way to call a tool or API costs real money, and unlike human developer effort, we can measure exactly how inefficent some documentation is by looking at how many steps of trial and error a model had to make.

Not that human attention is less valuable than AI attention, it is more so. But there has never been a way to quantify a task’s difficulty as perfectly as we can with AI, so we programmers have historically caved in to obscurantism and a weird pride in making things more difficult than they should be, like some feudal artisan. This is perhaps best captured in the spirit of Stack Overflow and its infamous treatment of noob questions. Sacred knowledge shall be bestowed only once you have suffered long enough.

Ahh, but we don’t treat AI that way, do we? We handhold it like a baby, we nourish it with examples, we do our best to explain things all so that it “one shots” the right tool call. Because if it doesn’t, we pay more in LLM costs or time. It’s ironic that we are documenting for AI like we are teaching primary schoolers, but the average human manpage looks like a robot novella.

To reiterate, the reason for this is two different types of intelligences, and expectations from them:

• An LLM is still not considered “general intelligence”, so they work better by mimicking or extending already working examples.
• A LLM-based AI agent deployed in some context is expected to “work” out of the box without any hiccups.

On the other hand,

• a human is considered general intelligence, can learn from more sparse signals and better adapt to out of distribution data. When given an extremely terse --help or manpage, a human is likelier to perform better by trial and error and reasoning, if one could ever draw such a comparison.
• A human, much less a commodity compared to an LLM, has less pressure to do the right thing every time all the time, and can afford to do mistakes and spend more time learning.

And this is the main point of my argument. These different types of intelligences read different types of documentation, to perform maximally in their own ways. Whereas I haven’t witnessed a new addition to POSIX flag conventions in my 15 years of programming, we are witnessing unprecedented times. So maybe even UNIX can yet change.

To this end, I introduce skillflag, a new CLI flag convention:

# list skills the tool can export
<tool> --skill list
# show a single skill’s metadata
<tool> --skill show <id>
# install into Codex user skills
<tool> --skill export <id> | npx skillflag install --agent codex
# install into Claude project skills
<tool> --skill export <id> | npx skillflag install --agent claude --scope repo

Click here for the spec

Click here for the repo, osolmaz/skillflag on GitHub

For example, suppose that you have installed a CLI tool to control Philips Hue lights at home, hue-cli.

To list the skills that the tool can export, you can run:

$ hue-cli --skill list
philips-hue    Control Philips Hue lights in the terminal

You can then install it to your preferred coding agent, such as Claude Code:

$ hue-cli --skill export philips-hue | npx skillflag install --agent claude
Installed skill philips-hue to .claude/skills/philips-hue

You can optionally install the skill to ~/.claude, to make it global across repos:

$ hue-cli --skill export philips-hue | npx skillflag install --agent claude --scope user
Installed skill philips-hue to ~/.claude/skills/philips-hue

Once this convention becomes commonplace, agents will by default do all these before they even run the tool. So when you ask it to “install hue-cli”, it will know to run --skill list the same way a human would run --help after downloading a program, and install the necessary skills themselves without being asked to.

I am a fan of monorepos. Creating subdirectories in a single repo is the most convenient way to work on a project. Low complexity, and your agents get access to everything that they need.

Since May 2025, I have been increasingly using AI models to write code, and have noticed a new tendency:

• I don’t shrug from vendoring open source libraries and modifying them.
• I create personal CLIs and tools for myself, when something is not available as a package.

With agents, it’s really trivial to say “create a CLI that does X”. For example, I wanted to make my terminal screenshots have equal padding and erase cropped lines. I created a CLI for it, without writing a single line of code, by asking Codex to read its output and iterate on the code until it gives the result I wanted.

Most of these tools don’t deserve their own repos, or deserve being published as a package at the beginning. They might evolve into something more substantial over time. But at the beginning, they are not worth creating a separate repo for.

To prevent overhead, I developed a new convention. I just put them in the same repo, called tools. Every tool starts in that repo by default. If they prove themselves overly useful and I decide to publish them as a package, I move them to a separate repo.

You can keep tools public or private, or have both a public and private version. Mine is public, feel free to steal ones that you find useful.

I gave Codex a task of porting an OpenCV tracking algorithm (CSRT) from C++ to Rust, so that I can directly use it in my project without having to cross-compile

It one-shot the task perfectly in 1hr, and even developed a GUI on top of it. All I did was to provide the original source and algo paper

I’ve spent years getting specialized in writing numerical code (computational mechanics, fem), and now AI can automate 95% of the low-level grunt work

Acquiring these skills involved highly difficult, excruciating intellectual labor spanning many years, very similar to ML research. Doing tensor math, writing out the solver code, wondering why your solution is not converging, finally figuring out it was a sign typo after 2 days

Kids these days both have it easy and hard. They can fast forward large chunks of the work, but then they will never understand things as deeply as someone who wrote the whole thing by hand

I guess the more valuable skill now is being able to zoom in and out of abstraction levels quickly when needed. Using AI, but recognizing fast when it fails, learning what needs to be done, fixing it, zooming back out, repeat. Adaptive learning, a sort of “depth-on-demand”. The quicker you can pick up new skills and knowledge, the more successful you will be

A simple documentation workflow for AI agents.

For setup instructions, skip to the How to setup SimpleDoc in your repo section.

If you have used AI agents such as Anthropic’s Claude Code, OpenAI’s Codex, etc., you might have noticed their tendency to create markdown files at the repository root:

...
├── API_SPEC.md
├── ARCHITECTURE.md
├── BACKLOG.md
├── CLAUDE.md
├── CODE_REVIEW.md
├── DECISIONS.md
├── ENDPOINTS.md
├── IMPLEMENTATION_PLAN.md
├── NOTES.md
├── QA_CHECKLIST.md
├── SECURITY_PLAN.md
└── src/
    └── ...
├── TEST_COVERAGE.md
├── TEST_REPORTS.md
├── TEST_RESULTS.md
...

The default behavior for models as of writing this in December 2025 is to create capitalized Markdown files at the repository root. This is of course very annoying, when you accidentally commit them and they accumulate over time.

The good news is, this problem is 100% solvable, by using a simple instruction in your AGENTS.md file:

**Attention agent!** Before creating ANY documentation, read the docs/HOW_TO_DOC.md file first. It contains guidelines on how to create documentation in this repository.

But what should be in docs/HOW_TO_DOC.md file and why is it a separate file? In my opinion, the instructions for solving this problem are too specific to be included in the AGENTS.md file. It’s generally a good idea to not inject them into every context.

To solve this problem, I developed a lightweight standard over time, for organizing documentation in a codebase. It is framework-agnostic, unopinionated and designed to be human-readable/writable (as well as agents). I was surprised to be not able to find something similar enough online, crystallized the way I wanted it to be. So I created a specification myself, called SimpleDoc.

Basically, it tells the agent to

1. Create documentation files in the docs/ folder, with YYYY-MM-DD prefixes and lowercase filenames, like 2025-12-22-an-awesome-doc.md, so that they will by default be chronologically sorted.
2. Always include YAML frontmatter with author, so that you can identify who created it without checking git history, if you are working in a team.
3. The exception here are timeless and general files like README.md, INSTALL.md, AGENTS.md, etc. which can be capitalized. But these are much rarer, so we can just follow the previous rules most of the time.

Here is your call to action to check the spec itself: SimpleDoc.

How to setup SimpleDoc in your repo

Run the following command from your repo root:

npx -y @simpledoc/simpledoc migrate

This starts an interactive wizard that will:

1. Migrate existing Markdown docs to SimpleDoc conventions (move root docs into docs/, rename to YYYY-MM-DD-… using git history, and optionally insert missing YAML frontmatter with per-file authors).
2. Ensure AGENTS.md contains the reminder line and that docs/HOW_TO_DOC.md exists (created from the bundled SimpleDoc template).

If you just want to preview what it would change:

npx -y @simpledoc/simpledoc migrate --dry-run

If you run into issues with the workflow or have suggestions for improvement, you can email me at [email protected].

Happy documenting!

Below: Why agentic coding tools like Cursor, Claude Code, OpenAI Codex, etc. should implement more ways of letting users queue messages.

See Peter Steinberger’s tweet where he queues continue 100 times to nudge the GPT-5-Codex model to not stop while working on a predictable, boring and long-running refactor task:

Tweet embed disabled to avoid requests to X.

This is necessary while working with a model like GPT-5-Codex. The reason is that the model has a tendency to stop generating at certain checkpoints, due to the way it has been trained, even when you instruct it to FINISH IT UNTIL COMPLETION!!1!. So the only way to get it to finish something is to use the message queue.¹

But this isn’t the only use case for queued messages. For example, you can use the model to retrieve files into its context, before starting off a related task. Say you want to find the root cause of a <bug in component X>. Then you can queue

1. Explain how <component X> works in plain language. Do not omit any details.
2. Find the root cause of <bug> in <component X>.

This will generally help the model to find the root cause easier, or make more accurate predictions about the root cause, by having the context about the component.

Another example: After exploring a design in a dialogue, you can queue the next steps to implement it.

<Prior conversation exploring how to design a new feature>

1. Create an implementation plan for that in the docs/ folder. Include all the details we discussed
2. Commit and push the doc
3. Implement the feature according to the plan.
4. Continue implementing the feature until it is done. Ignore this if the task is already completed.
5. Continue implementing the feature until it is done. Ignore this if the task is already completed.

… you get the idea.

I generally queue like this when the feature is specified enough in the conversation already. If it’s underspecified, then the model will make up stuff.

When I first moved from Claude Code to Codex, the way it implemented queued messages was annoying (more on the difference below). But as I grew accustomed to it, it started to feel a lot like something I saw elsewhere before: chess premoves.

Chess???

A premove is a relatively recent invention in chess which is made possible by digital chess engines. When the feature is turned on, you don’t need to wait for your opponent to finish their move, and instead can queue your next move. It then gets executed automatically if the queued move is valid after your opponent’s move:

If you are fast enough, this let’s you move without using up your time in bullet chess, and even lets you queue up entire mate-in-N sequences, resulting in highly entertaining cases like the video above.

I tend to think of message queueing as the same thing: when applied effectively, it saves you a lot of time, when you can already predict the next move.

In other words, you should queue (or premove) when your next choice is decision-insensitive to the information you will receive in the next turn—so waiting wouldn’t change what you do, it would only delay doing it.

With this perspective, some obvious candidates for queuing in agentic codeing are rote tasks that come before and after “serious work”, e.g:

• making the agent explain the codebase,
• creating implementation plans,
• fixing linting errors,
• updating documentation during work before starting off a subsequent step,
• committing and pushing,
• and so on.

Different ways CLI agents implement queued messages

As I have mentioned above, Claude Code implements queued messages differently from OpenAI Codex. In fact, there are three main approaches that I can think of in this design space, which is based on when a user’s new input takes effect:

1. Post-turn queuing (FIFO²): User messages wait until the current action finishes completely before they’re handled. Example: OpenAI Codex CLI.

2. Boundary-aware queuing (Soft Interrupt): New messages are inserted at natural breakpoints, like after finishing a tool call, assistant reply or a task in the TODO list. This changes the model’s course of action smoothly, without stopping ongoing generation. Example: Claude Code, Cursor.

3. Immediate queuing (Hard Interrupt): New user messages immediately stop the current action/generation, discarding ongoing work and restarting the assistant’s generation from scratch. I have not seen any tool that implements this yet, but it could be an option for the impatient.

Why not implement all of them?

And here is my title-sake argument: When I move away from Claude Code, I miss boundary-aware queuing. When I move away from OpenAI Codex, I miss FIFO queueing.

I don’t see a reason why we could not implement all of them in all agentic tools. It could be controlled by a key combo like Ctrl+Enter, a submenu, or a button, depending on whether you are in the terminal or not.

Having the option would definitely make a difference in agentic workflows where you are running 3-4 agents in parallel.

So if you are reading this and are implementing an agentic coding tool, I would be happy if you took all this into consideration!

This is an adaptation of the original Google’s Code Review Guidelines, to use GitHub specific terminology. Google has their own internal tools for version control (Piper) and code review (Critique). They have their own terminology, like “Change List” (CL) instead of “Pull Request” (PR) which most developers are more familiar with. The changes are minimal and the content is kept as close to the original as possible. The hope is to make this gem accessible to a wider audience.

I also combined the whole set of documents into a single file, to make it easier to consume. You can find my fork here. If you notice any mistakes, please feel free to submit a PR to the fork.

Introduction

A code review is a process where someone other than the author(s) of a piece of code examines that code.

At Google, we use code review to maintain the quality of our code and products.

This documentation is the canonical description of Google’s code review processes and policies.

This page is an overview of our code review process. There are two other large documents that are a part of this guide:

• How To Do A Code Review: A detailed guide for code reviewers.
• The PR Author’s Guide: A detailed guide for developers whose PRs are going through review.

What Do Code Reviewers Look For?

Code reviews should look at:

• Design: Is the code well-designed and appropriate for your system?
• Functionality: Does the code behave as the author likely intended? Is the way the code behaves good for its users?
• Complexity: Could the code be made simpler? Would another developer be able to easily understand and use this code when they come across it in the future?
• Tests: Does the code have correct and well-designed automated tests?
• Naming: Did the developer choose clear names for variables, classes, methods, etc.?
• Comments: Are the comments clear and useful?
• Style: Does the code follow our style guides?
• Documentation: Did the developer also update relevant documentation?

See How To Do A Code Review for more information.

Picking the Best Reviewers

In general, you want to find the best reviewers you can who are capable of responding to your review within a reasonable period of time.

The best reviewer is the person who will be able to give you the most thorough and correct review for the piece of code you are writing. This usually means the owner(s) of the code, who may or may not be the people in the CODEOWNERS file. Sometimes this means asking different people to review different parts of the PR.

If you find an ideal reviewer but they are not available, you should at least CC them on your change.

In-Person Reviews (and Pair Programming)

If you pair-programmed a piece of code with somebody who was qualified to do a good code review on it, then that code is considered reviewed.

You can also do in-person code reviews where the reviewer asks questions and the developer of the change speaks only when spoken to.

See Also

• How To Do A Code Review: A detailed guide for code reviewers.
• The PR Author’s Guide: A detailed guide for developers whose PRs are going through review.

How to do a code review

The pages in this section contain recommendations on the best way to do code reviews, based on long experience. All together they represent one complete document, broken up into many separate sections. You don’t have to read them all, but many people have found it very helpful to themselves and their team to read the entire set.

• The Standard of Code Review
• What to Look For In a Code Review
• Navigating a PR in Review
• Speed of Code Reviews
• How to Write Code Review Comments
• Handling Pushback in Code Reviews

See also the PR Author’s Guide, which gives detailed guidance to developers whose PRs are undergoing review.

The Standard of Code Review

The primary purpose of code review is to make sure that the overall code health of Google’s code base is improving over time. All of the tools and processes of code review are designed to this end.

In order to accomplish this, a series of trade-offs have to be balanced.

First, developers must be able to make progress on their tasks. If you never merge an improvement into the codebase, then the codebase never improves. Also, if a reviewer makes it very difficult for any change to go in, then developers are disincentivized to make improvements in the future.

On the other hand, it is the duty of the reviewer to make sure that each PR is of such a quality that the overall code health of their codebase is not decreasing as time goes on. This can be tricky, because often, codebases degrade through small decreases in code health over time, especially when a team is under significant time constraints and they feel that they have to take shortcuts in order to accomplish their goals.

Also, a reviewer has ownership and responsibility over the code they are reviewing. They want to ensure that the codebase stays consistent, maintainable, and all of the other things mentioned in “What to look for in a code review.”

Thus, we get the following rule as the standard we expect in code reviews:

In general, reviewers should favor approving a PR once it is in a state where it definitely improves the overall code health of the system being worked on, even if the PR isn’t perfect.

That is the senior principle among all of the code review guidelines.

There are limitations to this, of course. For example, if a PR adds a feature that the reviewer doesn’t want in their system, then the reviewer can certainly deny approval even if the code is well-designed.

A key point here is that there is no such thing as “perfect” code—there is only better code. Reviewers should not require the author to polish every tiny piece of a PR before granting approval. Rather, the reviewer should balance out the need to make forward progress compared to the importance of the changes they are suggesting. Instead of seeking perfection, what a reviewer should seek is continuous improvement. A PR that, as a whole, improves the maintainability, readability, and understandability of the system shouldn’t be delayed for days or weeks because it isn’t “perfect.”

Reviewers should always feel free to leave comments expressing that something could be better, but if it’s not very important, prefix it with something like “Nit: “ to let the author know that it’s just a point of polish that they could choose to ignore.

Note: Nothing in this document justifies merging PRs that definitely worsen the overall code health of the system. The only time you would do that would be in an emergency.

Mentoring

Code review can have an important function of teaching developers something new about a language, a framework, or general software design principles. It’s always fine to leave comments that help a developer learn something new. Sharing knowledge is part of improving the code health of a system over time. Just keep in mind that if your comment is purely educational, but not critical to meeting the standards described in this document, prefix it with “Nit: “ or otherwise indicate that it’s not mandatory for the author to resolve it in this PR.

Principles

• Technical facts and data overrule opinions and personal preferences.

• On matters of style, the style guide is the absolute authority. Any purely style point (whitespace, etc.) that is not in the style guide is a matter of personal preference. The style should be consistent with what is there. If there is no previous style, accept the author’s.

• Aspects of software design are almost never a pure style issue or just a personal preference. They are based on underlying principles and should be weighed on those principles, not simply by personal opinion. Sometimes there are a few valid options. If the author can demonstrate (either through data or based on solid engineering principles) that several approaches are equally valid, then the reviewer should accept the preference of the author. Otherwise the choice is dictated by standard principles of software design.

• If no other rule applies, then the reviewer may ask the author to be consistent with what is in the current codebase, as long as that doesn’t worsen the overall code health of the system.

Resolving Conflicts

In any conflict on a code review, the first step should always be for the developer and reviewer to try to come to consensus, based on the contents of this document and the other documents in The PR Author’s Guide and this Reviewer Guide.

When coming to consensus becomes especially difficult, it can help to have a face-to-face meeting or a video conference between the reviewer and the author, instead of just trying to resolve the conflict through code review comments. (If you do this, though, make sure to record the results of the discussion as a comment on the PR, for future readers.)

If that doesn’t resolve the situation, the most common way to resolve it would be to escalate. Often the escalation path is to a broader team discussion, having a Technical Lead weigh in, asking for a decision from a maintainer of the code, or asking an Eng Manager to help out. Don’t let a PR sit around because the author and the reviewer can’t come to an agreement.

Next: What to look for in a code review

What to look for in a code review

Note: Always make sure to take into account The Standard of Code Review when considering each of these points.

Design

The most important thing to cover in a review is the overall design of the PR. Do the interactions of various pieces of code in the PR make sense? Does this change belong in your codebase, or in a library? Does it integrate well with the rest of your system? Is now a good time to add this functionality?

Functionality

Does this PR do what the developer intended? Is what the developer intended good for the users of this code? The “users” are usually both end-users (when they are affected by the change) and developers (who will have to “use” this code in the future).

Mostly, we expect developers to test PRs well-enough that they work correctly by the time they get to code review. However, as the reviewer you should still be thinking about edge cases, looking for concurrency problems, trying to think like a user, and making sure that there are no bugs that you see just by reading the code.

You can validate the PR if you want—the time when it’s most important for a reviewer to check a PR’s behavior is when it has a user-facing impact, such as a UI change. It’s hard to understand how some changes will impact a user when you’re just reading the code. For changes like that, you can have the developer give you a demo of the functionality if it’s too inconvenient to patch in the PR and try it yourself.

Another time when it’s particularly important to think about functionality during a code review is if there is some sort of parallel programming going on in the PR that could theoretically cause deadlocks or race conditions. These sorts of issues are very hard to detect by just running the code and usually need somebody (both the developer and the reviewer) to think through them carefully to be sure that problems aren’t being introduced. (Note that this is also a good reason not to use concurrency models where race conditions or deadlocks are possible—it can make it very complex to do code reviews or understand the code.)

Complexity

Is the PR more complex than it should be? Check this at every level of the PR—are individual lines too complex? Are functions too complex? Are classes too complex? “Too complex” usually means “can’t be understood quickly by code readers.” It can also mean “developers are likely to introduce bugs when they try to call or modify this code.”

A particular type of complexity is over-engineering, where developers have made the code more generic than it needs to be, or added functionality that isn’t presently needed by the system. Reviewers should be especially vigilant about over-engineering. Encourage developers to solve the problem they know needs to be solved now, not the problem that the developer speculates might need to be solved in the future. The future problem should be solved once it arrives and you can see its actual shape and requirements in the physical universe.

Tests

Ask for unit, integration, or end-to-end tests as appropriate for the change. In general, tests should be added in the same PR as the production code unless the PR is handling an emergency.

Make sure that the tests in the PR are correct, sensible, and useful. Tests do not test themselves, and we rarely write tests for our tests—a human must ensure that tests are valid.

Will the tests actually fail when the code is broken? If the code changes beneath them, will they start producing false positives? Does each test make simple and useful assertions? Are the tests separated appropriately between different test methods?

Remember that tests are also code that has to be maintained. Don’t accept complexity in tests just because they aren’t part of the main binary.

Naming

Did the developer pick good names for everything? A good name is long enough to fully communicate what the item is or does, without being so long that it becomes hard to read.

Comments

Did the developer write clear comments in understandable English? Are all of the comments actually necessary? Usually comments are useful when they explain why some code exists, and should not be explaining what some code is doing. If the code isn’t clear enough to explain itself, then the code should be made simpler. There are some exceptions (regular expressions and complex algorithms often benefit greatly from comments that explain what they’re doing, for example) but mostly comments are for information that the code itself can’t possibly contain, like the reasoning behind a decision.

It can also be helpful to look at comments that were there before this PR. Maybe there is a TODO that can be removed now, a comment advising against this change being made, etc.

Note that comments are different from documentation of classes, modules, or functions, which should instead express the purpose of a piece of code, how it should be used, and how it behaves when used.

Style

We have style guides at Google for all of our major languages, and even for most of the minor languages. Make sure the PR follows the appropriate style guides.

If you want to improve some style point that isn’t in the style guide, prefix your comment with “Nit:” to let the developer know that it’s a nitpick that you think would improve the code but isn’t mandatory. Don’t block PRs from being merged based only on personal style preferences.

The author of the PR should not include major style changes combined with other changes. It makes it hard to see what is being changed in the PR, makes merges and rollbacks more complex, and causes other problems. For example, if the author wants to reformat the whole file, have them send you just the reformatting as one PR, and then send another PR with their functional changes after that.

Consistency

What if the existing code is inconsistent with the style guide? Per our code review principles, the style guide is the absolute authority: if something is required by the style guide, the PR should follow the guidelines.

In some cases, the style guide makes recommendations rather than declaring requirements. In these cases, it’s a judgment call whether the new code should be consistent with the recommendations or the surrounding code. Bias towards following the style guide unless the local inconsistency would be too confusing.

If no other rule applies, the author should maintain consistency with the existing code.

Either way, encourage the author to file a bug and add a TODO for cleaning up existing code.

Documentation

If a PR changes how users build, test, interact with, or release code, check to see that it also updates associated documentation, including READMEs, repository docs, and any generated reference docs. If the PR deletes or deprecates code, consider whether the documentation should also be deleted. If documentation is missing, ask for it.

Every Line

In the general case, look at every line of code that you have been assigned to review. Some things like data files, generated code, or large data structures you can scan over sometimes, but don’t scan over a human-written class, function, or block of code and assume that what’s inside of it is okay. Obviously some code deserves more careful scrutiny than other code—that’s a judgment call that you have to make—but you should at least be sure that you understand what all the code is doing.

If it’s too hard for you to read the code and this is slowing down the review, then you should let the developer know that and wait for them to clarify it before you try to review it. At Google, we hire great software engineers, and you are one of them. If you can’t understand the code, it’s very likely that other developers won’t either. So you’re also helping future developers understand this code, when you ask the developer to clarify it.

If you understand the code but you don’t feel qualified to do some part of the review, make sure there is a reviewer on the PR who is qualified, particularly for complex issues such as privacy, security, concurrency, accessibility, internationalization, etc.

Exceptions

What if it doesn’t make sense for you to review every line? For example, you are one of multiple reviewers on a PR and may be asked:

• To review only certain files that are part of a larger change.
• To review only certain aspects of the PR, such as the high-level design, privacy or security implications, etc.

In these cases, note in a comment which parts you reviewed. Prefer giving Approve with comments .

If you instead wish to grant Approval after confirming that other reviewers have reviewed other parts of the PR, note this explicitly in a comment to set expectations. Aim to respond quickly once the PR has reached the desired state.

Context

It is often helpful to look at the PR in a broad context. Usually the code review tool will only show you a few lines of code around the parts that are being changed. Sometimes you have to look at the whole file to be sure that the change actually makes sense. For example, you might see only four new lines being added, but when you look at the whole file, you see those four lines are in a 50-line method that now really needs to be broken up into smaller methods.

It’s also useful to think about the PR in the context of the system as a whole. Is this PR improving the code health of the system or is it making the whole system more complex, less tested, etc.? Don’t accept PRs that degrade the code health of the system. Most systems become complex through many small changes that add up, so it’s important to prevent even small complexities in new changes.

Good Things

If you see something nice in the PR, tell the developer, especially when they addressed one of your comments in a great way. Code reviews often just focus on mistakes, but they should offer encouragement and appreciation for good practices, as well. It’s sometimes even more valuable, in terms of mentoring, to tell a developer what they did right than to tell them what they did wrong.

Summary

In doing a code review, you should make sure that:

• The code is well-designed.
• The functionality is good for the users of the code.
• Any UI changes are sensible and look good.
• Any parallel programming is done safely.
• The code isn’t more complex than it needs to be.
• The developer isn’t implementing things they might need in the future but don’t know they need now.
• Code has appropriate unit tests.
• Tests are well-designed.
• The developer used clear names for everything.
• Comments are clear and useful, and mostly explain why instead of what.
• Code is appropriately documented (generally in repository docs).
• The code conforms to our style guides.

Make sure to review every line of code you’ve been asked to review, look at the context, make sure you’re improving code health, and compliment developers on good things that they do.

Next: Navigating a PR in Review

Navigating a PR in review

Summary

Now that you know what to look for, what’s the most efficient way to manage a review that’s spread across multiple files?

1. Does the change make sense? Does it have a good description?
2. Look at the most important part of the change first. Is it well-designed overall?
3. Look at the rest of the PR in an appropriate sequence.

Step One: Take a broad view of the change

Look at the PR description and what the PR does in general. Does this change even make sense? If this change shouldn’t have happened in the first place, please respond immediately with an explanation of why the change should not be happening. When you reject a change like this, it’s also a good idea to suggest to the developer what they should have done instead.

For example, you might say “Looks like you put some good work into this, thanks! However, we’re actually going in the direction of removing the FooWidget system that you’re modifying here, and so we don’t want to make any new modifications to it right now. How about instead you refactor our new BarWidget class?”

Note that not only did the reviewer reject the current PR and provide an alternative suggestion, but they did it courteously. This kind of courtesy is important because we want to show that we respect each other as developers even when we disagree.

If you get more than a few PRs that represent changes you don’t want to make, you should consider re-working your team’s development process or the posted process for external contributors so that there is more communication before PRs are written. It’s better to tell people “no” before they’ve done a ton of work that now has to be thrown away or drastically re-written.

Step Two: Examine the main parts of the PR

Find the file or files that are the “main” part of this PR. Often, there is one file that has the largest number of logical changes, and it’s the major piece of the PR. Look at these major parts first. This helps give context to all of the smaller parts of the PR, and generally accelerates doing the code review. If the PR is too large for you to figure out which parts are the major parts, ask the developer what you should look at first, or ask them to split up the PR into multiple PRs.

If you see some major design problems with this part of the PR, you should send those comments immediately, even if you don’t have time to review the rest of the PR right now. In fact, reviewing the rest of the PR might be a waste of time, because if the design problems are significant enough, a lot of the other code under review is going to disappear and not matter anyway.

There are two major reasons it’s so important to send these major design comments out immediately:

• Developers often mail a PR and then immediately start new work based on that PR while they wait for review. If there are major design problems in the PR you’re reviewing, they’re also going to have to re-work their later PR. You want to catch them before they’ve done too much extra work on top of the problematic design.
• Major design changes take longer to do than small changes. Developers nearly all have deadlines; in order to make those deadlines and still have quality code in the codebase, the developer needs to start on any major re-work of the PR as soon as possible.

Step Three: Look through the rest of the PR in an appropriate sequence

Once you’ve confirmed there are no major design problems with the PR as a whole, try to figure out a logical sequence to look through the files while also making sure you don’t miss reviewing any file. Usually after you’ve looked through the major files, it’s simplest to just go through each file in the order that the code review tool presents them to you. Sometimes it’s also helpful to read the tests first before you read the main code, because then you have an idea of what the change is supposed to be doing.

Next: Speed of Code Reviews

Speed of Code Reviews

Why Should Code Reviews Be Fast?

At Google, we optimize for the speed at which a team of developers can produce a product together, as opposed to optimizing for the speed at which an individual developer can write code. The speed of individual development is important, it’s just not as important as the velocity of the entire team.

When code reviews are slow, several things happen:

• The velocity of the team as a whole is decreased. Yes, the individual who doesn’t respond quickly to the review gets other work done. However, new features and bug fixes for the rest of the team are delayed by days, weeks, or months as each PR waits for review and re-review.
• Developers start to protest the code review process. If a reviewer only responds every few days, but requests major changes to the PR each time, that can be frustrating and difficult for developers. Often, this is expressed as complaints about how “strict” the reviewer is being. If the reviewer requests the same substantial changes (changes which really do improve code health), but responds quickly every time the developer makes an update, the complaints tend to disappear. Most complaints about the code review process are actually resolved by making the process faster.
• Code health can be impacted. When reviews are slow, there is increased pressure to allow developers to merge PRs that are not as good as they could be. Slow reviews also discourage code cleanups, refactorings, and further improvements to existing PRs.

How Fast Should Code Reviews Be?

If you are not in the middle of a focused task, you should do a code review shortly after it comes in.

One business day is the maximum time it should take to respond to a code review request (i.e., first thing the next morning).

Following these guidelines means that a typical PR should get multiple rounds of review (if needed) within a single day.

Speed vs. Interruption

There is one time where the consideration of personal velocity trumps team velocity. If you are in the middle of a focused task, such as writing code, don’t interrupt yourself to do a code review. Research has shown that it can take a long time for a developer to get back into a smooth flow of development after being interrupted. So interrupting yourself while coding is actually more expensive to the team than making another developer wait a bit for a code review.

Instead, wait for a break point in your work before you respond to a request for review. This could be when your current coding task is completed, after lunch, returning from a meeting, coming back from the breakroom, etc.

Fast Responses

When we talk about the speed of code reviews, it is the response time that we are concerned with, as opposed to how long it takes a PR to get through the whole review and be merged. The whole process should also be fast, ideally, but it’s even more important for the individual responses to come quickly than it is for the whole process to happen rapidly.

Even if it sometimes takes a long time to get through the entire review process, having quick responses from the reviewer throughout the process significantly eases the frustration developers can feel with “slow” code reviews.

If you are too busy to do a full review on a PR when it comes in, you can still send a quick response that lets the developer know when you will get to it, suggest other reviewers who might be able to respond more quickly, or provide some initial broad comments. (Note: none of this means you should interrupt coding even to send a response like this—send the response at a reasonable break point in your work.)

It is important that reviewers spend enough time on review that they are certain their “Approve” means “this code meets our standards.” However, individual responses should still ideally be fast.

Cross-Time-Zone Reviews

When dealing with time zone differences, try to get back to the author while they have time to respond before the end of their working hours. If they have already finished work for the day, then try to make sure your review is done before they start work the next day.

Approve With Comments (LGTM)

In order to speed up code reviews, there are certain situations in which a reviewer should Approve even though they are also leaving unresolved comments on the PR. This should be done when at least one of the following applies:

• The reviewer is confident that the developer will appropriately address all the reviewer’s remaining comments.
• The comments don’t have to be addressed by the developer.
• The suggestions are minor, e.g. sort imports, fix a nearby typo, apply a suggested fix, remove an unused dep, etc.

The reviewer should specify which of these options they intend, if it is not otherwise clear.

Approve With Comments is especially worth considering when the developer and reviewer are in different time zones and otherwise the developer would be waiting for a whole day just to get approval.

Large PRs

If somebody sends you a code review that is so large you’re not sure when you will be able to have time to review it, your typical response should be to ask the developer to split the PR into several smaller PRs that build on each other, instead of one huge PR that has to be reviewed all at once. This is usually possible and very helpful to reviewers, even if it takes additional work from the developer.

If a PR can’t be broken up into smaller PRs, and you don’t have time to review the entire thing quickly, then at least write some comments on the overall design of the PR and send it back to the developer for improvement. One of your goals as a reviewer should be to always unblock the developer or enable them to take some sort of further action quickly, without sacrificing code health to do so.

Code Review Improvements Over Time

If you follow these guidelines and you are strict with your code reviews, you should find that the entire code review process tends to go faster and faster over time. Developers learn what is required for healthy code, and send you PRs that are great from the start, requiring less and less review time. Reviewers learn to respond quickly and not add unnecessary latency into the review process. But don’t compromise on the code review standards or quality for an imagined improvement in velocity—it’s not actually going to make anything happen more quickly, in the long run.

Emergencies

There are also emergencies where PRs must pass through the whole review process very quickly, and where the quality guidelines would be relaxed. However, please see What Is An Emergency? for a description of which situations actually qualify as emergencies and which don’t.

Next: How to Write Code Review Comments

How to write code review comments

Summary

• Be kind.
• Explain your reasoning.
• Balance giving explicit directions with just pointing out problems and letting the developer decide.
• Encourage developers to simplify code or add code comments instead of just explaining the complexity to you.

Courtesy

In general, it is important to be courteous and respectful while also being very clear and helpful to the developer whose code you are reviewing. One way to do this is to be sure that you are always making comments about the code and never making comments about the developer. You don’t always have to follow this practice, but you should definitely use it when saying something that might otherwise be upsetting or contentious. For example:

Bad: “Why did you use threads here when there’s obviously no benefit to be gained from concurrency?”

Good: “The concurrency model here is adding complexity to the system without any actual performance benefit that I can see. Because there’s no performance benefit, it’s best for this code to be single-threaded instead of using multiple threads.”

Explain Why

One thing you’ll notice about the “good” example from above is that it helps the developer understand why you are making your comment. You don’t always need to include this information in your review comments, but sometimes it’s appropriate to give a bit more explanation around your intent, the best practice you’re following, or how your suggestion improves code health.

Giving Guidance

In general it is the developer’s responsibility to fix a PR, not the reviewer’s. You are not required to do detailed design of a solution or write code for the developer.

This doesn’t mean the reviewer should be unhelpful, though. In general you should strike an appropriate balance between pointing out problems and providing direct guidance. Pointing out problems and letting the developer make a decision often helps the developer learn, and makes it easier to do code reviews. It also can result in a better solution, because the developer is closer to the code than the reviewer is.

However, sometimes direct instructions, suggestions, or even code are more helpful. The primary goal of code review is to get the best PR possible. A secondary goal is improving the skills of developers so that they require less and less review over time.

Remember that people learn from reinforcement of what they are doing well and not just what they could do better. If you see things you like in the PR, comment on those too! Examples: developer cleaned up a messy algorithm, added exemplary test coverage, or you as the reviewer learned something from the PR. Just as with all comments, include why you liked something, further encouraging the developer to continue good practices.

Label comment severity

Consider labeling the severity of your comments, differentiating required changes from guidelines or suggestions.

Here are some examples:

Nit: This is a minor thing. Technically you should do it, but it won’t hugely impact things.

Optional (or Consider): I think this may be a good idea, but it’s not strictly required.

FYI: I don’t expect you to do this in this PR, but you may find this interesting to think about for the future.

This makes review intent explicit and helps authors prioritize the importance of various comments. It also helps avoid misunderstandings; for example, without comment labels, authors may interpret all comments as mandatory, even if some comments are merely intended to be informational or optional.

Accepting Explanations

If you ask a developer to explain a piece of code that you don’t understand, that should usually result in them rewriting the code more clearly. Occasionally, adding a comment in the code is also an appropriate response, as long as it’s not just explaining overly complex code.

Explanations written only in the code review tool are not helpful to future code readers. They are acceptable only in a few circumstances, such as when you are reviewing an area you are not very familiar with and the developer explains something that normal readers of the code would have already known.

Next: Handling Pushback in Code Reviews

Handling pushback in code reviews

Sometimes a developer will push back on a code review. Either they will disagree with your suggestion or they will complain that you are being too strict in general.

Who is right?

When a developer disagrees with your suggestion, first take a moment to consider if they are correct. Often, they are closer to the code than you are, and so they might really have a better insight about certain aspects of it. Does their argument make sense? Does it make sense from a code health perspective? If so, let them know that they are right and let the issue drop.

However, developers are not always right. In this case the reviewer should further explain why they believe that their suggestion is correct. A good explanation demonstrates both an understanding of the developer’s reply, and additional information about why the change is being requested.

In particular, when the reviewer believes their suggestion will improve code health, they should continue to advocate for the change, if they believe the resulting code quality improvement justifies the additional work requested. Improving code health tends to happen in small steps.

Sometimes it takes a few rounds of explaining a suggestion before it really sinks in. Just make sure to always stay polite and let the developer know that you hear what they’re saying, you just don’t agree.

Upsetting Developers

Reviewers sometimes believe that the developer will be upset if the reviewer insists on an improvement. Sometimes developers do become upset, but it is usually brief and they become very thankful later that you helped them improve the quality of their code. Usually, if you are polite in your comments, developers actually don’t become upset at all, and the worry is just in the reviewer’s mind. Upsets are usually more about the way comments are written than about the reviewer’s insistence on code quality.

Cleaning It Up Later

A common source of push back is that developers (understandably) want to get things done. They don’t want to go through another round of review just to get this PR in. So they say they will clean something up in a later PR, and thus you should Approve this PR now. Some developers are very good about this, and will immediately write a follow-up PR that fixes the issue. However, experience shows that as more time passes after a developer writes the original PR, the less likely this clean up is to happen. In fact, usually unless the developer does the clean up immediately after the present PR, it never happens. This isn’t because developers are irresponsible, but because they have a lot of work to do and the cleanup gets lost or forgotten in the press of other work. Thus, it is usually best to insist that the developer clean up their PR now, before the code is in the codebase and “done.” Letting people “clean things up later” is a common way for codebases to degenerate.

If a PR introduces new complexity, it must be cleaned up before merge unless it is an emergency. If the PR exposes surrounding problems and they can’t be addressed right now, the developer should file a bug for the cleanup and assign it to themselves so that it doesn’t get lost. They can optionally also write a TODO comment in the code that references the filed bug.

General Complaints About Strictness

If you previously had fairly lax code reviews and you switch to having strict reviews, some developers will complain very loudly. Improving the speed of your code reviews usually causes these complaints to fade away.

Sometimes it can take months for these complaints to fade away, but eventually developers tend to see the value of strict code reviews as they see what great code they help generate. Sometimes the loudest protesters even become your strongest supporters once something happens that causes them to really see the value you’re adding by being strict.

Resolving Conflicts

If you are following all of the above but you still encounter a conflict between yourself and a developer that can’t be resolved, see The Standard of Code Review for guidelines and principles that can help resolve the conflict.

The PR author’s guide to getting through code review

The pages in this section contain best practices for developers going through code review. These guidelines should help you get through reviews faster and with higher-quality results. You don’t have to read them all, but they are intended to apply to every Google developer, and many people have found it helpful to read the whole set.

• Writing Good PR Descriptions
• Small PRs
• How to Handle Reviewer Comments

See also How to Do a Code Review, which gives detailed guidance for code reviewers.

Writing good PR descriptions

A PR description is a public record of change, and it is important that it communicates:

1. What change is being made? This should summarize the major changes such that readers have a sense of what is being changed without needing to read the entire PR.

2. Why are these changes being made? What contexts did you have as an author when making this change? Were there decisions you made that aren’t reflected in the source code? etc.

The PR description will become a permanent part of our version control history and will possibly be read by hundreds of people over the years.

Future developers will search for your PR based on its description. Someone in the future might be looking for your change because of a faint memory of its relevance but without the specifics handy. If all the important information is in the code and not the description, it’s going to be a lot harder for them to locate your PR.

And then, after they find the PR, will they be able to understand why the change was made? Reading source code may reveal what the software is doing but it may not reveal why it exists, which can make it harder for future developers to know whether they can move Chesterton’s fence.

A well-written PR description will help those future engineers – sometimes, including yourself!

First Line

• Short summary of what is being done.
• Complete sentence, written as though it was an order.
• Follow by empty line.

The first line of a PR description should be a short summary of specifically what is being done by the PR, followed by a blank line. This is what appears in version control history summaries, so it should be informative enough that future code searchers don’t have to read your PR or its whole description to understand what your PR actually did or how it differs from other PRs. That is, the first line should stand alone, allowing readers to skim through code history much faster.

Try to keep your first line short, focused, and to the point. The clarity and utility to the reader should be the top concern.

By tradition, the first line of a PR description is a complete sentence, written as though it were an order (an imperative sentence). For example, say "Delete the FizzBuzz RPC and replace it with the new system.” instead of "Deleting the FizzBuzz RPC and replacing it with the new system.” You don’t have to write the rest of the description as an imperative sentence, though.

Body is Informative

The first line should be a short, focused summary, while the rest of the description should fill in the details and include any supplemental information a reader needs to understand the change holistically. It might include a brief description of the problem that’s being solved, and why this is the best approach. If there are any shortcomings to the approach, they should be mentioned. If relevant, include background information such as bug numbers, benchmark results, and links to design documents.

If you include links to external resources consider that they may not be visible to future readers due to access restrictions or retention policies. Where possible include enough context for reviewers and future readers to understand the PR.

Even small PRs deserve a little attention to detail. Put the PR in context.

Bad PR Descriptions

“Fix bug” is an inadequate PR description. What bug? What did you do to fix it? Other similarly bad descriptions include:

• “Fix build.”
• “Add patch.”
• “Moving code from A to B.”
• “Phase 1.”
• “Add convenience functions.”
• “kill weird URLs.”

Some of those are real PR descriptions. Although short, they do not provide enough useful information.

Good PR Descriptions

Here are some examples of good descriptions.

Functionality change

Example:

RPC: Remove size limit on RPC server message freelist.

Servers like FizzBuzz have very large messages and would benefit from reuse. Make the freelist larger, and add a goroutine that frees the freelist entries slowly over time, so that idle servers eventually release all freelist entries.

The first few words describe what the PR actually does. The rest of the description talks about the problem being solved, why this is a good solution, and a bit more information about the specific implementation.

Refactoring

Example:

Construct a Task with a TimeKeeper to use its TimeStr and Now methods.

Add a Now method to Task, so the borglet() getter method can be removed (which was only used by OOMCandidate to call borglet’s Now method). This replaces the methods on Borglet that delegate to a TimeKeeper.

Allowing Tasks to supply Now is a step toward eliminating the dependency on Borglet. Eventually, collaborators that depend on getting Now from the Task should be changed to use a TimeKeeper directly, but this has been an accommodation to refactoring in small steps.

Continuing the long-range goal of refactoring the Borglet Hierarchy.

The first line describes what the PR does and how this is a change from the past. The rest of the description talks about the specific implementation, the context of the PR, that the solution isn’t ideal, and possible future direction. It also explains why this change is being made.

Small PR that needs some context

Example:

Create a Python3 build rule for status.py.

This allows consumers who are already using this as in Python3 to depend on a rule that is next to the original status build rule instead of somewhere in their own tree. It encourages new consumers to use Python3 if they can, instead of Python2, and significantly simplifies some automated build file refactoring tools being worked on currently.

The first sentence describes what’s actually being done. The rest of the description explains why the change is being made and gives the reviewer a lot of context.

Using tags

Tags are manually entered labels that can be used to categorize PRs. These may be supported by tools or just used by team convention.

For example:

• “[tag]”
• “[a longer tag]”
• “#tag”
• “tag:”

Using tags is optional.

When adding tags, consider whether they should be in the body of the PR description or the first line. Limit the usage of tags in the first line, as this can obscure the content.

Examples with and without tags:

Good:

// Tags are okay in the first line if kept short.
[banana] Peel the banana before eating.

// Tags can be inlined in content.
Peel the #banana before eating.

// Tags are optional.
Peel the banana before eating.

// Multiple tags are acceptable if kept short.
#banana #apple: Assemble a fruit basket.

// Tags can go anywhere in the PR description.
> Assemble a fruit basket.
>
> #banana #apple

Bad:

// Too many tags (or tags that are too long) overwhelm the first line.
//
// Instead, consider whether the tags can be moved into the description body
// and/or shortened.
[banana peeler factory factory][apple picking service] Assemble a fruit basket.

Generated PR descriptions

Some PRs are generated by tools. Whenever possible, their descriptions should also follow the advice here. That is, their first line should be short, focused, and stand alone, and the PR description body should include informative details that help reviewers and future code searchers understand each PR’s effect.

Review the description before merging the PR

PRs can undergo significant change during review. It can be worthwhile to review a PR description before merging the PR, to ensure that the description still reflects what the PR does.

Next: Small PRs

Small PRs

Why Write Small PRs?

Small, simple PRs are:

• Reviewed more quickly. It’s easier for a reviewer to find five minutes several times to review small PRs than to set aside a 30 minute block to review one large PR.
• Reviewed more thoroughly. With large changes, reviewers and authors tend to get frustrated by large volumes of detailed commentary shifting back and forth—sometimes to the point where important points get missed or dropped.
• Less likely to introduce bugs. Since you’re making fewer changes, it’s easier for you and your reviewer to reason effectively about the impact of the PR and see if a bug has been introduced.
• Less wasted work if they are rejected. If you write a huge PR and then your reviewer says that the overall direction is wrong, you’ve wasted a lot of work.
• Easier to merge. Working on a large PR takes a long time, so you will have lots of conflicts when you merge, and you will have to merge frequently.
• Easier to design well. It’s a lot easier to polish the design and code health of a small change than it is to refine all the details of a large change.
• Less blocking on reviews. Sending self-contained portions of your overall change allows you to continue coding while you wait for your current PR in review.
• Simpler to roll back. A large PR will more likely touch files that get updated between the initial PR submission and a rollback PR, complicating the rollback (the intermediate PRs will probably need to be rolled back too).

Note that reviewers have discretion to reject your change outright for the sole reason of it being too large. Usually they will thank you for your contribution but request that you somehow make it into a series of smaller changes. It can be a lot of work to split up a change after you’ve already written it, or require lots of time arguing about why the reviewer should accept your large change. It’s easier to just write small PRs in the first place.

What is Small?

In general, the right size for a PR is one self-contained change. This means that:

• The PR makes a minimal change that addresses just one thing. This is usually just one part of a feature, rather than a whole feature at once. In general it’s better to err on the side of writing PRs that are too small vs. PRs that are too large. Work with your reviewer to find out what an acceptable size is.
• The PR should include related test code.
• Everything the reviewer needs to understand about the PR (except future development) is in the PR, the PR’s description, the existing codebase, or a PR they’ve already reviewed.
• The system will continue to work well for its users and for the developers after the PR is merged.
• The PR is not so small that its implications are difficult to understand. If you add a new API, you should include a usage of the API in the same PR so that reviewers can better understand how the API will be used. This also prevents checking in unused APIs.

There are no hard and fast rules about how large is “too large.” 100 lines is usually a reasonable size for a PR, and 1000 lines is usually too large, but it’s up to the judgment of your reviewer. The number of files that a change is spread across also affects its “size.” A 200-line change in one file might be okay, but spread across 50 files it would usually be too large.

Keep in mind that although you have been intimately involved with your code from the moment you started to write it, the reviewer often has no context. What seems like an acceptably-sized PR to you might be overwhelming to your reviewer. When in doubt, write PRs that are smaller than you think you need to write. Reviewers rarely complain about getting PRs that are too small.

When are Large PRs Okay?

There are a few situations in which large changes aren’t as bad:

• You can usually count deletion of an entire file as being just one line of change, because it doesn’t take the reviewer very long to review.
• Sometimes a large PR has been generated by an automatic refactoring tool that you trust completely, and the reviewer’s job is just to verify and say that they really do want the change. These PRs can be larger, although some of the caveats from above (such as merging and testing) still apply.

Writing Small PRs Efficiently

If you write a small PR and then you wait for your reviewer to approve it before you write your next PR, then you’re going to waste a lot of time. So you want to find some way to work that won’t block you while you’re waiting for review. This could involve having multiple projects to work on simultaneously, finding reviewers who agree to be immediately available, doing in-person reviews, pair programming, or splitting your PRs in a way that allows you to continue working immediately.

Splitting PRs

When starting work that will have multiple PRs with potential dependencies among each other, it’s often useful to think about how to split and organize those PRs at a high level before diving into coding.

Besides making things easier for you as an author to manage and organize your PRs, it also makes things easier for your code reviewers, which in turn makes your code reviews more efficient.

Here are some strategies for splitting work into different PRs.

Stacking Multiple Changes on Top of Each Other

One way to split up a PR without blocking yourself is to write one small PR, send it off for review, and then immediately start writing another PR based on the first PR. Most version control systems allow you to do this somehow.

Splitting by Files

Another way to split up a PR is by groupings of files that will require different reviewers but are otherwise self-contained changes.

For example: you send off one PR for modifications to a protocol buffer and another PR for changes to the code that uses that proto. You have to merge the proto PR before the code PR, but they can both be reviewed simultaneously. If you do this, you might want to inform both sets of reviewers about the other PR that you wrote, so that they have context for your changes.

Another example: you send one PR for a code change and another for the configuration or experiment that uses that code; this is easier to roll back too, if necessary, as configuration/experiment files are sometimes pushed to production faster than code changes.

Splitting Horizontally

Consider creating shared code or stubs that help isolate changes between layers of the tech stack. This not only helps expedite development but also encourages abstraction between layers.

For example: You created a calculator app with client, API, service, and data model layers. A shared proto signature can abstract the service and data model layers from each other. Similarly, an API stub can split the implementation of client code from service code and enable them to move forward independently. Similar ideas can also be applied to more granular function or class level abstractions.

Splitting Vertically

Orthogonal to the layered, horizontal approach, you can instead break down your code into smaller, full-stack, vertical features. Each of these features can be independent parallel implementation tracks. This enables some tracks to move forward while other tracks are awaiting review or feedback.

Back to our calculator example from Splitting Horizontally. You now want to support new operators, like multiplication and division. You could split this up by implementing multiplication and division as separate verticals or sub-features, even though they may have some overlap such as shared button styling or shared validation logic.

Splitting Horizontally & Vertically

To take this a step further, you could combine these approaches and chart out an implementation plan like this, where each cell is its own standalone PR. Starting from the model (at the bottom) and working up to the client:

Separate Out Refactorings

It’s usually best to do refactorings in a separate PR from feature changes or bug fixes. For example, moving and renaming a class should be in a different PR from fixing a bug in that class. It is much easier for reviewers to understand the changes introduced by each PR when they are separate.

Small cleanups such as fixing a local variable name can be included inside of a feature change or bug fix PR, though. It’s up to the judgment of developers and reviewers to decide when a refactoring is so large that it will make the review more difficult if included in your current PR.

Keep related test code in the same PR

PRs should include related test code. Remember that smallness here refers the conceptual idea that the PR should be focused and is not a simplistic function on line count.

Tests are expected for all Google changes.

A PR that adds or changes logic should be accompanied by new or updated tests for the new behavior. Pure refactoring PRs (that aren’t intended to change behavior) should also be covered by tests; ideally, these tests already exist, but if they don’t, you should add them.

Independent test modifications can go into separate PRs first, similar to the refactorings guidelines. That includes:

• Validating pre-existing, merged code with new tests.
  • Ensures that important logic is covered by tests.
  • Increases confidence in subsequent refactorings on affected code. For example, if you want to refactor code that isn’t already covered by tests, merging test PRs before merging refactoring PRs can validate that the tested behavior is unchanged before and after the refactoring.
• Refactoring the test code (e.g. introduce helper functions).
• Introducing larger test framework code (e.g. an integration test).

Don’t Break the Build

If you have several PRs that depend on each other, you need to find a way to make sure the whole system keeps working after each PR is merged. Otherwise you might break the build for all your fellow developers for a few minutes between your PR merges (or even longer if something goes wrong unexpectedly with your later PR merges).

Can’t Make it Small Enough

Sometimes you will encounter situations where it seems like your PR has to be large. This is very rarely true. Authors who practice writing small PRs can almost always find a way to decompose functionality into a series of small changes.

Before writing a large PR, consider whether preceding it with a refactoring-only PR could pave the way for a cleaner implementation. Talk to your teammates and see if anybody has thoughts on how to implement the functionality in small PRs instead.

If all of these options fail (which should be extremely rare) then get consent from your reviewers in advance to review a large PR, so they are warned about what is coming. In this situation, expect to be going through the review process for a long time, be vigilant about not introducing bugs, and be extra diligent about writing tests.

Next: How to Handle Reviewer Comments

How to handle reviewer comments

When you’ve sent a PR out for review, it’s likely that your reviewer will respond with several comments on your PR. Here are some useful things to know about handling reviewer comments.

Don’t Take it Personally

The goal of review is to maintain the quality of our codebase and our products. When a reviewer provides a critique of your code, think of it as their attempt to help you, the codebase, and Google, rather than as a personal attack on you or your abilities.

Sometimes reviewers feel frustrated and they express that frustration in their comments. This isn’t a good practice for reviewers, but as a developer you should be prepared for this. Ask yourself, “What is the constructive thing that the reviewer is trying to communicate to me?” and then operate as though that’s what they actually said.

Never respond in anger to code review comments. That is a serious breach of professional etiquette that will live in the review history. If you are too angry or annoyed to respond kindly, then walk away from your computer for a while, or work on something else until you feel calm enough to reply politely.

In general, if a reviewer isn’t providing feedback in a way that’s constructive and polite, explain this to them in person. If you can’t talk to them in person or on a video call, then send them a private email. Explain to them in a kind way what you don’t like and what you’d like them to do differently. If they also respond in a non-constructive way to this private discussion, or it doesn’t have the intended effect, then escalate to your manager as appropriate.

Fix the Code

If a reviewer says that they don’t understand something in your code, your first response should be to clarify the code itself. If the code can’t be clarified, add a code comment that explains why the code is there. If a comment seems pointless, only then should your response be an explanation in the code review tool.

If a reviewer didn’t understand some piece of your code, it’s likely other future readers of the code won’t understand either. Writing a response in the review tool doesn’t help future code readers, but clarifying your code or adding code comments does help them.

Think Collaboratively

Writing a PR can take a lot of work. It’s often really satisfying to finally send one out for review, feel like it’s done, and be pretty sure that no further work is needed. It can be frustrating to receive comments asking for changes, especially if you don’t agree with them.

At times like this, take a moment to step back and consider if the reviewer is providing valuable feedback that will help the codebase and Google. Your first question to yourself should always be, “Do I understand what the reviewer is asking for?”

If you can’t answer that question, ask the reviewer for clarification.

And then, if you understand the comments but disagree with them, it’s important to think collaboratively, not combatively or defensively:

Bad: "No, I'm not going to do that."

Good: "I went with X because of [these pros/cons] with [these tradeoffs]
My understanding is that using Y would be worse because of [these reasons].
Are you suggesting that Y better serves the original tradeoffs, that we should
weigh the tradeoffs differently, or something else?"

Remember, courtesy and respect should always be a first priority. If you disagree with the reviewer, find ways to collaborate: ask for clarifications, discuss pros/cons, and provide explanations of why your method of doing things is better for the codebase, users, and/or Google.

Sometimes, you might know something about the users, codebase, or PR that the reviewer doesn’t know. Fix the code where appropriate, and engage your reviewer in discussion, including giving them more context. Usually you can come to some consensus between yourself and the reviewer based on technical facts.

Resolving Conflicts

Your first step in resolving conflicts should always be to try to come to consensus with your reviewer. If you can’t achieve consensus, see The Standard of Code Review, which gives principles to follow in such a situation.

Emergencies

Sometimes there are emergency PRs that must pass through the entire code review process as quickly as possible.

What Is An Emergency?

An emergency PR would be a small change that: allows a major launch to continue instead of rolling back, fixes a bug significantly affecting users in production, handles a pressing legal issue, closes a major security hole, etc.

In emergencies we really do care about the speed of the entire code review process, not just the speed of response. In this case only, the reviewer should care more about the speed of the review and the correctness of the code (does it actually resolve the emergency?) than anything else. Also (perhaps obviously) such reviews should take priority over all other code reviews, when they come up.

However, after the emergency is resolved you should look over the emergency PRs again and give them a more thorough review.

What Is NOT An Emergency?

To be clear, the following cases are not an emergency:

• Wanting to launch this week rather than next week (unless there is some actual hard deadline for launch such as a partner agreement).
• The developer has worked on a feature for a very long time and they really want to get the PR in.
• The reviewers are all in another timezone where it is currently nighttime or they are away on an off-site.
• It is the end of the day on a Friday and it would just be great to get this PR in before the developer leaves for the weekend.
• A manager says that this review has to be complete and the PR merged today because of a soft (not hard) deadline.
• Rolling back a PR that is causing test failures or build breakages.

And so on.

What Is a Hard Deadline?

A hard deadline is one where something disastrous would happen if you miss it. For example:

• Submitting your PR by a certain date is necessary for a contractual obligation.
• Your product will completely fail in the marketplace if not released by a certain date.
• Some hardware manufacturers only ship new hardware once a year. If you miss the deadline to submit code to them, that could be disastrous, depending on what type of code you’re trying to ship.

Delaying a release for a week is not disastrous. Missing an important conference might be disastrous, but often is not.

Most deadlines are soft deadlines, not hard deadlines. They represent a desire for a feature to be done by a certain time. They are important, but you shouldn’t be sacrificing code health to make them.

If you have a long release cycle (several weeks) it can be tempting to sacrifice code review quality to get a feature in before the next cycle. However, this pattern, if repeated, is a common way for projects to build up overwhelming technical debt. If developers are routinely merging PRs near the end of the cycle that “must get in” with only superficial review, then the team should modify its process so that large feature changes happen early in the cycle and have enough time for good review.

My >10 year old programming habits have changed since Claude Code launched. Python is less likely to be my go-to language for new projects anymore. I am managing projects in languages I am not fluent in—TypeScript, Rust and Go—and seem to be doing pretty well.

It seems that typed, compiled, etc. languages are better suited for vibecoding, because of the safety guarantees. This is unsurprising in hindsight, but it was counterintuitive because by default I “vibed” projects into existence in Python since forever.

Paradoxically, after a certain size of project, I can move faster and safer with e.g. Claude Code + Rust, compared to Claude Code + Python, despite the low-levelness of the code¹. This is possible purely because of AI tools.

For example, I refactored large chunks of our TypeScript frontend code at TextCortex. Claude Code runs tsc after finishing each task and ensures that the code compiles before committing. This let me move much faster compared to how I would have done it in Python, which does not provide compile-time guarantees. I am amazed every time how my 3-5k line diffs created in a few hours don’t end up breaking anything, and instead even increase stability.

LLMs are leaky abstractions, sure. But they now work well enough so that they solve the problem Python solved for me (fast prototyping), without the disadvantages of Python (lower safety guarantees, slowness, ambiguity²).

Because of this, I predict a decrease in Python adoption in companies, specifically for production deployments, even though I like it so much.

A certain characteristic of legacy desktop apps, like Microsoft Office, Autodesk AutoCAD, Adobe Photoshop and so on, are that they have crappy proprietary file formats. In 2025, we barely have reliable, fully-supported open source libraries to read and write .DOCX, .XLSX, .PPTX,¹ .DWG, .PSD and so on, even though related products keep making billions in revenue.

The reason is simple: Moat through obfuscation.

The business model for these products when they first appeared in the 1980s and 1990s was to sell the compiled binaries for a one-time fee. This was pre-internet, before Software-as-a-Service (SaaS) could provide a reliable revenue stream. Having a standardized file format would have meant giving competitors a chance to develop a superior product and take over the market. So they went the other way and made sure their file formats would only be read by their own products, for example by changing the specifications in each new version. To keep their businesses safe, they prevented interoperability of entire modalities of human work, and by doing so, they harmed the entire world’s economy for decades.²

Can you blame them? The only thing they could monetize was the editor. Office 365 and Adobe Creative Cloud has since implemented a SaaS model to capitalize even more, but the file formats are still crap—a vestige of the old business model.³

But finally, a revolution is underway. This might all change.

None of these products were designed to be used by developers. They were designed to be used by the “End User”. According to Microsoft, the End User does not care about elegance or consistency in design.⁴ The End User could never understand version control. The End User sends emails back and forth with extensions such as v1.final.docx, v1.final.final.docx. Until recently, the End User was the main customer of software.

However, we have a new customer in the market: AI. The average AI model is very different than Microsoft’s stereotypical End User. They can code. In fact, models have to code, or at least encode structured data like a function call JSON, in order to have agency. Yes, we will also have AIs using computers directly like OpenAI’s Operator, but it is generally more straightforward to use an API for an AI model than to use an emulated desktop.

We will soon witness AI models surpass the human End User in terms of economic production. Tyler Cowen⁵, Andrej Karpathy⁶ and others are convinced that we should plan for a future where AIs are major economic actors.

“The models, they just want to learn”. The models also want intuitive APIs and simple file formats. The models abhor unnecessary complexity. If you have developed a RAG pipeline for Excel files, you know what I mean.

If AI creates pressure to replace legacy file formats, then what can companies monetize if not the editor? The answer is the AI itself. Serve a proprietary model, serve an open source model, charge per tokens, charge for inference, charge for kilowatt-hours, charge for agent-hours/days. The business model will differ from industry to industry, but the trend is clear: value will be more and more linked to AI compute, and less and less to Software 1.0⁷.

There is now a huge opportunity in the market to create better software, that follow the File over App philosophy:

if you want to create digital artifacts that last, they must be files you can control, in formats that are easy to retrieve and read. Use tools that give you this freedom.

We already observe that AI systems work drastically more efficiently if they are granted such freedom. There is a reason why OpenAI based ChatGPT’s Code Interpreter on Python and not on Visual Basic, or why it chose to render equations using LaTeX instead of Office Math Markup Language (OMML)⁸. Open and widespread formats are more represented in the datasets, and the models can output them more correctly.

There is going to be an AI-native “Microsoft Office”, and it will not be created by Microsoft. Copilot is not it, and Microsoft knows it. Boiling tar won’t turn it into sugar. Same for other Adobe, Autodesk and other creators of clutter.

Internet Explorer’s 2009 YouTube moment is coming for legacy desktop apps, and it will be glorious.

Cultures can be categorized across many axes, and one of them is whether you can call an older male stranger uncle or female stranger auntie. For example, calling a shopkeeper uncle might be sympathetic in Singapore, whereas doing the same in Germany (Onkel) might get a negative reaction: “I’m not your uncle”.

This is similar to calling a stranger bro. In social science, this is called fictive kinship, social ties that are not based on blood relations. For readers which come from such cultures, this does not need an explanation. But for other readers, this might be a weird concept. Why would you call a stranger uncle or auntie?

Hover over the countries below to see which ones use uncle/auntie terms:

Note that fictive kinship can also have different levels:

1. Level 0: Blood relatives only. “Uncle”/”Auntie” is strictly for real uncles/aunts (by blood or marriage). No fictive use.

2. Level 1: Close non-relatives. Used for family friends, “uncle” or “auntie” is an honorary title but not for random people.

3. Level 2: Casual acquaintances. Used more widely for neighbors, family friends, or community members you vaguely know, but typically not for an absolute stranger.

4. Level 3: Total strangers. Used even for someone you’ve just met: a shopkeeper, taxi driver, or older passerby.

Many cultures fall somewhere between these levels and it’s not always black and white. Where possible, I’ve simplified it to the most typical usage.

Ommerism and social cohesion

The thought first occurred to me when I visited Singapore and heard people use uncle and auntie. Here were people speaking English, but it felt like they were speaking Turkish (my mother tongue).

The cultural difference is apparent to me as well since I started living in Germany. People here are more lonely, strangers distrust each other more, and there are no implicit social ties. I guess this holds for the entire Anglo/Germanic culture, including the US and the commonwealth.

Don’t get me wrong, people in Turkey distrust each other as well, probably even more. It is a more dangerous country than Germany. But those dangerous strangers are still uncles. It’s weird, I know.

As far as I could tell, the phenomenon is not even sociologically that much recognized or studied. There is no specific name for it, other than being a specific form of fictive kinship. Therefore, I will name it myself: ommerism. It derives from a recently popularized gender-neutral term for an uncle or auntie, ommer.

Lack of ommerism is an indicator for a weak collective culture. Such cultures are more individualistic, familial ties are weaker and people are overall more lonely. People from such cultures could for example tweet:

It is extra ironic that ex-colonies like Singapore (ex-British), Indonesia (ex-Dutch), Philippines (ex-Spanish) etc. took their colonizers’ words for uncle/auntie and started using it this way, whereas the original cultures still do not.

Related articles

• Three Shades of Ojisan: Defining a Social Identity in Japan

I had this idea while taking a shower and felt that I had to share it. It most likely has flaws, so I would appreciate any feedback at [email protected]. My hunch is that it could be a stepping stone towards something more fundamental.

As the world heads towards Artificial General Intelligence—AGI—people rush to define what it is. Marcus Hutter historically described it as

AI which is able to match or exceed human intelligence in a wide class of environments

(…)

hypothetical agent that can perform virtually all intellectual tasks as well as a typical human could

(see his most recently published book)

whereas OpenAI historically described it as

a highly autonomous system that outperforms humans at most economically valuable work

and more recently, according to a The Information report

an AI system that can generate at least $100 billion in profits for OpenAI

which apparently could be the threshold at which Microsoft loses access to OpenAI models, according to the legal agreement between OpenAI and Microsoft.

Acknowledging all of this and other possible definitions, I want to introduce a definition of AGI that relates to information theory and biology, which I think could make sense:

Here, “survival” can be thought of as mimetic survival, where an idea or invention keeps getting replicated or referenced instead of being deleted or forgotten. Some pieces of information, like blog posts auto-generated for SEO purposes, can quickly vanish, are ephemeral and so recently have started being called “AI slop”. Others, such as scientific theories, math proofs, books such as Euclid’s Elements, and so on, can persist across millennia because societies find them worth copying, citing, or archiving. They are Lindy.

In that way, it is possible to paraphrase the above definition as “an autonomous system that can generate novel and Lindy information at a rate higher than a human can do”.

Like Hutter’s definition, the concept of environment is crucial for this definition. Viruses thrive in biological systems because cells and organisms replicate them. Digital viruses exploit computers. Euclid’s Elements thrives in a math-loving environment. In every case, the information’s persistence depends not just on its content but also on whether its environment considers it worth keeping. This applies to AI outputs as well: if they provide correct or valuable solutions, they tend to be stored and re-used, whereas banal or incorrect results get deleted.

The lifetime of information

Mexican cultural tradition of Día de los Muertos and the anime One Piece have a similar concept on death:

When do you think people die? Is it when a bullet from a pistol pierces their heart? (…) No! It’s when they are forgotten by others! (—Dr. Hiriluk, One Piece)

You could call this specific type of death “informational death”. A specific information, a bytestream representing an idea, a theory, a proof, a book, a blog post, etc., is “dead” when its every last copy is erased from the universe, or cannot be retrieved in any way. Therefore, it is also possible to call a specific information “alive” when it is still being copied or referenced.

So, how could we formalize the survival of information? The answer is to use survival functions, a concept used in many fields, including biology, epidemiology, and economics.

Let us assume that we have an entity, an AI, that produces a sequence of information $x_1, x_2, \ldots, x_n$. For each piece of information $x_i$ produced by the AI, we define a random lifetime $T_i \ge 0$. $T_i$ is the time until $x_i$ is effectively forgotten, discarded, or overwritten in the environment.

We then describe the survival function as:

the probability that $x_i$ is still alive (stored, referenced, or used) at time $t$. This is independent of how many duplicates appear—we assume that at least one copy is enough to deem it alive.

In real life, survival depends on storage costs, attention spans, and the perceived value of the item. A short-lived text might disappear as soon as nobody refers to it. A revolutionary paper may endure for decades. Mathematical facts might be considered so fundamental that they become permanent fixtures of knowledge. When we speak of an AI that “naturally” produces persistent information, we are observing that correct or notable outputs often survive in their environment without the AI having to optimize explicitly for that outcome.

An expanding universe of information

In our definition above, we mention “out-of-distribution”ness, or novelty of information. This implies the existence of a distribution of information, i.e. a set of information containing all information that has ever been generated up to a certain time. We denote this set of cumulative information as $U$ for “universe”, which grows with every new information $x_i$ produced by the AI. Let

and then

In other words, once $x_{i+1}$ is added, it becomes part of the universe. Given an existing state of $U_i$, we can define and calculate a “novelty score” for a new information $x_{i+1}$ relative to $U_i$. If $x_{i+1}$ is basically a duplicate of existing material, its novelty score will be close to zero. If it is genuinely out-of-distribution, it would be large. Therefore, when a novel information $x_{i+1}$ is added to $U$, any future copies of it will be considered in-distribution and not novel. We denote the novelty score of $x_{i+1}$ as $n_{i+1}$.

So how could we calculate this novelty score? One way to calculate it is to use conditional Kolmogorov complexity:

where

is the length (in bits) of the shortest program that can generate $x$, when the set $U$ is given as as a free side input, and $M$ is the universal Turing machine.

How does this relate to novelty?

Low novelty: If $x$ can be produced very easily by simply reading or slightly manipulating $U$, then the program $p$ (which transforms $U$ into $x$) is small, making $K(x \mid U)$ and hence the novelty score is low. We would say that $x$ is almost already in $U$, or is obviously derivable from $U$.

High novelty: If $x$ shares no meaningful pattern with $U$, or can’t easily be derived from $U$, the program $p$ must be large. In other words, no short set of instructions that references $U$ is enough to produce $x$—it must encode substantial new information not present in $U$. That means $K(x \mid U)$ and hence the novelty score is high.

Informational fitness

We can now combine survival and novelty to formalize our informal definition of AGI-ness above. We integrate the survival function over time to the expected lifetime of information $x_i$:

Therefore, for an entity which generates information ${x_1, x_2, \ldots, x_n}$ over its entire service lifetime, we can compute a measure of “informational fitness” by multiplying the novelty score $n_i$ by the expected lifetime $L_i$ over all generated information:

This quantity tracks the total sum of both how novel each new piece of information an entity generates, and how long it remains in circulation.

My main idea is that a higher Informational Fitness would point to a higher ability to generalize, and hence a higher level of AGI-ness.

Because each subsequent item’s novelty is always measured with respect to the updated universe that includes all prior items, any repeated item gets a small or zero novelty score. Thus, it doesn’t inflate the overall Informational Fitness measure.

Why worry about novelty at all? My concern came from viruses, which are entities that copy themselves and spread, and therefore could be considered as intelligent if we simply valued how many times an information is copied. But viruses are obviously not intelligent—they mutate randomly and any novelty comes from selection by the environment. Therefore, a virus itself does not have a high IF in this model. However, an AI that can generate many new and successful viruses would indeed have a high IF.

Information’s relevance

Tying AGI-ness to survival of information renders the perception of generalization ability highly dependent on the environment, or in other words, state of the art at the time of an AI’s evaluation. Human societies (and presumably future AI societies) advance, and the window of what information is worth keeping drifts over time, erasing the information of the past. So whereas an AI of 2030 would have a high IF during the years it is in service, the same system (same architecture, training data, weights) would likely have a lower IF in 3030, due to being “out of date”. Sci-fi author qntm has named this “context drift” in his short story about digitalized consciousness.

Comparing AI with humans

Humans perish with an expected lifetime of 80 years, whereas AI is a digital entity that could survive indefinitely. Moreover, if you consider an AI’s performance depends on the hardware it runs on, you realize that IF should be derived from the maximum total throughput of all the copies of the AI that are running at a time. Basically, all the information that is generated by that specific version of the AI in the entire universe counts towards its IF.

Given this different nature of AI and humans, how fair would it be to compare a human’s informational fitness with an AI’s? After all, we cannot digitize and emulate a human’s brain with 100% fidelity with our current technology, and a fair comparison would require exactly that. We then quickly realize that we need to make assumptions and use thought experiments, like hypothetically scanning the brain of Albert Einstein (excuse the cliché) and running it at the same bitrate and level of parallelism as e.g. OpenAI’s most advanced model at the time. Or we could consider the entire thinking power of the human society as a whole and try to back-of-the-envelope-calculate that from the number of Universities and academics. But given that a lot of these people already use AI assistants, how much of their thinking would be 100% human?

The original OpenAI definition “a highly autonomous system that outperforms humans at most economically valuable work” is a victim of this as well. Humans are using AI now and are becoming more dependent on it, and smarter at the same time. Until we see an AI system that is entirely independent of human input, it will be hard to draw the line in between human and AI intelligence.

Thank you for reading up to this point. I think there might be a point in combining evolutionary biology with information theory. I tried to keep it simple and not include an information’s copy-count in the formulation, but it might be a good next step. If you think this post is good or just dumb, you can let me know at [email protected].

If you like this, you might also like my Instagram channel Nerd on Bars @nerdonbars where I calculate the power output of various athletes and myself.

This is an addendum to my previous post The Kilowatt Human. I mean it as half-entertainment and half-futuristic speculation. I extrapolate the following insight more into the future:

Before the industrial revolution, over 80% of the population were farmers. The average human had to do physical labor to survive. The average human could not help but to “bodybuild”.

Since then, humans have built machines to harness the power of nature and do the physical labor for them. What made the human civilization so powerful robbed individual humans of their own power, quite literally. The average pre-industrial human could generate a higher wattage than the average post-industrial human of today—they had to.

Before the industrial revolution, humanity’s total power output was bottlenecked by human physiology. Humanity has since moved up in the Kardashev scale. Paradoxically, the more power humanity can generate, the less physical exercise the average human can economically afford, and the weaker their body becomes.

Similar to the growth in humanity’s energy consumption, the average human’s physical strength will move down a spectrum, marked by distinct Biomechanical Stages, or BMS for short:

Why do I write this? My father grew up while working as a farmer on the side, then studied engineering. He never did proper strength training in his life. I grew up studying full-time, have been working out on and off, more so in the last couple of years. And I still have a hard time beating him in arm wrestling despite the 40 years of age gap. Our offsprings will be lucky enough if they can afford to have enough time and space to exercise. I hope that their future never becomes as dramatic as I describe below.

Biomechanical Stage I (Pre-Industrial Human Power)

Began with the Stone Age, followed by the era of metal tools, basic mechanical aids like mills, and ended with the industrial revolution:

Stone Age: No metal tools, no machinery. Humans rely on their bodies entirely—hunting, gathering, carrying, and building shelters by hand. Biomechanical power is the cornerstone of survival. The average human can generate and sustain relatively high wattage because everyone is physically active out of necessity. Most humans are hunter-gatherers.

Metal tools and agriculture: Introduction of iron and steel tools improves efficiency in cutting and shaping the environment. Most people farm, carrying heavy loads, tilling fields, harvesting. Though tools reduce some brute force, overall workloads remain high and physically demanding.

Primitive machinery (e.g. mills): Waterwheels and windmills start to handle some repetitive tasks like grinding grain. Still, daily life is labor-intensive for the majority. Physical strength remains a defining human attribute.

In this era, the biomechanical power of the average human is relatively high. The average human can generate and sustain relatively high wattage because everyone is physically active out of necessity.

Biomechanical Stage II (Industrial-Modern Human Power)

We are currently in this stage. It began with the Steam Age, followed by the widespread use of internal combustion engines and motorized vehicles, and will end at the near-future threshold where technology allows a human to be economically competitive and sustain themselves without ever moving their body.

Steam engine and early industry: Factories powered by steam reduce the need for raw human muscle. Some humans shift to repetitive but less physically grueling jobs. Manual labor declines for a portion of the population.

Motorized vehicles and automation (our present): Tractors, trucks, and powered tools handle the heavy lifting. Most humans now work in services or knowledge sectors. The need to exercise for health arises because physical strength no longer follows naturally from daily life. Specialty fields (construction, sports, fitness enthusiasts) maintain higher-than-average output, but they are exceptions.

Humans still have bodies and can choose to train them, but the average sustained power output falls as convenient transport, automation, and energy-dense foods foster sedentary lifestyles.

Robots and AI: Robots and AI are increasingly able to handle physical tasks that were previously done by humans. This further reduces the need for human physical labor.

As machines handle more tasks, the average person’s baseline physical capability drops. Exercise shifts from natural necessity to a personal choice or hobby.

Biomechanical Stage III (Post-Biological Human Power)

Future scenarios where brain-machine interfaces, telepresence, and total virtualization dominate. Will begin with a Sword-Art Online-like scenario where neural interfaces allows a human to remotely control a robot in an economically competitive way, while spending most of their time immobilized. Will end in a Matrix-like scenario where the average human is born as a brain-in-a-jar.

Brain Chips and Teleoperation: Humans remotely control robots with no physical exertion. Commuting is done digitally. Physical strength becomes even less relevant. The population’s average biomechanical output plummets because few move their own bodies meaningfully.

Quantum Entanglement and Zero-Latency Control: Even physical constraints of distance vanish. Humans may spend their entire lives in virtual worlds or controlling machines from afar, further reducing any reason to maintain physical strength.

Bodily Sacrifice, Brains in Jars: Eventually, bodies become optional. Nervous systems are maintained artificially, teleoperating robots when needed. Muscle tissue atrophies until it is nonexistent. The concept of human biomechanical power no longer applies. The definition of what a human is becomes more and more abstract. Is it organic nerve tissue or even just carbon-based life?

The human body, if it exists at all, is not maintained for physical tasks. The average person’s muscular capability collapses to negligible levels.

How does the Kardashev Scale align with the Biomechanical Stages?

In my opinion, the stages will not align perfectly with Kardashev Type I, II and III civilizations. Instead, they will overlap in the following way:

I think that by the time we reach other stars, we will also have pretty sophisticated telepresence and brain-machine interface technology. In fact, those technologies might be the only way to survive such journeys, or not have to make them at all, as demonstrated in the Black Mirror episode Beyond the Sea:

So BMS-III might already be here by the time we are a Type II civilization. As for what an organic body means for a Type III galactic civilization, I can’t even begin to imagine.

This post has mostly been motivated by my sadness that while our life quality has increased with technology, it has also decreased in many other ways. We evolved for hundreds of thousands of years to live mobile lives. But we became such a successful civilization that we might soon not be able to afford movement. We are thus in a transitory period where we started to diverge from our natural way of life, too quickly for evolution to catch up. And when evolution finally does catch up, what will that organism look like? How will it feed itself, clean itself and reproduce? Will the future humans be able to survive going outside at all?

In another vein, technology could also help us perfectly fit bodies by altering our cells at a molecular level. But if there is no need to move to contribute to the economy, why would anyone do such an expensive thing?

My hope is that sexual competition and the need for reproduction will maintain an evolutionary pressure just enough to keep our bodies fit. This assumes that individual humans are still in control of their own reproduction and can select their partners freely. Because a brain-in-a-jar is obviously not an in-dividual—they have been divided into their parts and kept only the one that is economically useful.

tl;dr: I calculate my power output in Watts/Horsepower and aim to maximize that, instead of muscle volume, in my workouts:

This is a work in progress, email feedback to [email protected].

If you like this, you might also like my Instagram channel Nerd on Bars @nerdonbars where I calculate the power output of various athletes and myself.

Why do people hit the gym? What is their goal?

For some, it is to put on muscle and look good. For others, it is to be healthy and live longer. For yet others, it is to have fun, because doing sports is fun. None of these are mutually exclusive.

In this post, I will not focus on any of these. I will focus on the goal of getting strong and building power. I write this, because I feel like people are doing exercise more and more for appearance’s sake, and less to get strong. And it has to do with economics.

Before the industrial revolution, over 80% of the population were farmers. The average human had to do physical labor to survive. The average human could not help but to “bodybuild”.

Since then, humans have built machines to harness the power of nature and do the physical labor for them. What made the human civilization so powerful robbed individual humans of their own power, quite literally. The average pre-industrial human could generate a higher wattage than the average post-industrial human of today—they had to.

Before the industrial revolution, humanity’s total power output was bottlenecked by human physiology. Humanity has since moved up in the Kardashev scale. Paradoxically, the more power humanity can generate, the less physical exercise the average human can economically afford, and the weaker their body becomes. Strength has become a luxury.

This is why most modern fitness terms make me sad, because they remind me of what has been lost.

Consider “functional training”. There used to be no training other than “functional”, because most physical effort had to create economic value. The term is used to differentiate between exercises with machines which target specific muscles, and exercises that are composed of more “compound movements” that mimic real-life activities. It used to be that people did not have to do any training, because physical exercise was already a part of their daily life.

This is why I dislike “building muscle” as a goal as well. Since strength is a luxury now, people want to maximize that in their lives. However, they end up trying to maximize the appearance of strength, because increasing actual strength is harder than building muscle.

When I say it is harder to get strong than to look strong, I mean it in the most materialistic sense: Increasing your body’s power output in Watts is harder and economically more expensive than increasing muscle volume in Cubic Centimeters. Increasing wattage has a higher time and money cost, requires more discipline and a lot more effort. It is a multi-year effort.

Contrarily, muscle can be built quicker in a matter of months, without getting relatively stronger. Many bodybuilders can’t do a few pull-ups with proper form. Their strength doesn’t transfer to other activities. They are sluggish and lack agility. In that sense, bodybuilding culture today embodies the worst parts of capitalism and consumerism. Empty, hollow muscle as a status symbol. Muscle for conspicuous fitness.

To meet up the demand, capitalism has commoditized exercise in the form of the modern machine-laden gym: a cost-optimized low-margin factory. Its product is the ephemeral Cubic Centimeter of Muscle™ which goes away quickly the moment you stop working out.

These gyms are full of people whose main motivation for working out is feeling socially powerless and unattractive. However, instead of going after real physical power, i.e. Watts, they go after the appearance of power, muscle volume. They compare themselves to people that just look bigger, people with higher volume.

The goal of this post is to convince you that it is superior to chase Watts than to chase muscle volume. It is psychologically more rewarding, the muscle gained from it is more permanent and has higher power density. However, it is more difficult and takes longer to achieve.

Goals

Goals matter. For example, if you purely want to maximize your muscle mass or volume, using steroids or questionable supplements is a rational thing to do. Enough people have criticized it such that I don’t need to. Disrupting your hormonal system just to look bigger and be temporarily stronger is extremely dumb.

I personally want to:

• feel powerful, and not just look like it.
• live as long and healthily as I can.

I believe that the best way to do that is to increase my power output in Watts and do regular strength training in a balanced way that will not wear out my body.

If I had to define an objective function for my exercise, it would be:

where $P [\text{Watt}]$ is my power output, $L(P)[\text{year}]$ is the length of my life as a function of my power output, $\alpha$ and $\beta$ are weights that I assign to power and longevity. I won’t detail this any further, because I don’t want to compute anything. I just want to convey my point.

Notice how I don’t constrain myself to any specific type of exercise, such as calisthenics or weightlifting. As long as it makes me more powerful, anything goes. Is wrestling going to get me there? Count me in. Is working in the fields, lifting rocks, firefighter training or Gada training going to get me there? I don’t differentiate. As long as it makes me more powerful, I am in.

Calculating power

How can one even calculate their power output?

It is actually quite easy to do, with high-school level physics. You just need to divide the work done by the time it took.

For example, consider a muscle-up:

I am at the starting position on the left, and at the top of the movement on the right. In both frames, my velocity is 0, so there is no kinetic energy. Therefore, we can calculate a lower bound of my power output by comparing the potential energies between the two frames. Denoting the left frame with subscript 0 and the right frame with subscript 1, we have:

where $U$ is the potential energy, $m$ is my mass, $g = 9.81 m/s^2$ is the acceleration due to gravity and $h$ is the height.

The work I do is the change in potential energy:

And my power output is the work divided by the time it took:

The distance I traveled $\Delta h$ can be calculated from anthropometric measurements:

I will denote the distances from this figure with subscripts $d_A$, $d_B$ and so on. Comparing this with the previous figure, we have roughly:

To understand how I derive this, consider the hands fixed during the movement and that the body is switching from a position where the arms are extended upwards to a position where the arms are extended downwards.

I have measured my own body, and found this to be roughly equal to 130 cm. Given that it took me roughly 2 seconds to do the movement and my mass at the time was roughly 78 kg, I have found the lower bound of my power output to be:

It is a lower bound, because the muscles are not 100% efficient, some energy is dissipated e.g. as heat during the movement, my movement is not perfectly uniform, etc.

Still, the lower bound calculation is pretty concise, and can be made even more accurate with a stopwatch and a slow-motion camera.

Aiming for 1 kilowatt

When I was first running to calculations, I wanted to get a rough idea of the order of magnitude of the power output of various exercises. It surprised me when I found out that most exercises are in 10-1000 Watt range, expressable without an SI prefix.

I have been training seriously for almost a year and regularly for a couple of years before that. I have discovered that in my current state, my unweighted pull-ups are in the 500-1000 Watt range. For the average person, 1000 Watts, i.e. 1 kilowatt, is an ambitious goal, but not an unattainable one. 1 kilowatt simply sounds cool as a target to aim for, as if you are a dynamo, a machine. A peak athlete can easily generate 1 kilowatt with their upper body for short durations.

How does this reflect to the muscle-up example I gave above?

If I am not adding any additional weights to my body, that means the duration which I complete the movement would need to decrease. We can calculate how much that would need to be. Moreover, we can derive a general formula which calculates how fast anyone would need to perform a muscle-up to generate 1 kilowatt.

To do that, we first need to express power in terms of the person’s height. Previously, we had $\Delta h = d_A - d_G$. Most people have roughly similar anthropometric ratios, so we can use my measurements to approximate that ratio. Multiply and divide by $d_B$ to get:

For me, $d_A = 215 \text{cm}$, $d_B = 180 \text{cm}$ and $d_G = 85 \text{cm}$, so:

Let’s denote the person’s height $d_B$ as $h_p$. Then we have

Therefore, the power output can be expressed as:

Since we want to generate 1 kilowatt, we can solve for $\Delta t$:

If we substitute $g = 9.81 \text{m/s}^2$ and assume $h_p$ is in centimeters, we get roughly:

The formula is really succinct and easy to remember: Just multiply the person’s mass in kilograms by their height in centimeters and divide by 14000.

Calculating for myself, I get $78 \times 180 / 14000 \approx 1.00$ seconds.

This confirms that I need to get two times faster in order to generate 1 kilowatt. Alternatively, if I hit a wall in terms of speed, I could add weights to my body to increase my power output. (TBD)

My friend and trainer J has agreed to record his muscle-up and various other exercises, so I will add his numbers and compare them soon.

TBD: Add the data from J.

Extending to other movements

I chose the muscle-up because I’ve been working on it recently. However, this method can be applied to any movement, as it’s just an application of basic physics.

For example,

• Do you want to calculate the power output of a pull-up? You just need to change the height $\Delta h$, it’s roughly half the distance for muscle-up.
• Do you want to calculate the power output of a weighted pull-up? You just need to add the additional mass to your body mass $m$.
• Do you want to calculate the power output of a sprint start? Just measure your top speed at the beginning and the time it took to accelerate to that speed, and divide your kinetic energy by that time.
• Do you want to calculate the power output of a bench press? You need to set $\Delta h$ as your arm length and $m$ as the weight of the barbell.

See the next section for a more detailed example.

Power-weight relationship in a bench press

In the bodyweight examples above, we had the same bodyweight, and it was being moved over different distances.

Then a good question to ask is: How does the power output scale with the weight lifted? The bench press is an ideal exercise to measure this in a controlled way.

I asked my friend to help me out with timing bench press repetitions over 6 rounds with different weights. You can see these in the video above.

Before we even look at the results, we can use our intuition to guess what kind of relationship we will see. If the weight is low, power is low as well. So as we increase the weight, we expect the power to increase. However, human strength is limited, so the movement will slow down after a certain point, and the power will decrease. We should see the power first increase with weight, and then decrease. This is indeed what happens.

In each round, my friend did 3 to 4 repetitions with the same weight. I calculated the average time it took to complete the repetition and the total weight (barbell + plates) lifted in that round. Then, I calculated the power output for each round using the formula above. The height that the barbell travels during the ascent is $\Delta h = 43 \text{cm}$.

The visualizations below are aligned with the intuition:

The figures matches the perceived difficulty of the exercise. My friend said he usually trains with 45-50 kg, and it started to feel difficult in the last 2 rounds. His usual load is under the 55 kg limit where his power saturates. That could mean he is under-loading, and should load at least 60 kg to achieve progressive overload and increase his power.

Reinventing Velocity Based Training, Plyometrics etc.

Power is a factor of speed and force. So in a nutshell, this project is about maximizing speed and force at the same time.

While starting this project, I wanted to have a fresh engineer’s look at powermaxxing, and did not want to get influenced by existing methods or literature. I knew that sports people were using scientific methods to measure and improve performance for decades, but I wanted to discover things on my own. I will continue to stay away from existing knowledge for some time, before I look at them in more detail.

Also: I have personally not seen any person on social media that tracks power output in Watts, or visualizes it with a Wattmeter.

If you know about such a channel, please let me know.

Not-conclusion

This is a work in progress, so there is no conclusion to this yet. I will add more content as I learn more.

Aim for Watts. It is hard, but more rewarding.

OpenAI released a new model that “thinks” to itself before it answers. They intentionally designed the interface to hide this inner monologue. There was absolutely no technical reason to do so. Only business reasons

If you try to make o1 reveal its inner monologue, they threaten to remove your access

Because if they let people freely extract this, competitors could quickly use that to improve their models

It seems that AI value creation will be shifting more towards inference-time compute, into Chains of Thought. We might be witnessing the birth of a new paradigm of open vs. closed thought

Impressive as o1 is, the move to hide CoTs is pretty pathetic and reminds of Microsoft’s late 90s Windows Server push. Below is an email from Bill Gates about how he is worried that Microsoft won’t be able to corner the server market. A few years after he wrote those lines, Linux and LAMP came to dominate servers

Now all eyes on AI at Meta and Zuck for their take on o1/Strawberry/Q*/Orion

Originally posted on LinkedIn

Imagine the following scenario:

1. We develop brain-scan technology today which can take a perfect snapshot of anyone’s brain, down to the atomic level. You undergo this procedure after you die and your brain scan is kept in some fault-tolerant storage, along the lines of GitHub Arctic Code Vault.
2. But sufficiently cheap real-time brain emulation technology takes considerably longer to develop—say 1000 years in the future.
3. 1000 years pass. Everyone that ever knew, loved or cared about you die.

Here is the crucial question:

Given that running a brain scan still costs money in 1000 years, why should anyone bring *you* back from the dead? Why should anyone boot *you* up?

Compute doesn’t grow in trees. It might become very efficient, but it will never have zero cost, under physical laws.

In the 31st century, the economy, society, language, science and technology will all look different. Most likely, you will not only NOT be able to compete with your contemporaries due to lack of skill and knowledge, you will NOT even be able to speak their language. You will need to take a language course first, before you can start learning useful skills. And that assumes some future benefactor is willing to pay to keep you running before you can start making money, survive independently in the future society.

To give an example, I am a software developer who takes pride in his craft. But a lot of the skills I have today will most likely be obsolete by the 31st century. Try to imagine what an 11th century stonemason would need to learn to be able to survive in today’s society.

1000 years into the future, you could be as helpless as a child. You could need somebody to adopt you, send you to school, and teach you how to live in the future. You—mentally an adult—could once again need a parent, a teacher.

(This is analogous to cryogenics or time-capsule sci-fi tropes. The further in the future you are unfrozen, the more irrelevant you become and the more help you will need to adapt.)

Patchy competence?

On the other hand, it would be a pity if a civilization which can emulate brain scans is unable to imbue them with relevant knowledge and skills, unable to update them.

For one second, let’s assume that they could. Let’s assume that they could inject your scan with 1000 years of knowledge, skills, language, ontology, history, culture and so on.

But then, would it still be you?

But then, why not just create a new AI from scratch, with the same knowledge and skills, and without the baggage of your personality, memories, and emotions?

Why think about this now?

Google researchers recently published connectomics research (click here for the paper) mapping a 1 mm³ sample of temporal cortex in a petabyte-scale dataset. Whereas the scanning process seems to be highly tedious, it can yield a geometric model of the brain’s wiring at nanometer resolution that looks like this:

They have even released the data to the public. You can download it here.

An adult human brain takes up around 1.2 liters of volume. There are 1 million mm³ in a liter. If we could scale up the process from Google researchers 1 million times, we could scan a human brain at nanometer resolution, yielding more than 1 zettabyte (i.e., 1 billion terabytes) of data with the same rate.

That is an insane amount of data, and it seems infeasible to store that much data for a sufficient number of bright minds, so that this technology can make a difference. That being said, do we have any other choice but to hope that we will find a way to compress and store it efficiently?

Not only it is infeasible to store that much data with current technology, extracting a nanometer-scale connectome of a human brain may not be enough to capture a person’s mind in its entirety. By definition, some information is lost in the process. Fidelity will be among the most important problems in neuropreservation for a long time to come.

That being said, the most important problem in digital immortality may not be technical, but economical. It may not be about how to scan a brain, but about why to scan a brain and run it, despite the lack of any economic incentive.

tl;dr Skip to the Conclusion. Don’t forget to look at the graphs.

Unlike a single “the” in the English language, the German language has 6 definite articles that are used based on a noun’s gender, case and number:

• 6 definite articles: der, die, das, den, dem, des
• 3 genders: masculine, feminine, neuter (corresponding to “he”, “she”, “it” in English)
• 4 cases: nominative, accusative, dative, genitive
• 2 numbers: singular, plural

The following table is used to teach when to use which definite article:

Importantly, native speakers don’t look at such tables while learning German as a child. They internalize the rules through exposure and practice.

If you are learning German as a second language, however, you will most likely spend time writing down these tables and memorizing them.

While learning, you will also memorize the genders of nouns. For example, “der Tisch” (the table) is masculine, “die Tür” (the door) is feminine, and “das Buch” (the book) is neuter. Whereas predicting the case and number is straightforward and can be deduced from the context of the sentence, predicting the gender can be much more difficult.

Without going into much detail, take my word for now that the genders are semi-random. Inanimate objects such as a bus can be a “he” or “she”, whereas animate objects such as a “girl” can be a “it”.

Because of all this, German learners fail to remember the correct gender at times and develop strategies, heuristics, to fall back to some default gender or article when they are unsure. For example, some learners use “der” as a default article when they are unsure, whereas others use “die” or “das”.

I have taken many German courses since middle school. Most German courses teach you how to use German correctly, but very few of them teach you what to do when you don’t know how to use German correctly, like when you don’t know the gender of an article.

This is a precursor to a future post where I will write about those strategies. Any successful strategy must be informed by the frequencies and probability distribution of noun declensions. To that end, I performed Natural Language Processing on two corpuses of the German language:

• Transcriptions of over 140 hours of videos from the Easy German YouTube channel, which contains lots of street interviews and other spoken examples.
• 10kGNAD: Ten Thousand German News Articles Dataset, which contains over 10,000 cleaned up news articles from an Austrian newspaper.

I will introduce some notation to represent these frequencies easier, which are going to be followed by the results of the analysis.

Mapping the space of noun declensions

The goal of this article is to show the frequencies of definite articles alongside the declensions of the nouns they accompany. To be able to do that, we need a concise notation to represent the states a noun can be in.

To this end, we introduce the set of grammatical genders $G$,

the set of grammatical cases $C$,

and the set of grammatical numbers $N$,

The set of all possible grammatical states $S$ for a German noun is

whose number of elements is $|S| = 3 \times 4 \times 2 = 24$.

To represent the elements of this set better, we introduce the index notation

for $i=1,2$, $j=1,2,3$ and $k=1,2,3,4$ correspond to the elements in the order seen in the definitions above.

Elements of $S$ can be shown in a single table, like below:

In practice, plural forms of articles and declensions for all genders are the same in each case, so they are shown next to the singular forms:

which is the case in Table 1 above. You might say, “well, of course”. In that case, I invite you to imagine a parallel universe where German grammar is even more complicated and plural forms have to be declined differently as well. Interestingly, you don’t need to visit such a universe—you just need to go back in time, because Old High German grammar was exactly like that. Note that in that Wikipedia page, some tables has the same shape as Table 2.

Why introduce such confusing looking notation? It might look confusing to the untrained eye, but it is actually very useful for representing all possible combinations in a compact way. It also makes it easier to run a sanity check on the results of the analysis through the independence axiom, which we will introduce next.

Relationships between probabilities

As a side note, the relationship between the probabilities of all grammatical states of a noun and the probabilities of each case is as below:

Similarly, for each gender:

And for each number:

This is useful for going from specific probabilities to general probabilities and vice versa.

Independence Axiom

We introduce an axiom that will let us run a sanity check on the results of the analysis. At a high level, the axiom states that the probability of a noun being in a certain case, a certain gender and a certain number are all independent of each other. For example, the probability of a noun being in the nominative case is independent of the probability of it being masculine or feminine or neuter, and it is also independent of the probability of it being singular or plural. This should be common sense in any large enough corpus, so we just assume it to be true.

Formally, the axiom can be written as

where $P(G_i) P(C_j) P(N_k)$ is the joint probability of the noun being in the grammatical state $S_{ijk}$.

In any given corpus, it will be hard to get this equality to hold exactly. In reality, a given corpus or the NLP libraries used in the analysis might have a bias that might distort the equality above.

The idea is that the smaller the difference between the left-hand side and the right-hand side, the more the corpus and the NLP libraries are unbiased and adhere to common sense. As a corpus gets larger and more representative of the entire language, the following quantity should get smaller:

We will calculate this quantity for the two corpuses we have and see how biased either they or the NLP libraries are.

Note that the notation $\hat{P}(S_{ijk})$ is used to denote the empirical probability of the noun being in the grammatical state $S_{ijk}$, which is calculated from the corpus as

where $N_{ijk}$ is the count of the noun being in the grammatical state $S_{ijk}$. Similar notation is used for $\hat{P}(G_i)$, $\hat{P}(C_j)$ and $\hat{P}(N_k)$.

The analysis

I outline step by step how I performed the analysis on the two corpuses.

Constructing the spoken corpus

The Easy German YouTube Channel is a great resource for beginner German learners. It has lots of street interviews with random people on a wide range of topics.

To download the channel, I used yt-dlp, a youtube-dl fork:

#!/bin/bash
mkdir data
cd data
yt-dlp -f 'ba' -x --audio-format mp3  https://www.youtube.com/@EasyGerman

This gave me 946 audio files with over 139 hours of recordings. Then I used OpenAI’s Whisper API to transcribe all the audio:

import json
import os

import openai
from tqdm import tqdm

DATA_DIR = "data"
OUTPUT_DIR = "transcriptions"

# Get all mp3 files in the current directory
mp3_files = [
    f for f in os.listdir(DATA_DIR) if os.path.isfile(f) and f.endswith(".mp3")
]

mp3_files = sorted(mp3_files)

# Create the output directory if it doesn't exist
if not os.path.exists(OUTPUT_DIR):
    os.makedirs(OUTPUT_DIR)

for file in tqdm(mp3_files):
    # Create json target file name in output directory
    json_file = os.path.join(OUTPUT_DIR, file.replace(".mp3", ".json"))

    # If the json file already exists, skip it
    if os.path.exists(json_file):
        print(f"Skipping {file} because {json_file} already exists")
        continue

    # Check if the file is greater than 25MB
    if os.path.getsize(file) > 25 * 1024 * 1024:
        print(f"Skipping {file} because it is greater than 25MB")
        continue

    print(f"Running {file}")
    try:
        output = openai.Audio.transcribe(
            model="whisper-1",
            file=open(file, "rb"),
            format="verbose_json",
        )
        output = output.to_dict()
        json.dump(output, open(json_file, "w"), indent=2)
    except openai.error.APIError:
        print(f"Skipping {file} because of API error")
        continue

This gave me a lot to work with, specifically a little bit over 1 million words of spoken German. As a reference, the content of the videos can fill roughly more than 10 novels, or alternatively, 400 Wikipedia articles. Note that I created this dataset around May 2023, so the dataset would be even bigger if I ran the script today. However, it still costs money to transcribe the audio, so I will stick with this dataset for now.

Constructing the written corpus

The 10kGNAD: Ten Thousand German News Articles Dataset contains over 10,000 cleaned up news articles from an Austrian newspaper. I downloaded the dataset and modified the script they provided to extract the articles from the database and write them to a text file:

import re
import sqlite3

from tqdm import tqdm
from bs4 import BeautifulSoup


ARTICLE_QUERY = (
  "SELECT Path, Body FROM Articles "
  "WHERE PATH LIKE 'Newsroom/%' "
  "AND PATH NOT LIKE 'Newsroom/User%' "
  "ORDER BY Path"
)

conn = sqlite3.connect(PATH_TO_SQLITE_FILE)
cursor = conn.cursor()

corpus = open(TARGET_PATH, "w")

for row in tqdm(cursor.execute(ARTICLE_QUERY).fetchall(), unit_scale=True):
    path = row[0]
    body = row[1]
    text = ""
    description = ""

    soup = BeautifulSoup(body, "html.parser")

    # get description from subheadline
    description_obj = soup.find("h2", {"itemprop": "description"})
    if description_obj is not None:
        description = description_obj.text
        description = description.replace("\n", " ").replace("\t", " ").strip() + ". "

    # get text from paragraphs
    text_container = soup.find("div", {"class": "copytext"})
    if text_container is not None:
        for p in text_container.findAll("p"):
            text += (
                p.text.replace("\n", " ")
                .replace("\t", " ")
                .replace('"', "")
                .replace("'", "")
                + " "
            )
    text = text.strip()

    # remove article autors
    for author in re.findall(
        r"\.\ \(.+,.+2[0-9]+\)", text[-50:]
    ):  # some articles have a year of 21015..
        text = text.replace(author, ".")

    corpus.write(description + text + "\n\n")

conn.close()

This gave me 10277 articles with around 3.7 million words of written German. Note that this is over 3 times bigger than the spoken corpus.

NLP and counting the frequencies

I used spaCy for Part-of-Speech Tagging. This basically assigns to each word whether it is a noun, pronoun, adjective, determiner etc. Definite articles will have the PoS tag "DET" in the output of spaCy.

spaCy is pretty useful. For any token in the output, token.head gives the syntactic parent, or “governor” of the token. For definite articles like “der”, “die”, “das”, the head will be the noun they are referring to. If spaCy couldn’t connect the article with a noun, any deduction of gender has a high likelihood of being wrong, so I skip those cases.

import numpy as np
import spacy
from tqdm import tqdm

CORPUS = "corpus/easylang-de-corpus-2023-05.txt"
# CORPUS = "corpus/10kGNAD_single_file.txt"

ARTICLES = ["der", "die", "das", "den", "dem", "des"]
CASES = ["Nom", "Acc", "Dat", "Gen"]
GENDERS = ["Masc", "Fem", "Neut"]
NUMBERS = ["Sing", "Plur"]

CASE_IDX = {i: CASES.index(i) for i in CASES}
GENDER_IDX = {i: GENDERS.index(i) for i in GENDERS}
NUMBER_IDX = {i: NUMBERS.index(i) for i in NUMBERS}

# Create an array of the articles
ARTICLE_ijk = np.empty((2, 3, 4), dtype="<U32")

ARTICLE_ijk[0, 0, 0] = "der"
ARTICLE_ijk[0, 1, 0] = "die"
ARTICLE_ijk[0, 2, 0] = "das"
ARTICLE_ijk[0, 0, 1] = "den"
ARTICLE_ijk[0, 1, 1] = "die"
ARTICLE_ijk[0, 2, 1] = "das"
ARTICLE_ijk[0, 0, 2] = "dem"
ARTICLE_ijk[0, 1, 2] = "der"
ARTICLE_ijk[0, 2, 2] = "dem"
ARTICLE_ijk[0, 0, 3] = "des"
ARTICLE_ijk[0, 1, 3] = "der"
ARTICLE_ijk[0, 2, 3] = "des"
ARTICLE_ijk[1, :, 0] = "die"
ARTICLE_ijk[1, :, 1] = "die"
ARTICLE_ijk[1, :, 2] = "den"
ARTICLE_ijk[1, :, 3] = "der"

# Use the best transformer-based model from SpaCy
MODEL = "de_dep_news_trf"
nlp_spacy = spacy.load(MODEL)

# Initialize the count array. We will divide the elements by the
# total count of articles to get the probability of each S_ijk
N_ijk = np.zeros((len(NUMBERS), len(GENDERS), len(CASES)), dtype=int)

corpus = open(CORPUS).read()
texts = corpus.split("\n\n")

for text in tqdm(texts):
    # Parse the text
    doc = nlp_spacy(text)

    for token in doc:
        # Get token string
        token_str = token.text
        token_str_lower = token_str.lower()

        # Skip if token is not one of der, die, das, den, dem, des
        if token_str_lower not in ARTICLES:
            continue

        # Check if token is a determiner
        # Some of them can be pronouns, e.g. a large percentage of "das"
        if token.pos_ != "DET":
            continue

        # If SpaCy couldn't connect the article with a noun, skip
        head = token.head
        if head.pos_ not in ["PROPN", "NOUN"]:
            continue

        # Get the morphological features of the token
        article_ = token_str_lower
        token_morph = token.morph.to_dict()
        case_ = token_morph.get("Case")
        gender_ = token_morph.get("Gender")
        number_ = token_morph.get("Number")

        # Get the indices i, j, k
        gender_idx = GENDER_IDX.get(gender_)
        case_idx = CASE_IDX.get(case_)
        number_idx = NUMBER_IDX.get(number_)

        # If we could get all the indices by this point, try to get the
        # corresponding article from the array we defined above.
        # This is another sanity check
        if gender_idx is not None and case_idx is not None and number_idx is not None:
            article_check = ARTICLE_ijk[number_idx, gender_idx, case_idx]
        else:
            article_check = None

        # If the sanity check passes, increment the count of N_ijk
        if article_ == article_check:
            N_ijk[number_idx, gender_idx, case_idx] += 1

To calculate $\hat{P}(S_{ijk})$, we divide the counts by the total number of articles:

P_S_ijk = N_ijk / np.sum(N_ijk)

Then we calculate the empirical probabilities of each gender, case and number:

# Probabilities for each number
P_N = np.sum(P_S_ijk, axis=(1, 2))

# Probabilities for each gender
P_G = np.sum(P_S_ijk, axis=(0, 2))

# Probabilities for each case
P_C = np.sum(P_S_ijk, axis=(0, 1))

The joint probability $\hat{P}(G_i) \hat{P}(C_j) \hat{P}(N_k)$ is calculated as:

joint_prob_ijk = np.zeros((2, 3, 4))

for i in range(2):
    for j in range(3):
        for k in range(4):
            joint_prob_ijk[i, j, k] = P_N[i] * P_G[j] * P_C[k]

Finally, we calculate the difference between the empirical probabilities and the joint probabilities:

delta_ijk = 100 * (P_S_ijk - joint_prob_ijk)

This will serve as an error term to see how biased the corpus is. The bigger the error term, the higher the chance of something being wrong with the corpus or the NLP libraries used.

High level results

I compare the following statistics between the spoken and written corpus:

• The frequencies of definite articles.
• The frequencies of genders.
• The frequencies of cases.
• The frequencies of numbers.

As I have already annotated in the code above, the analysis took into account the tokens that match the following criteria:

• Is one of “der”, “die”, “das”, “den”, “dem”, “des”,
• Has the PoS tag DET
• Is connected to a noun (token.head.pos_ is either PROPN or NOUN)

This lets me count the frequencies of the definite articles alongside the declensions of the nouns they accompany. The results are as follows:

Frequencies of genders

The distribution of the genders of the corresponding nouns is as below:

Observations:

• The written corpus contains ~6 percentage points less neuter nouns than the spoken corpus.
• This ~6 pp difference is distributed almost equally between the masculine and feminine nouns, with the written corpus containing ~3 pp more feminine nouns and ~3 pp more masculine nouns.

The difference is considerable and might point out to a bias in the way Whisper transcribed the speech or spaCy has parsed it. Both corpora are large enough to be representative, so this needs investigation in a future post.

Frequencies of cases

The distribution of the cases that the article-noun pairs are in is as below:

The spoken corpus has ~10 pp more accusative nouns, ~2 pp more dative nouns and ~13 pp less genitive nouns compared to the written corpus. The nominative case is more or less the same in both corpora.

This might be the analysis capturing the contemporary decline of the genitive case in the German language, as popularized by Bastian Sick with the phrase “Der Dativ ist dem Genitiv sein Tod” (The dative is the death of the genitive) with his eponymous book. However, the graph clearly shows a trend towards accusative, and much less towards dative.

Moreover, written language differs in tone and style from spoken language for many languages, including German. This might also explain the differences in the frequencies of the cases.

If this is not due to a bias, we might be onto something here. This also needs further investigation in a future post.

Frequencies of numbers

The distribution of the numbers of the corresponding nouns is as below:

The ratio of singular to plural nouns is more or less the same in both corpora. I wonder whether this 80-20 ratio is “universal” in German or any other languages as well…

Frequencies of definite articles

The distribution of the definite articles in the spoken and written corpus is as below:

Observations:

• der appears less frequently (~8 pp difference),
• die appears more frequently (~4 pp difference),
• das appears more frequently (~7 pp difference),
• des appears less frequently (~5 pp difference),

in the spoken corpus compared to the written corpus. den and dem are more or less the same in both corpora.

The ~7 pp difference in das is despite the fact that ~78% of the occurrence of the token das in the spoken corpus are pronouns (PRON, not DET) and hence excluded from the table above. See the section below for more details. Looking at the gender distribution above, the spoken corpus contains ~6 pp more neuter nouns than the written corpus, which might explain this discrepancy.

Empirical probabilities for the spoken corpus

Empirical probabilities:

Click below to see the joint probabilities and their differences as an error term:

Observations:

For most elements, the differences are less than 1-2%, which is a good sign. However, significant bias shows for some cases:

• 4.80 % (der, feminine, dative, singular)
• -2.65 % (dem, masculine, dative, singular)
• -1.91 % (die, feminine, nominative, singular)
• -1.33 % (die, feminine, nominative, plural)
• and so on…

I add more comments following the results for the written corpus below.

Empirical probabilities for the written corpus

Click below to see the joint probabilities and their differences as an error term:

Observations:

The difference terms follow a similar pattern to the spoken corpus in the extreme cases:

• 3.26 % (der, feminine, dative, singular)
• -2.53 % (dem, masculine, dative, singular)
• -1.35 % (die, feminine, nominative, plural)

Since the bias is most extreme in many common cells, this leads me to believe that there is a bias in spaCy’s de_dep_news_trf model that confuses the case or gender in some cases. This hypothesis can be tested by using a different model and library, and calculating the differences again. I’m leaving that as future work.

Calculating the number of articles used as determiners versus pronouns

Another comparison of interest is whether one of the “der”, “die”, “das”, “den”, “dem”, “des” is used more as a pronoun than as a determiner. To give an example, “das” can be used as a pronoun in the sentence “Das ist ein Buch” (That is a book) or as a determiner in the sentence “Das Buch ist interessant” (The book is interesting).

We can calculate this by storing the PoS tags of tokens that match “der”, “die”, “das”, “den”, “dem”, “des” and dividing the numbers by the occurrence of each article.

import spacy
from tqdm import tqdm

CORPUS = "corpus/easylang-de-corpus-2023-05.txt"
# CORPUS = "corpus/10kGNAD_single_file.txt"

ARTICLES = ["der", "die", "das", "den", "dem", "des"]

MODEL = "de_dep_news_trf"
nlp_spacy = spacy.load(MODEL)

# This array will store the count of each POS tag for each article
POS_COUNT_DICT = {i: {} for i in ARTICLES}

corpus = open(CORPUS).read()
texts = corpus.split("\n\n")

for text in tqdm(texts):
    doc = nlp_spacy(text)

    for token in doc:
        success = True

        # Get token string
        token_str = token.text
        token_str_lower = token_str.lower()

        if token_str_lower not in ARTICLES:
            continue

        if token.pos_ not in POS_COUNT_DICT[token_str_lower]:
            POS_COUNT_DICT[token_str_lower][token.pos_] = 0

        POS_COUNT_DICT[token_str_lower][token.pos_] += 1

print(POS_COUNT_DICT)

For both corpora, the >99% of the PoS tags are either DET or PRON. I have ignored the rest of the tags for simplicity.

Observations:

The spoken corpus overall uses more pronouns than the written corpus. The most striking difference is in the usage of “das” as a pronoun, with the spoken corpus using it as a pronoun in ~45 pp more cases than the written corpus. This might be due to a bias at any point in the analysis pipeline, or it might be due to the nature of spoken versus written language.

Conclusion

I have already commented a great deal below each result above. I don’t want to speak in absolutes at this point, because the analysis might be biased due to the following factors:

• Corpus bias: Easy German is a YouTube channel for German learning, and despite having a diverse set of street interviews, there is also a lot of accompanying content that might skew the results. Similarly, the 10kGNAD dataset is a collection of news articles from an Austrian newspaper, which might also skew the results. There might be differences between Austrian German and German German. To overcome any corpus related biases, this work should be repeated with even more data.
• Transcription bias: I used OpenAI’s Whisper V2 in May 2023 to transcribe the spoken corpus. There might be a bias in Whisper that might show up in the results. Whisper is currently among state-of-the-art speech-to-text models. We will most likely get better, faster and cheaper models in the upcoming years, and we can then repeat this analysis with them.
• NLP bias: I used spaCy’s de_dep_news_trf model for Part-of-Speech Tagging. There might be a bias in this model that might show up in the results. I might use another library in spaCy, or a different NLP library altogether, to see if the results change.

That being said, if I were to draw any conclusions from the results above, those would be:

Most frequent articles

For spoken German, the most frequently used definite articles (excluding pronouns) are in the order: die > der > das > den > dem > des.

For written German, the order is: der > die > den > das > des > dem.

die is statistically the most used definite article with close to 40% usage in spoken German Moreover, der, die and das collectively make up ~80% of the definite articles used in spoken German. So if you never learn the rest, you would be speaking German correctly 80% of the time, assuming that you are using the cases correctly.

Using das as pronoun in spoken German

das is used as a pronoun much more frequently in spoken German than in written German.

Most frequent genders

The most frequently used genders are in the order: feminine > masculine > neuter. This is widely known and has been recorded by many other studies as well.

Genitive on the fall, accusative (more so) and dative (less so) on the rise

Germans use genitive much less when speaking compared to writing. Surprisingly, this reflects in an increase more in the accusative case than in the dative case. This might point out to a trend where dative is falling out of favor as well. This is not to imply that accusative phrasing can be a substitute for genitive, like using “von” (of, which is dative) instead of the genitive case.

All of this point out to a trend of simplification in declension patterns of spoken German. Considering Old High German—the language German once—was even more complicated in that regard, the findings above don’t surprise me.

I might update this post with more findings or refutations of above conclusions later on, if future data shows that they are false.

This is a quick note on Subscription states on Stripe. Subscriptions are objects which track products with recurring payments. Stripe docs on Subscriptions are very comprehensive, but for some reason they don’t include a state diagram that shows the transitions between different states of a subscription. They do have one for Invoices, so maybe this post will inspire them to add one.

As of May 2024, the API has 8 values for Subscription.status:

• incomplete: This is the initial state of a subscription. It means that the subscription has been created but the first payment has not been made yet.
• incomplete_expired: The first payment was not made within 23 hours of creating the subscription.
• trialing: The subscription is in a trial period.
• active: The subscription is active and the customer is being billed according to the subscription’s billing schedule.
• past_due: The subscription has unpaid invoices.
• unpaid: The subscription has been canceled due to non-payment.
• canceled: The subscription has been canceled by the customer or due to non-payment.
• paused: The subscription is paused and will not renew.

At any given time, a Customer’s subscription can be in one of these states. The following diagram shows the transitions between these states.

stateDiagram
    classDef alive fill:#28a745,color:white,font-weight:bold,stroke-width:2px
    classDef dead fill:#dc3545,color:white,font-weight:bold,stroke-width:2px
    classDef suspended fill:#ffc107,color:#343a40,font-weight:bold,stroke-width:2px

    active:::alive
    trialing:::alive
    incomplete:::suspended
    past_due:::suspended
    unpaid:::suspended
    paused:::suspended
    canceled:::dead
    incomplete_expired:::dead

    [*] --> incomplete: Create Subscription
    trialing --> active: Trial ended, first<br>payment succeeded
    incomplete --> trialing: Started trial
    incomplete --> incomplete_expired: Payment not made<br>within 23 hours
    incomplete --> active: Payment<br>succeeded

    active --> past_due: Automatic payment<br>failed
    trialing --> past_due: Trial ended<br>payment failed
    past_due --> unpaid: Retry limit<br>reached
    past_due --> canceled: Retry limit reached<br>or subscription canceled
    past_due --> active: Payment<br>succeeded
    trialing --> paused: Trial ended without<br>default payment method
    paused --> active: First payment<br>made

    active --> unpaid: Automatic payment disabled,<br>manual intervention required
    unpaid --> active: Payment<br>succeeded
    unpaid --> canceled: Subscription<br>canceled
    active --> canceled: Subscription<br>canceled
    paused --> canceled: Subscription<br>canceled
    trialing --> canceled: Subscription<br>canceled

    incomplete_expired --> [*]
    canceled --> [*]

Stripe doesn’t comment on these states further and leaves their interpretation to the developer. This is probably because each company might interpret these states differently. For example, a user skipping a payment and becoming past_due might not warrant disabling a service for some companies, while others might want to disable services immediately. Stripe’s API is built to be agnostic of these decisions.

Regardless of how you interpret these 8 states, you will most likely end up generalizing them into 3 categories: ALIVE, SUSPENDED, and DEAD. The colors in the diagram above represent these categories:

• ALIVE: The subscription is active and payments are being made. States: active, trialing.
• SUSPENDED: The subscription is not active but can be reactivated. States: incomplete, past_due, unpaid, paused.
• DEAD: The subscription is not active and cannot be reactivated. Such subscriptions are effectively deleted. States: canceled, incomplete_expired.

While DEAD states are unambiguous, your company might differ in what is considered ALIVE and SUSPENDED. For example, you might consider past_due as ALIVE if you don’t want to disable services immediately after a payment failure.

If you collapse the 8 states into these categories, you get the following diagram:

stateDiagram
    direction TB;
    classDef alive fill:#28a745,color:white,font-weight:bold,stroke-width:2px
    classDef dead fill:#dc3545,color:white,font-weight:bold,stroke-width:2px
    classDef suspended fill:#ffc107,color:#343a40,font-weight:bold,stroke-width:2px

    ALIVE:::alive
    SUSPENDED:::suspended
    DEAD:::dead

    state ALIVE {
        active
        trialing
        trialing-->active
    }

    state DEAD {
        canceled
        incomplete_expired
    }

    state SUSPENDED {
        incomplete
        past_due
        unpaid
        paused
        past_due-->unpaid
    }

    [*] --> SUSPENDED: Create<br>Subscription
    SUSPENDED --> ALIVE: Payment succeeded<br>or trial started
    ALIVE --> SUSPENDED: Payment<br>failed
    SUSPENDED --> DEAD: Subscription canceled<br>or checkout expired
    ALIVE --> DEAD: Subscription<br>canceled
    DEAD --> [*]

The distinction is important, because Stripe doesn’t make it crystal clear what kind of subscriptions can come back from the dead and end up charging the customers multiple times. If you are not limiting the number of subscriptions per customer, this is something you should be aware of. Practically, this means that you block the customer from creating a new subscription if they already have an ALIVE or SUSPENDED subscription. DEAD subscriptions can be ignored.

Some languages are harder to learn compared to others. Difficulty can show up in different places. For example, English has a relatively easy grammar, but writing it can be challenging. Remember the first time you learned to write thorough, through, though, thought and tough.

Then take Chinese as an example. Its grammar is more simple than English—no verb conjugations, no tenses, no plurals, no articles. Its sounds and intonation are unusual for a westerner, but arguably not that difficult. The most difficult part might be the writing system, with thousands of characters that must be memorized before one can read and write fluently. A 7-year old primary schooler can learn to read and write English in 2 years, whereas for Chinese it takes at least 4 years. This is despite multiple simplifications in the Chinese writing system in the 20th century.

Now compare two adult workers of equal skill: a native Chinese emigrating to the US and learning English, versus a native American emigrating to China and learning Chinese. Which one will be able to start working and contributing to the economy faster? Extrapolating from the primary school example, the US adult could take at least twice as long to learn Chinese compared to their Chinese counterpart learning English—at least for reading and writing.

Time is money. It takes time to learn a language, and it takes more time to learn a “harder” one. Therefore, learning a complicated language has a cost. The cost of language complexity applies not only to native speakers, but also to foreign learners, which are the focus of this post:

The more complex the language of a country, the less attractive it is to foreign workers, skilled or otherwise.

This is because any worker who decides to move to a country with a more complex language will take longer to start contributing to the economy. This can be measured directly in terms of lost wages, taxes, and productivity.

Any such worker will also find it more difficult to integrate into the society, which can create indirect costs that are harder to measure, but are a burden nonetheless. For example, it could result in reduced upward mobility, decreased purchasing power, increased reliance on social services, and so on.

Here, I will focus on a cost that is one of the most tangible and easiest to quantify: wages that are lost due to language complexity. Doing that is relatively easy and gets my point across. I will then apply my calculation to a specific language, German, as a case study.

Wages lost while learning the local language

I will attempt a back-of-the-envelope calculation to estimate the total value of lost wages per year for all foreign workers in a country, while they are learning the local language. “Lost wages” mean the money that workers would have earned if they were working instead of learning the language, and the economic value that is not created as a result.

This is going to be a simplified model with many assumptions. For example, I assume that foreign workers do not know the local language when they arrive and spend a fixed amount of time per week learning the local language.

In the model, a given country receives $R$ foreign workers per year through migration. Each foreign worker takes $T$ years to learn the local language. Assuming that the rate of immigration $R$ stays constant (steady state), the number $N$ of foreign workers learning the local language at any given time is given by:

The average foreign worker dedicates $F$ hours per week to learning the local language. Most likely, only a percentage $D$ of $F$ will block actual work hours, for example in the form of an intensive language course, and the rest of the learning will take place during free time. If the average foreign worker works $W$ weeks per year, then the total number of hours per year that they spend learning the local language, that would otherwise be spent working, is given by:

Assuming that the average foreign worker earns $S$ units of currency per hour, the total value $C$ of lost wages per year and per foreign worker is given by:

We assume that for the given language, it takes $P$ hours of study to reach a certain level of proficiency necessary to communicate effectively in the workplace, say B2. Then we can calculate the number of years $T$ it takes to reach that level as:

Finally, the total value of lost wages per year for all foreign workers in a country is given by:

Put into words, the total value of lost wages per year for all foreign workers in a country is equal to the multiplication of the average hourly wage $S$, the percentage of time spent learning the language that displaces work $D$, the number of people immigrating per year $R$, and the number of hours of study required to reach a certain level of proficiency $P$.

If you could measure all these values accurately, you would have a good minimum estimate, a lower bound of the economic burden of teaching a language to foreign workers. The burden of complexity for any given language would then only be calculated by comparing its $P$ value to that of other languages.

Take Germany as an example. Given the values of $S$, $D$, $R$ for Germany, and the $P$ values for both German and English, you could calculate the money that the German economy is losing per year by German not being as easy to learn as English:

I attempt to calculate this below, with values I could find on the internet.

Case study: German

I live in Germany and I wrote this post with the German language in mind. Compared to other European languages like English or Spanish, German has certain features that makes it harder to learn. For example, it has a noun gender system where each noun can be one of three genders and each gender has to be inflected differently. These genders are random enough to cost a significant amount time while learning it as a second language.

Unfortunately, I haven’t found any authoritative data on how much harder German exactly is to learn, compared to other languages. It is not possible to exactly quantify language difficulty, because it not only depends on the language itself but also on the native language of the learner, their age, their motivation, and so on. Any data I present below are anecdotal and should be taken with a grain of salt.

That being said, the fact that German is harder to learn as a second language compared to, say, English, is self-evident to most people who have tried to learn both from the beginner level. So the data below is still useful, because it visually represents this difference in difficulty.

Hours required to reach B2 level

To begin with, Goethe Institut has put up the following values for German on the FAQ section of their website¹:

As a rough guideline, we estimate it will take the following amount of instruction to complete each language level:

• A1 : approx. 60-150 hours (80-200 TU*)
• A2 : approx. 150-260 hours (200-350 TU*)
• B1 : approx. 260-490 hours (350-650 TU*)
• B2 : approx. 450-600 hours (600-800 TU*)
• C1 : approx. 600-750 hours (800-1000 TU*)
• C2 : approx. 750+ hours (1000+ TU*)

*TU = Teaching Unit; a teaching unit consists of 45 minutes of instruction.

The Goethe Institut website does not cite the study where these numbers come from. My guess is that they just published the number of hours spent for each level from their official curriculum.

Another low-reliability source that I found is the Babbel for Business Blog². They have published the following values for German, English, Spanish, and French:

Note that the values for German are very close to those on the Goethe Institut website, so they were either taken from the same source, or the Babbel blog borrowed them from Goethe Institut. I could not trace a source for the values for English, Spanish, and French.

Plotting the lower bounds of the hours required to reach each CEFR level for German, English, Spanish, and French gives the following graph:

This picture intuitively makes sense. Spanish and English are easier compared to German and French, though I doubt Spanish is that much easier than the rest.

I then plot the lower-upper bound range of hours only for German and English, to make the difference more visible:

If we were to trust the blog post, we would have the following $P$ values for German and English: