The person who knows best at using AI is building a company for it

Have you ever had this experience--

Let AI help you do a slightly longer job. At first, it performed very well, with clear thinking and response. You felt relieved and felt that this thing was reliable, so you let it go and stopped staring at it.

Then he looked back and was dumbfounded. When it comes to the end, it will forget the requirements you have repeatedly emphasized before, and even correct the things you have confirmed and corrected long ago, and return them to the wrong ones. It also looked confident, as if everything was under control.

I have been cheated more than once myself. What is most angry is not that it did wrong, but that it fell apart when you trusted it the most. The more amazing it was at the beginning, the more assured you, and the more caught you off guard when it broke out behind.

It's not your fault that you can't use it, nor is it that a certain AI can't do it. Whether you usually use ChatGPT, Claude, bean buns, DeepSeek, or various domestic Agents, this problem is the same. It is a tough problem for this generation of AI-especially Agents who can work continuously on their own-that cannot be corrected.

Today, I want to explain it thoroughly: why it "broke out", how the industry has been trying so hard to treat it in the past two years, can it be cured, and what I finally figured out-those who are the most good at using AI are not doing "training a smarter brain" at all, but building a "company" for AI.

It is not "memorizing" at all, it is rereading compressed notes one time

First, we need to understand a counter-intuitive thing: AI actually has no "memory."

You chat with it, or ask an Agent to work continuously, and it seems like it always "remembers" what you said before. But that's not the case at the bottom.

Every time it takes a call and takes the next step, the system will repackage "everything so far"-your instructions, its replies, the files it has read, the results returned by the tool-, tuck it in and show the model over. The model itself remembers nothing, it just re-reads the dossier every time and then goes on.

Here comes the problem: There is an upper limit to the amount of this file that can be stuffed (this upper limit is called the context window). The longer the work is done, the thicker the file will be, and sooner or later the limit will be exceeded. What should I do if I exceed it?

The mainstream practice is called compaction (context compression): the system packs the big lump in front into a summary, throws away the original, leaves the summary and adds it to the most recent rounds, makes up a thinner new file, and then continues to work.

!

You appreciate this process. You think you and AI are a continuous conversation and it has always been with you; in fact, at the bottom level, the file it read is compressed and replaced round after round. The "continuity" and "memory" you see are illusions created by engineering. And every time you press it, it is a lossy compression-if you press the thick one into thin one, you will inevitably lose something. What the abstract retains is what it "feels important", and what it loses may be the detail that you care about most: a specific constraint, a file path, and the sentence "Don't move here."

This is the root of its "fragment". It's not that it is stupid, but that the notes it re-reads every time, the more it reads, the more it becomes, the more blurred it becomes.

Programming Agents like Claude Code simply put this on the table-it has a /compact command, which automatically triggers compression when the window is almost full, and you can see with your own eyes its "reorganization of memories." Convenience is convenient, but it is up to it to decide what to keep and throw away when compressing, and it is not always what you think.

The larger the ## window, the more it looks like a placebo

Then someone said: Wouldn't it be enough to make the window bigger? The more files you can pack, won't you have to compress them?

This has been the case for manufacturers in the past two years, with context windows rolling from tens of thousands to millions. It sounds beautiful. But here is a heart-breaking fact: The nominal window size is far from the part that can really be used well.

Measured in the industry, only about half of the points in the nominal window can be used reliably. What is even more troublesome is a phenomenon called "lost in the middle"-information is placed at the beginning and end of the file, and it can see clearly; when placed in the middle, it often ignores it.

How obvious is it? Take the test of "looking for a specified information in a pile of content", the latest cutting-edge models can achieve 96% and 99% on a million-level length, which looks perfect. But replacing it with "Find 8 messages at the same time"-which is close to real use-and the score immediately drops to 74%, 89%. The more and more complex a thing is, the more it leaks out.

!

In a more chilling direction: Some studies have found that after cutting-edge models read hundreds of thousands of tokens, the probability of missing judgments on dangerous operations that they should be alert to will increase exponentially. It's not that it didn't read it, but that there was too much content, and it was really "invisible".

So the popular idea that "the window is big enough to remember everything" was very popular in the past two years, but now it is basically bankrupt. Large windows are useful, but it is not an antidote, it is more like a placebo-making you think the problem is solved, but in fact it just pushes it back. What's more, the bigger the window, the slower the more expensive it is: the content doubles, and the response time almost doubles.

AI will "panic" and will also "show off"

The mechanics are a little dry. I tell you two particularly picturesque rollover scenes, and you immediately understand what this "fragment" looks like.

The first is called "panic."

When the files are almost full and the space is almost empty, the model will enter a state similar to "contextual anxiety"-it seems to realize that it is running out of space, so it will rush to finish it : If you should have done it step by step, just do it hastily, save it wherever you can, and give you the result. It's like the exam time is approaching, and the big questions behind are scribbled and handed in.

The second type, on the contrary, is called "showing off".

You give it a big job, but it doesn't know how to tear it apart. It insists on eating it into a fat person, wanting to finish the whole thing in one round. As a result, it made changes one by one in the limited files, creating a mess of random changes in which there were fights with each other without any records. You want to look back and see what it did, but you can't make sense.

!

One is cowardly and the other is reckless. But have you noticed that these two types of rollover have the same root- There is only so much "working memory" that it can hold firmly. When the space is tight, it panic, and the work is too big and beyond what it can hold in one go. Forgetfulness is the common theme of these two types of rollover.

masters no longer "order" AI, but build systems for it

Speaking of this, you may think: Can this AI still be used?

Yes. Moreover, in the past two years, the industry has not sat back and waited for the model to become stronger. Instead, it has invented a complete set of projects specifically to deal with this amnesia. The evolution of this project itself is quite illustrative:

The first thing everyone argued was prompt engineering; later it was context engineering; now the hottest one is called harness engineering.

Hardness can be understood as the "skeleton" outside the model-cycles, tools, context management, memory, guardrails, all inside. There is a particularly concise formula in the circle: Agent = Model + Harness. The model is the brain that will fragment, and the harness is the system that is wrapped around so that it can work stably.

!

How does this system deal with amnesia? Here are a few things I think are smart:

• Only when you need it : You don't stuff all tool descriptions, documents, and rules into the file at one time, but only feed the information needed at that step at which step is used, saving valuable space.
• Good steel is used on the cutting edge : Use expensive strong models for planning and final acceptance of such heavy tasks; use cheap and fast models for errands in the middle. In a file, money and attention are spent on key points.
• Stop it and finish it hastily : As soon as it wants to be lazy and hand in the paper in advance, the system will re-feed your original request to force it to get back on track.
• Dividing work : The dirty and hard work is distributed to the "sub-agents". Each person uses his own clean file to do it, and hands over the results after completion without polluting each other.
• Only appends, does not overwrite : Never go back and change previous records (once you change it will destroy the cache and make it easy to mess), only appends later.

What best illustrates the wind direction is a sentence publicly said by Claude Code manager Boris Cherny-he said that he no longer prompts AI personally, but writes a bunch of "loops" and lets the loops drive AI by themselves., thinking about what to do next,"My job is to write loops".

This is the popular loop engineering now: you no longer order AI every word, but design a process that you can turn around on your own-discover tasks, dispatch, accept, record, decide the next step, and let this process promote AI. There is a saying that I particularly agree with: A cycle where nothing can "pick" it is equivalent to letting the AI agree with itself. Therefore, a good cycle must be filled with "picking" links such as testing, inspection, and acceptance.

But has this project really cured the disease?

This is the question I want to talk to you most, because there are too many voices blowing about this project, and some cold water has to be poured on it.

Let's start with useful evidence, which is indeed hard:

Put the same model-such as the same Claude-into different skeletons and run the same set of programming tests, and the score will differ by 16 points (one 93, one 77). The model is exactly the same, but the difference is all due to the outside system. There is also a team that did not change a word of the model. Just optimizing the skeleton, they moved their Agent from 30th to 5th on a certain list. There is even a counter-intuitive finding: Setting the model to the maximum "thinking budget" is not as effective as moderate-appropriate constraints are more effective than letting it think hard.

These all show that: The model is becoming more and more like a standard part, and what opens up the gap is the project outside.

But-has it cured the problem? My judgment is: No.

!

Several negative voices that cannot be ignored: An independent evaluation organization has found that the advantages of these carefully designed Agent tools in many tasks are actually "within the error range" and are not so impressive. The more critical point is-- Long context does not equal long task . Currently, the strongest batch of Agents can get more than 70% on standard programming tests, but when it comes to tests that test long tasks (which require continuous changes to dozens of files), they can drop to 25%. This shows that the project can help it "find and organize the materials well," but its foundation for continuous reasoning on the long chain is still not strong enough. If the feed is right, it may still think wrong.

Others-including researchers within OpenAI-believe that the plug-in project may be temporary: when the model becomes stronger, the outer skeleton will slowly be absorbed into the model and no longer needed.

So my conclusion is: This project is really useful, but it is a hedge, not a cure. It covers most of the problems of AI forgetting, panic, and bravado, allowing Agents to work longer and longer. But the ceiling is still there-in the model's own long-task reasoning capabilities. There are boundaries where the project can be carried.

In the final analysis, ## is a replica of a company for AI

When I unfolded this set of things, I later tasted it and realized a taste--

These tricks include distributing tasks to sub-agents, using external documents to make handover records, letting AI review past experiences, passing test acceptance at every step, and not letting one context carry too much... Isn't this what a company manages people?

People are also forgetful. People will also make mistakes when they are about to get off work, and they will also overestimate themselves to take on too many jobs at one go. How is human organization solved? I never expect anyone to remember everything-it depends on division of labor, writing documents and handover, reviewing, processes and checklists, and not exhausting a person.

!

You see, whether harness or loop engineering is essentially building a "working system" for AI, allowing a group of "employees"(Agents) with limited memories and occasionally confused to rely on the system to collaborate. Reliable results.

After thinking this through, one thing becomes clear: "Let AI remember everything" has been a misunderstanding from the beginning. We should not pursue a super brain that can remember everything, just as a good company does not rely on a superman who can remember everything, but on good systems. The evolution direction of AI collaboration is not a bigger brain, but a better system.

So how should you use it

When it comes to yourself, there are two kinds of people who say it.

If you are an ordinary user and treat AI as a "talented colleague who can fragment", you are right:

• The key requirements should not be stated only once . Important constraints and preferences should be repeated if necessary. Don't expect them to always remember.
• Keep your own record . If you really need it, you can save a copy of it yourself and post it again when you need it. Don't bet that it isn't lost.
• Take apart long-term jobs and do them . Don't give it a huge task at a time, break it into several paragraphs, and you will check and then move forward after completing each paragraph.
• It reopens when it starts to float . Find that it doesn't follow the words and starts to forget things. Don't talk hard in that conversation, open a new one and feed the key background again.

If you are a developer and working with an Agent:

• External memory . Write critical information to external files or databases before compression occurs, and don't put it all in the context of the model.
• Write cycle, plug acceptance . Don't feed hands, design a process that can rotate, and test and inspect this "pricking" mechanism at every link.
• Removal and isolation . Use sub-agents to isolate the context to do dirty work, and keep the main process clean.
• Change indicator . Don't measure an Agent by "how long it can continue to work", use "how much useful work is produced for every dollar spent."

Last

In this round of AI Agent competition, I increasingly feel that what opens the gap is not whose model is stronger and whose window is bigger, but who knows better how to manage its forgetfulness-who gave it the "working system" is stronger.

So next time, don't ask "How long can this AI continue to work?" What should be asked is: Can this working system accommodate it?

The model will become stronger from generation to generation, that's for sure. But "forgetfulness in long tasks" is the real boundary of this generation of AI and cannot be circumvented in the short term. Only people who understand this boundary-who know where the project can be carried out, where it has to be watched by people, and where it can only wait for the model to improve-can they use AI stably and effectively.

After all, the reliability of today's AI does not grow in the model. It grows in the system you built for it.