My AI agent's 'second brain': a memory that survives every session

I work all day with an AI agent (Claude Code, in my case) and for months I lived the same silly friction: every session started from zero. I'd re-explain who I was, how the project was laid out, what we'd decided last week. The AI is brilliant inside one conversation and amnesiac between conversations. Close the window, everything resets.

One day I searched Google for "second brain", hoping to find a fix. I only found the usual: the PARA method, Notion, Obsidian, a second brain for a human to take notes. Nothing about giving memory to an agent. So I stopped searching and built my own. This post is the pattern that survived after cleaning house about three times, generalized, for you to adapt. There's a template to download at the end.

The problem: a context window is not memory

The confusion starts here, so let's separate the two:

• Context window = the short-term memory of one conversation. It's huge today, but volatile: session ends, it evaporates.
• Persistent memory = knowledge that lives outside the conversation, in files. The agent reads it at the start of each session and writes to it during the work. It survives.

Pasting the context at the top of every conversation "solves" it, the way writing a phone number on your palm solves it. It doesn't scale, and you forget half of it. What scales is giving the agent a fixed, versioned place it reads from and writes to on its own.

The 3-layer architecture

After trial and error, the format that proved robust has three layers, from general to specific:

The trick with layer 3 is economic: the index is paid for in tokens every session. If it's a paragraph per fact, you burn context for nothing. One line per fact keeps the map light and leaves the detail for when (and if) it's needed.

Durability: version it, but by whitelist

Good memory is memory that doesn't vanish when your drive dies and that follows you across machines. The answer is old and boring: git. You commit the brain to a private repo and mirror it to the cloud. New machine? git clone and the agent already knows everything.

But there's a security trap almost everyone falls into. If you version by blacklist (ignore .env, ignore *.key...), forgetting one pattern is enough to leak a secret into the repo. The inversion saves your skin: ignore everything by default and allow only the knowledge.

# ❌ blacklist — dangerous: miss one pattern, it leaks
# .env
# *.key

# ✅ whitelist — ignore EVERYTHING, allow only knowledge
*
!*/
!**/*.md
!.gitignore

With a whitelist, a file with a secret created by mistake simply doesn't get in. Git never even looks at it. Credentials stay in the .env (out of version control). And if you work with client data, raise a hard wall: the client's data and code live only on their infrastructure; at most the distilled learning enters your personal brain, never the raw data.

The lessons Google won't tell you

• Memory is a photo of the past, not live state. What's written was true when it was written. Before acting on a memory, the AI confirms it against the code/reality. Otherwise it confidently repeats a fact that already changed.
• One fact per file. Updating means opening a file, not hunting for a paragraph inside a giant document. And you can delete what turned out wrong without fear.
• Condense what's finished. A completed story doesn't need its 12 steps. It becomes a one-line summary ("done: X solved on such date"). The brain stays lean.
• The index is sacred. It's the only file the AI always reads. Keeping it short and honest is what makes the whole system work without burning context.

The template to start today

So you don't build from scratch, I left a sanitized skeleton, the same format I use, with nothing specific to me. Download it, rename the placeholders, and you have a working brain in 10 minutes.

It's plain text. Open it, read it, adapt it. It's also in my Library, alongside other free resources.

In short

An AI agent doesn't have to be amnesiac. You can give it a second brain: three layers (global rules · per-project memory · index), versioned with git using a whitelist so no secret leaks, portable across machines. It's not magic. It's well-organized text files and a simple discipline to keep. But the day-to-day difference is huge: you stop re-explaining and start building from where you left off.

Related reading: What AI skills are. Memory is half of what makes an agent useful; skills are the other half (what it knows how to do). Both are part of the guide How to work with an AI agent.

I want a custom system for my business

That's what I do: turn your business problem into software, with applied AI, automation and an obsession with security. If that fits your case, let's talk.