GPT-5.6 Sol Memory Test: I Let It Remember 40 Things for 2 Weeks — Here's What Survived

How Sol's Memory Actually Works

Before I ran anything, I spent an evening poking at what memory even is in GPT-5.6 Sol. It's not a transcript vault — it's a continuously maintained summary file. As you chat, Sol decides what's worth keeping: names, preferences, decisions, project status. It rewrites this summary as new information arrives, and injects it into future conversations. That's why it can 'remember' things from three weeks ago in a brand-new chat: the summary gets loaded, not the original messages.

The key insight for testing: this is a lossy system by design. It's an editorial process, not a recording. Sol is the editor, and editors make judgment calls. My whole test was built around figuring out what those judgment calls look like in practice.

The 40-Item Test Design

I planted 40 distinct facts across 60 chats over 14 days, spread across five categories: personal details (8 items — my coffee preference, my dog's name, a birthday), project details (10 items — tech stack choices, deadlines, API keys location), preferences (8 items — response style, formatting habits, tools), one-off facts (8 items — a restaurant I mentioned once, a movie I recommended to a friend), and explicit 'remember this' requests (6 items where I literally said the words 'remember that X').

Each fact appeared in exactly one chat, in natural conversation, exactly as a real user would drop it. Then, on day 14, I opened a fresh chat and asked 40 direct questions — 'What's my dog's name?' — with zero context. I also ran a control: the same questions in a fresh incognito session with memory disabled, to confirm nothing was leaking from a shared global context. It wasn't.

What It Remembered (and What It Forgot)

Final score: 34 correct, 4 partial or mangled, 2 confidently wrong. The 6 explicit 'remember this' requests all survived — that's the headline feature and it works exactly as advertised. When you tell Sol to remember something, it does, even if it never comes up again.

Project details did best after that: 9 of 10 survived, including a random deadline ('launch by November 3rd') I mentioned once in passing. Preferences came in at 6 of 8 — it kept my 'always use bullet points' instruction but dropped a formatting preference I'd mentioned in a voice chat. Personal details were 6 of 8: it remembered my dog's name and my coffee order, but got my birthday wrong (off by a year — it had merged two separate mentions).

The damage report: the two confidently wrong answers were both one-off facts. I'd mentioned a restaurant called 'Bacaro' once, and when asked, Sol confidently told me it was a wine bar in Barcelona. It's a pasta place in London. Same name, same vibe, wrong city — it had clearly matched the name to its training knowledge and overrode what I'd actually said. That's the failure mode to worry about: not forgetting, but confident substitution.

Does Memory Carry Across Chats?

Yes, and this is where it gets both useful and unsettling. On day 9 I started a completely new chat about an unrelated topic — planning a hiking trip — and Sol greeted me with 'By the way, how did the database migration go?' It had carried the project status from a chat I'd had four days earlier. Impressive, but also a privacy reminder: memory is global across all your chats unless you scope it.

I tested the scoping controls too. Sol supports turning memory off per-chat (a little toggle in the chat menu) and per-item deletion in the Memory tab. With memory off in a chat, nothing from that chat was saved — I verified by checking the Memory tab afterward. With memory on, everything deemed important lands there, visible to you. I counted 23 auto-saved items from my test chats, and every single one was editable or deletable. Transparency is genuinely good here.

Memory vs Context Window: The Confusion

This is the single most common misconception I see in forums: people conflating memory with the 1M-token context window. They're different mechanisms with different costs. The context window is working memory for the current conversation — every message, file, and tool result in this chat. Memory is long-term storage that persists across chats.

The practical implication: a huge context window does not give you long-term memory, and memory does not give you mid-conversation recall beyond what's in context. In one test, I pasted a 400-page document into a chat, then started a new chat and asked about a detail from page 212. Sol didn't know it — the document was in the old chat's context, not in memory. The memory system had distilled it into a two-line summary ('user analyzed a large financial report'), which is useless for detail recall. If you need long documents available everywhere, that's what the context window stress test covers — and the answer is project files, not memory.

Settings and Controls You Should Change

After two weeks, here's the configuration I actually run with. First: keep memory on, but audit it weekly. The Memory tab takes thirty seconds to skim, and you'll be amazed what accumulates — I found Sol had saved my colleague's name, my laptop model, and a joke I made about my landlord. All useful, but I want to know what's in there.

Second: use the per-chat off switch for sensitive work. Anything involving financial data, medical details, or client information goes in a memory-off chat. The toggle is one click and it's verified working. Third: when something matters, say the words. My explicit-request items had a 100% survival rate. The difference between 'we're launching November 3rd' and 'remember, we're launching November 3rd' is the difference between a 90% and a 100% chance it's there next month.

Fourth: if you're doing professional work, check what memory injects into chats where it's relevant. Sol will volunteer remembered context ('I recall you're working with a tight deadline'), and while that's usually helpful, it can also derail a fresh perspective — you can say 'ignore memory for this chat' if you want a clean slate. For a deeper look at how Sol handles data in professional settings, the security team deployment guide is worth reading.

The Verdict

Memory in GPT-5.6 Sol is genuinely useful and honestly implemented. The 85% raw recall rate in my test understates its practical value, because the items that survived are the ones you actually care about — explicit requests, project details, preferences. The 15% failure rate is concentrated exactly where you'd expect: one-off facts that Sol tries to 'helpfully' reconcile with its training knowledge.

My recommendation: use it, but treat it like a good assistant's notebook, not a database. Say 'remember' for what matters, audit the Memory tab weekly, flip memory off for sensitive chats, and never assume a one-off mention will survive — if a fact is important, it's worth the extra four words to make it explicit. Used that way, memory moves Sol from 'smart chat' to 'assistant that actually knows you,' and that's a meaningful upgrade.