Rust manages memory. Just not the memory I needed.

Two weeks ago I wrote that I hate Rust more than C++. Then I wrote another app in it. I do not have a good explanation for this.

The app is called Unstrung. It is a desktop app for roleplay chat with AI characters, and everything runs on your own PC. A local chat model writes the replies. A local picture model paints the scene when you ask. Nothing goes to a server. In six days it grew to 152 commits and about 28,000 lines of Rust.

Rust has one famous promise: memory safety. No dangling pointers, no double frees, no leaks that you did not ask for. That promise is real. This post is about the fine print.

The memory Rust cannot see

The borrow checker protects memory that my process owns. In Unstrung, the memory that matters is on the graphics card, and two other programs own it. The chat model lives inside the Ollama engine. The picture model lives inside sd-cli.exe, the command line tool of stable-diffusion.cpp.

To Rust, a 6 GB model is an HTTP request and a child process. The compiler is happy. The graphics card is full. Every rule about that memory, I had to write and check by hand, the same way I would in C.

Two models, one 8 GB card

My test laptop has an RTX 4060 Laptop GPU with 8 GB of VRAM. The recommended chat model is Gemma 4 E4B, a 6.15 GB download. The recommended picture model is Z-Image Turbo. It made a 1024 px picture in 67 seconds: 35 seconds to load, 9 steps of about 2.2 seconds each, and 11 seconds to decode. At its peak, the picture took 5.4 GB of the 8 GB.

They do not fit together. So every picture is a small moving job: take the chat model off the card, paint, then put the chat model back.

Unload means ask, then wait

There is no free() for a model in another process. Ollama's way to unload a model is a generate request with keep_alive set to 0. The engine answers at once. The memory is free later.

So the app asks the engine for its list of loaded models every 250 milliseconds, until the chat model is gone. After 15 seconds it gives up and tries the picture anyway. My first version wrote "chat model unloaded" in the log even when the wait ran out. The log was lying, and Rust compiled the lie without a word.

The picture engine gets two flags for bad days. --offload-to-cpu keeps the weights in RAM and moves them to the card as needed, so a model larger than the free VRAM still runs. --vae-tiling decodes the picture in tiles for the same reason. Slower, but it finishes.

Fearless concurrency, fearful VRAM

Rust also promises fearless concurrency: no data races. This was not a data race. It was worse, because every task did exactly what I told it to do.

A picture can use the profile pictures of the characters, so the same face comes back each time. If that fails, the app tries again from text tags only. The old flow was:

1. Unload the chat model. Paint with profile pictures. It fails.
2. Start a background task to load the chat model again.
3. Start the retry from tags, which unloads the chat model first.

Steps 2 and 3 ran at the same time. Sometimes the load landed after the unload, so the chat model sat on the card during the retry, and the retry failed too. The fallback that existed for the bad case failed in the bad case.

The fix was boring. One function frees the card before the first try and loads the chat model once, after the last try. The test now breaks a profile picture on purpose and expects a picture anyway. It failed before the change and passes after it.

The borrow checker knows who owns a byte. Nobody told it who owns the graphics card.

Load it once, at the right size

Ollama loads a model with a context size, num_ctx: how many tokens of the chat the model can see at once. The memory for that context sits on the card next to the model. Every load in my app, at start, after a model change and after a picture, sent no size. So the engine loaded the model at its default size, then loaded it again at the first reply, when the chat asked for 8192.

Every model in the catalog uses 8192 tokens. A comment in the catalog says it best: the model can take more, but VRAM sets the limit. The engine also runs with flash attention and stores that context memory at 8 bits instead of 16, so more of the chat fits in the same space. Once loaded, a chat model stays for 30 minutes.

The safe app reads the registry

To suggest a model that fits, the app must know how much VRAM the card has. Windows reports it through WMI, but WMI cuts the number at 4 GB, so my 8 GB card looks like a 4 GB card. The driver APIs would give the real number, and they need unsafe.

Unstrung's Cargo.toml says unsafe_code = "forbid". After 54 unsafe blocks in my PDF editor, I wanted none. So the app opens the display adapter keys in the Windows registry and reads HardwareInformation.qwMemorySize.

My memory-safe app finds out how much memory it has by reading the registry. To be fair, the 4 GB cut is a Windows problem, and the no-unsafe rule was my own choice. I am still annoyed at Rust for it. It is my blog.

The crash that compiled

One more, because it hurt twice. The app shell is Tauri. A Tauri command that is a plain fn runs on the main thread. The download code starts its work with tokio::spawn, which panics when no async runtime runs on that thread. And a panic in this app aborts it.

So the Download button on the Models screen closed the whole app, every time, with no message. The app has no console, so the panic went nowhere. The fix was one word, async. Two days later the same bug closed the app on "Turn on pictures".

Both versions compiled with no warning. Now every panic goes to app.log with a backtrace, and the model download returns an error instead of a panic when it runs outside the runtime. To be fair again, this is a tokio and Tauri rule, not a language rule. But the compiler that refuses so much happily let this through.

The other memory

The word "memory" has a second meaning in a chat app. The model sees 8192 tokens. In a long chat, the older messages fall out of that window, and the character forgets you.

So Unstrung keeps notes outside the window: what you came for, the story so far, and facts. A side pass of the same model writes them after a reply. You can read, edit or hide each note in the Memory panel. A hidden note is out of the prompt. The test for this is a chat of 200 messages that must still know facts from message 10. It passes.

This memory was the easy one. It is rows in SQLite, owned by my process, and Rust is very good at that.

So, Rust again?

Yes. None of the bugs in this post was a use-after-free or a leak. Every one of them was memory that Rust could not see: VRAM in another process, a load that raced an unload, a context size that nobody sent, a 4 GB limit in Windows, and a thread with no runtime on it.

That is the honest version. Rust keeps the memory it can see clean. With local AI, most of the memory is somewhere it cannot see. You still manage that by hand, with logs, waits and tests, like it is 1999.

Unstrung is open source, and all eight build phases are done except code signing. There is no release to download yet. The code is on GitHub. If you want a local model that does more than chat, read how my desktop AI companion learned to say no.