0x0 Memory

← back 0x0 Memory Jul 19, 2026

The new first-root recovery path could build both native 0x0 compiler artifacts without generating C. The original monolithic seed command had peaked at 12,667,784 KiB of RSS. Splitting planning from emission brought the measured compiler builder down to 3,345,556 KiB.

That was a real improvement. It also meant a 714,488-byte executable still needed more than three GiB of resident memory to recover itself.

The caveat was doing useful work. It was telling me that the bootstrap proof was correct, but its memory model was not finished.

So I kept going.

The final C-free builder recovery now peaks at 344,860 KiB. The native compiler succession runs between 70,232 and 77,984 KiB per generation. The accepted native Linux memory soak ran for 86,400 seconds, completed one million configured leak iterations and 577,424 soak batches, ended at 10,380 KiB peak RSS, and returned every tracked live counter to zero.

The useful part is not only that the numbers are smaller.

The useful part is that 0x0 now has an answer to a more important question:

Which memory is allowed to live at this point in the program, and who is responsible for ending its lifetime?

That question changed the seed, the compiler, the direct-ELF runtime, the test strategy, and the release contract.

It is difficult to optimize memory when every number means something different.

One tool reports the largest child process. Another reports the shell wrapper. One workload includes parsing and emission. Another begins after a cached compiler image already exists. An application has a small RSS because it never releases anything, but also never runs long enough for that to matter.

Those results can all be numerically true and still describe different systems.

The first phase of the work was making the measurements difficult to lie with.

0x0 now freezes workload identities with source hashes, commands, input rules, output assertions, owners, and budget classes. Memory telemetry uses a versioned tab-separated envelope with run, identity, phase, metric, and terminal records.

The validator rejects:

unknown schema versions or metrics;
missing or duplicated phases;
non-increasing event sequences;
negative unsigned counters;
decreasing cumulative counters;
conflicting source or compiler identities;
missing terminal status.

The budgets are separate too:

C-free seed recovery          512 MiB process-group RSS
normal compiler succession    256 MiB per process
explicit heavy compiler lane    4 GiB process-group RSS
native Linux applications      64 MiB RSS
completed native owner scope     0 live-byte delta
completed resource scope         0 open-resource delta

There is no combined score. A small application cannot hide a leaking compiler. A passing compiler cannot hide a file descriptor leak. A bounded seed cannot excuse an unbounded native request loop.

This sounds bureaucratic until a measurement improves suspiciously. Then a source hash, phase identity, and independent budget are much cheaper than guessing what changed.

The old seed recovery workers repeatedly loaded and normalized the compiler source. More importantly, the runtime did not have a clean distinction between the compiler state that had to survive and the scratch that could disappear.

The new path first constructs one immutable normalized compiler image.

compiler/main.0x0
        |
        v
parse and normalize once
        |
        v
0X0NIMG2 immutable image
        |
        +----> layout measurement worker
        |
        +----> function emission worker

The image contains a version, a complete build identity, a module identity, a canonical dense function-name table, and normalized function forms in the same order. It deliberately excludes raw source, lexer scratch, parser stacks, and output buffers.

Its identity includes the source hash, compiler-image hash, target, ABI, and representation version. Reusing an image after any of those change is an error, not a cache hit.

The binary format is deterministic and pointer-free. It uses little-endian integers, a bounded string table, explicit tags, recursive pairs, and dense vectors. The loader rejects unknown tags, truncated values, invalid intern indexes, duplicate strings, excessive nesting, and trailing bytes.

Publication is transactional enough for the bootstrap boundary: write a temporary file, flush it, fsync it, close it, and rename it over the final path. A sidecar records the identity, size, and content hash. Every planning and emission phase validates the manifest and image again.

The result is not merely a cache. It is a lifetime boundary. Everything inside the image is immutable and intentionally retained. Everything used to produce it is eligible to die.

The first bounded recovery implementation reclaimed memory by terminating a worker process after a batch. It worked, but process exit was doing the job the runtime should eventually understand.

The new seed has resettable scratch arenas.

One persistent measurement worker and one persistent emission worker load the compiler environment and normalized image once. Each request creates an arena checkpoint, performs one bounded unit of work, publishes only its stable result, and resets to the checkpoint.

immutable image and compiler environment
                  |
                  v
          checkpoint scratch
                  |
                  v
       measure or emit one function
                  |
                  v
          publish stable result
                  |
                  v
             reset scratch

The scratch arena reserves 512 MiB of virtual address space with MAP_NORESERVE; it does not make 512 MiB resident. Pages are committed on demand. Reset discards whole scratch pages with madvise(MADV_DONTNEED), restores counters and the used offset, and invalidates the checkpoint.

Reset is also a safety operation. It fails if the checkpoint is stale, nested, over capacity, or owns a resource that was opened after the mark and never closed. The image serializer rejects any value whose address belongs to the active scratch arena, including scratch children hidden inside a pair.

That prevents a temporary value from quietly escaping into the supposedly immutable compiler image.

The full compiler currently has 910 functions. Recovery resets measurement scratch once per function and emission scratch 911 times: once for each function and once for the ABI note.

Process isolation remains available as a diagnostic mode. It is no longer the production reclamation mechanism.

Lifetime fixes stop old work from accumulating. Representation fixes reduce the cost of the state that really must remain alive.

The seed's generic value cell used to occupy 56 bytes. It now occupies 24. The common tag and two payload words are enough for integers, text and symbol references, pairs, closures, builtins, and dense vectors. Function metadata moved into a separate descriptor instead of inflating every value.

That is a 57.14 percent reduction in cell allocator bytes.

Repeated text and symbols are interned in deterministic first-use order. A hash match still compares the complete bytes, so collisions cannot make two different names alias. Textual token and node kinds become stable integer IDs. Line and column values are packed into checked 16-bit fields. Indexed top-level function forms use dense vectors instead of a cons cell for every entry.

The point was not to turn the language into a pile of clever bit tricks.

The point was to spend rich tree structures where the compiler has trees, and dense indexed structures where the compiler has tables.

The next accidental lifetime was the artifact itself.

A compiler that constructs an entire ELF image as hexadecimal text may hold the AST, layout, machine bytes, hex encoding, concatenated output, and decoded file representation at the same time. The final file may be small while the path to it is extremely expensive.

Seed recovery now uses a bounded 65,536-byte binary buffer. Exact writes retry interruptions, advance after short writes, reject zero progress, and fail on disk-full or invalid-path errors. Full buffers are flushed and reused.

The recovery protocol is deliberately boring:

write ELF header and startup
for each canonical function:
    emit one bounded chunk
    append its bytes
    reset emission scratch
append ABI note
verify exact planned file size
chmod 0755

The partial file starts as mode 0600. It becomes executable only after its size matches the frozen plan and every append succeeds. chmod is still the small commit operation from the previous design; the difference is that the bytes no longer travel through one giant retained text value.

The compiler's current x86-64 function emitter still creates a hexadecimal encoding internally. That value is bounded to one function and dies at the next reset. Hex remains useful for diagnostics. It is no longer the artifact transport.

The normal native compiler needed the same treatment.

A compiler pipeline has an obvious sequence:

source -> tokens -> normalized forms -> validated forms
       -> optimized forms -> layout -> machine code -> artifact

It is easy to implement that sequence while retaining every arrow's input and output until process exit.

0x0 now has an explicit retention graph. Raw source, tokens, validation worklists, type and effect constraints, optimizer candidates, rendered OISA records, and generated function chunks are phase-local. The normalized forms and final native address table are the main compilation-lifetime structures.

The frontend copies strings, symbols, and packed spans into normalized nodes, then releases source and lexer storage. Validation keeps the validated module, not its lookup tables and branch joins. Optimizer passes run in scratch and copy only the accepted form graph into persistent storage. Layout retains names, sizes, and addresses. Emission writes one function and resets.

Diagnostics still keep line and column information because normalized nodes own compact spans. Releasing the parser does not mean releasing the ability to explain an error.

Compiler dumps became observations rather than phases. Optimizer variants run sequentially. Each process constructs one requested representation, streams it, records a compact comparison summary, and exits before the next variant. There is no in-process list containing O0, every intermediate pass, O1, SSA, LIR, and disassembly at once.

The historical interpreter multi-dump peak was 7,852,696 KiB. The accepted sequential native optimizer lane measured 90,916 KiB. That is not one magical allocator optimization; it is the result of changing execution, retention, and serialization together.

Normal native and OISA compiler output is streamed too. Writers maintain an online byte count and FNV-1a-64 digest while writing, so callers can validate the result without retaining or rereading a complete output value.

Three compiler generations now reproduce the same artifact at:

Generation	Peak RSS	Parity
1	77,984 KiB	trusted parent, byte-identical
2	70,232 KiB	generation 1, byte-identical
3	77,984 KiB	generation 2, byte-identical

That is the ordinary self-host path, not the expensive assembly-seed recovery path.

Compiler memory was only half of the roadmap.

I only care about 0x0 as a Linux application platform right now. For that target, process-lifetime allocation is not enough. A server cannot call mmap, forget every result, and claim success because the kernel cleans up when the process eventually dies.

The direct ELF backend now implements a managed native profile with:

unique ownership and moves;
lexical drops;
bounded regions with escape checks;
precise non-atomic reference counting for shared acyclic values;
request and task arena resets;
destructors for files, sockets, and supervised child processes.

This code is emitted directly into the Linux x86-64 executable. It does not call C, Python, a tracing collector, or a hidden helper runtime.

A unique value has one owner. Moving it transfers that ownership. The compiler inserts terminal drops in reverse lexical order and rejects double drops, use-after-move, and ownership-class mismatches.

A region owns a bounded collection of allocations. Releasing the region releases the collection together. Returning a value whose region ends at that return is rejected before native layout. Cleanup is emitted on normal exits and panic paths.

A genuinely shared value uses precise strong reference counting. Retains make aliases; releases destroy the value when the count reaches zero. The current profile is single-threaded, so these counters are intentionally non-atomic. Shared graphs are immutable and acyclic. Explicit cycles are rejected; cyclic working state belongs in a bounded region.

Each unique, shared, or region owner receives a 16 MiB demand-mapped address range. Again, virtual capacity is not resident memory. Only touched pages consume RSS. Text and list allocations made under that owner use the range, and the terminal drop or reset reclaims the complete composite with munmap(2).

Generation-checked owner slots catch stale handles even when an alias crosses a function boundary.

Memory safety is not complete if the heap returns to zero while the process leaks a socket on every request.

Files, sockets, and supervised child processes are ownership-bearing native values. File and socket destruction calls close(2). Child destruction sends SIGKILL and waits with wait4(2), preventing zombies and orphans. Resources registered inside an owner are destroyed before its mapping is released.

The runtime publishes logical counters for live slots, live bytes, regions, shared values, resources, drops, request resets, task resets, mapped spans, and mapped bytes.

RSS is useful evidence. It is not the only evidence.

A flat RSS curve can hide a logical leak inside an allocator that reuses pages. Zero logical counters can hide an allocator that never gives pages back to the kernel. The release gate requires both the ownership counters and the resident-memory plateau.

It also includes negative controls. One test deliberately suppresses inferred cleanup. The leak detector must catch that program. A leak suite that cannot detect an intentional leak has only proved that its happy path is quiet.

Fast loops catch obvious imbalance. They do not catch every periodic cleanup bug, counter wrap, child-process leak, or slow RSS staircase.

The final release operation therefore had an inconvenient acceptance test:

MEMOPT_RELEASE_SOAK=1 \
MEMOPT_LEAK_ITERATIONS=1000000 \
MEMOPT_SOAK_SECONDS=86400 \
make native-memory-soak-check

The accepted run recorded:

Measurement	Result
Duration	86,400 seconds
Configured leak iterations	1,000,000
Soak batches	577,424
Iterations per soak child	16
Warm peak RSS	10,104 KiB
Final peak RSS	10,380 KiB
Live allocations	0
Live resources	0
Mapped spans	0

The runner writes accepted evidence only after the full duration. An interrupted or failed run publishes nothing. The evidence records the compiler and fixture hashes, so changing the compiler invalidates the result.

A one-second test cannot be renamed into a release soak. A fixture cannot stand in for wall-clock evidence. The final cross-domain validator fails closed if the durable run is missing, too short, stale, or attached to another compiler revision.

After the run returned native memory soak check ok, the release decision could finally change to pass.

The seed-recovery progression now looks like this:

Recovery design	Peak RSS
Monolithic seed ELF builder	12,667,784 KiB
Bounded process workers	3,345,556 KiB
Image + arenas + compact values + binary output	344,860 KiB

The final full recoveries were:

Artifact	Peak RSS	Limit	Parity
ELF compiler builder	343,524 KiB	524,288 KiB	stage 2/stage 3 identical
ELF OISA compiler	343,344 KiB	524,288 KiB	trusted OISA identical
Builder ratchet rerun	344,860 KiB	524,288 KiB	seed builder identical

The 512 MiB release ceiling leaves room above the measured host without pretending every machine has identical RSS behavior.

More importantly, the explanation changed.

The old answer was, "the seed VM retains a lot, so give it four GiB."

The new answer names the retained image, scratch arenas, value layout, output buffer, compiler phase graph, native owners, resource destructors, and release evidence. Each has an owner, a boundary, and a gate.

What This Does Not Claim🔗

The current production statement is intentionally narrow:

0x0 has a bounded memory and lifetime profile for direct ELF Linux x86-64 programs using Unit, Bool, I64, Text, and List values.

I started this work because 3.2 GiB felt like an embarrassing footnote on a successful self-host.

The result is better: memory is no longer an unnamed pile owned by the process. The compiler knows which state must persist. Workers know what can be reset. Native values know who drops them. Resources have destructors. The release knows what was measured.

And after twenty-four hours, everything that was supposed to be dead was still dead.