Metal memory on macOS

Mold separates installed shared RAM from the memory budget available to Metal. Device details in Studio (web, desktop and mobile), mold gpu list --json, and the devices API report the inference host's effective Metal capacity, allocation headroom, working-set recommendation and kernel-limit mode. Older hosts omit this optional telemetry.

The effective capacity is the smallest of Metal's working-set recommendation, a positive iogpu.wired_limit_mb setting, and installed RAM minus max(15% of RAM, 8 GiB). Allocation headroom also accounts for Mold's existing Metal allocations and live available host RAM. Allocated bytes include cached Metal buffers; they do not measure other applications' GPU allocations. The legacy shared-RAM totals remain hardware/system telemetry.

A kernel value of 0 means automatic, not zero usable memory. An unavailable supported probe prevents further Metal admission until it recovers; it does not silently substitute all installed RAM. If this optional kernel key is absent, a valid Metal recommendation can still supply a budget. CPU and CUDA behavior is unchanged.

Other GPU apps share this budget

Metal's working-set recommendation covers the whole Mac, and every GPU app draws on it. Mold's allocation figures do not include other applications, so a large render beside another GPU app, such as LM Studio with a model loaded, can take the machine's total over the budget. Metal then reports kIOGPUCommandBufferCallbackErrorOutOfMemory to whichever app is submitting work at that moment. That can be Mold or the other app.

Mold's own use stays close to what the model needs. Metal computes in 32-bit floats, so a half-precision checkpoint takes about twice its file size once loaded: a 12.3 GB Z-Image checkpoint peaks at about 25 GiB. Other models can peak above that size; FLUX.2 Klein 9B reached about 53 GiB on a 128 GiB Mac.

If either app reports this error, close the other app or unload its model, then render again. Mold releases its memory after a failed render, so you do not need to toggle the device first.

Inspect this machine

mold system metal-memory status
mold system metal-memory status --json

These commands always inspect the local machine, ignoring MOLD_HOST. They run before Mold config/database initialization. The allocation and headroom in this command belong to its own inspection process. To inspect a running server's allocations, use mold gpu list --json against that host instead.

Change the local machine-wide limit

sudo mold system metal-memory set 16384
sudo mold system metal-memory reset

The integer is in MiB (16384 MiB = 16 GiB). set accepts positive unsigned 32-bit integers and refuses values that consume Mold's host-memory floor; use reset for automatic mode. The explicit administration command requires root, writes only iogpu.wired_limit_mb, and verifies the kernel readback. Mold never invokes sudo internally, stores passwords, or elevates its server. There is no remote HTTP/MCP mutation endpoint for this setting.

This is a system-wide setting affecting other GPU applications. It does not reserve memory for Mold or guarantee a workload will fit. Mold refreshes its budget at admission, load and existing streaming boundaries; changing the limit does not asynchronously evict buffers from an active GPU command. A decrease is clamped immediately on subsequent samples, even if Metal's recommendation is stale. After an increase or reset, restart an idle inference process if its recommendation has not refreshed; Mold retains the lower observed budget.

Apply again after reboot

sudo mold system metal-memory set 16384 --persist
sudo mold system metal-memory reset --persist

Persistence installs only Mold's fixed root-owned boot policy at /Library/LaunchDaemons/io.utensils.mold.metal-memory.plist. Its one-shot job runs /usr/sbin/sysctl -w iogpu.wired_limit_mb=16384 at subsequent boots; it does not run Mold as root or continuously overwrite settings. Reset with --persist removes that owned policy and any loaded registration. Without --persist, set/reset changes the live value and reports any boot policy left in place.

Mold refuses foreign contents, symlinks and untrusted permissions. Concurrent Mold administration is serialized. A persistence failure attempts conditional rollback and reports partial state if another administrator has changed it; inspect status before retrying. Other tools can still change this system-wide key independently.

The kernel key is feature-detected and not a promised macOS compatibility API. Metal's working-set recommendation is a performance recommendation, distinct from MLX's process residency control. Boot persistence follows Apple's LaunchDaemon lifecycle.