macOS 27.0 (26A428): Core ML multifunction ML Program is recognized by MLModelAsset but fails to load

Update after further testing on the same build: one correction, and a bisection down to a ~1 MB reproducer

1. Correction — the failure IS compute-unit dependent. My original post said otherwise. Measured on the same compiled .mlmodelc:

Only .cpuAndNeuralEngine fails, and only infer. prefill, in the same package, specializes for the Neural Engine successfully. My app loads infer first with .cpuAndNeuralEngine, which is why it initially looked like every named function failed.

2. It reproduces with a single-function model. Splitting the package down to infer alone (coremltools MultiFunctionDescriptor.remove_function("prefill")) gives the identical E5RT failure under .cpuAndNeuralEngine; the same package loads and predicts correctly with .cpuOnly and .all. So both functionName must be nil and doesn't support the multi-function description syntax look like secondary symptoms: the ANE specialization fails, and Core ML falls back to a loader that cannot handle the description format. That is also why functionName = nil is not a workaround.

3. It reproduces at ~1 MB with no quantization. A synthetic 8-layer / hidden-256 / ctx-256 / fp16 model, exported two ways through the same conversion code:

Bisection on this pair ruled out, each by direct experiment: quantization (fp16 and 4-bit palettized both fail); scale (8 layers is enough; layer counts 1/8/16/32 and context 256/1024 do not change the outcome); GQA (num_key_value_heads == num_attention_heads still fails); the update_mask input (removed from the function signature, from the operation that consumes it, and from the model description, then recompiled successfully — still fails); and the in-graph one-hot mask construction (supplying the mask from an input instead of building it with arange/equal still fails).

Separately, a second decode export of the same architecture — in which the state write is instead an O(1) slab slice_update at a shape-derived offset, with no full-length blend anywhere — also fails with the identical E5RT (11). So more than one decode graph shape breaks ANE specialization here; the blend below is the difference I could isolate in a controlled pair, not necessarily the only trigger.

4. In the controlled pair, one structural difference remains. Differencing the operation sets of the failing decode graph and the passing prefill graph: the decode graph performs an element-wise read-modify-write over the full KV state (old*(1-mask) + new*mask — one sub, plus 32 extra mul and 32 extra add per 8 layers). The prefill graph never does this; it writes the state with tile plus a constant-offset slice_update. Across all five graphs I built, sub appears in every failure and in no success.

5. Log sequence (identical each time, 10 ms apart):

begin load, computeUnits = .cpuAndNeuralEngine
[com.apple.e5rt:e5rt]      E5RT encountered an STL exception. msg = <private>.
[com.apple.coreml:coreml]  E5RT: <private> (11)
[com.apple.coreml:coreml]  `MLModelConfiguration`'s `.functionName` property must be `nil` unless the model type is ML Program.

The E5RT message body is redacted as <private>. When the failing function happens to be the model's default function — so no functionName is involved — Core ML surfaces Failed to build the model execution plan using a model architecture file '<private>' with error code: -14 instead, which looks like the real underlying failure. It is deterministic and reproduces with an empty ANE specialization cache; the infer attempt leaves no compiled artifact behind while the prefill attempt in the same session leaves a complete ANE program. xcrun coremlcompiler compile succeeds on every package.

6. This is a regression, and I have no ANE workaround. The same graph compiled for, and ran on, the Neural Engine on macOS 26 — measured at 46.8 tok/s with computeUnits = .cpuAndNeuralEngine. The only workaround I found is .all, which loads but builds an MPS graph: decode measured 37.9 ms/step, indistinguishable from .cpuAndGPU.

Questions

1. Is this a known macOS 27 regression in ANE specialization of ML Program functions? It reproduces with a single-function model, so it does not appear to involve the multifunction loading path.
2. What does E5RT (11) indicate here, and is there any way to make the underlying STL exception message visible instead of <private>?
3. Is the full-length KV-state read-modify-write in (4) a known ANE-compiler limitation on macOS 27? A decode step needs a data-dependent write position; what is the supported way to express that?
4. An alternative decode graph that uses a shape-derived dynamic slice_update fails ANE compilation on macOS 26 with (ANECompiler) Too many fvmlibs (more than 255), but on macOS 27 fails with E5RT (11) and no fvmlibs diagnostic. Same limit with a different downstream failure, or two unrelated mechanisms?
5. Does this reproduce on M1- or M3-generation hardware, or is it specific to this chip generation?
6. Other than exporting each function as a separate model — which we tested and which does not avoid the failure — is there a supported path to Neural Engine execution here?

Withdrawal: my original post mentioned a system-provided multifunction ML Program exhibiting the same failure. I cannot identify that model precisely enough to be actionable, so please disregard that claim.

Root cause confirmed. I believe this is a Core ML bug, and I have a one-row fix verified end-to-end on the production model.

Following up on my previous reply: I have isolated the cause and worked around it, applied the change to the real 1.8B model, and measured the result. Posting it here in case it saves someone else the bisection.

1. The remedy: make the KV state one row deeper than the region the decode graph writes.

The failing graph has a state with N rows on the position axis and a decode step whose state write covers all N of them. Allocating the state with N + 1 rows while leaving the write window at [0, N) makes the ANE specialization succeed. The extra row is never read and never written; it exists only so the write window is no longer full-extent.

The boundary is exact, from a controlled sweep that changed nothing but the state depth:

One row is enough, and zero rows is not. Where the window starts does not matter as long as it is not the whole dimension: [1, 257) in a 257-row state and [16, 272) / [32, 288) in a 288-row state all build.

I am reporting this as a bug, not as a modelling error. The package that fails and the package that succeeds are the same graph: identical operations, identical shapes for every input and output, the identical write window, identical numerics. The sole difference is that an inert buffer is exactly as long as the region being written. coremlcompiler compiles both without complaint, neither uses anything outside the documented MIL operation set, and no public API constrains or documents the relationship between a state's extent and the extent of the writes into it. The rejection happens inside ANE specialization as an internal C++ exception, for a property of the graph that the supported tooling gives a developer no way to observe. That is a defect in the compiler, not something the model is doing wrong.

2. It works on the real model, not just the reproducer.

The model is a 1.8B-parameter translation model — 32 layers, hidden 2048, 4 KV heads, context 1024, 4-bit lookup-table (mixed-precision) weights, exported as a multifunction ML Program with infer and prefill sharing one MLState, ~1.7 GB. Same architecture and same quantization as the build that fails; the only change is the one extra state row in both functions.

• The decode function now specializes for the Neural Engine. MLComputePlan reports 6756 operations for infer, of which 2659 prefer MLNeuralEngineComputeDevice, 9 are CPU (mask arithmetic: select, greater_equal, gather, cast), and 0 are GPU. The E5RT program descriptor confirms it from the other side: it names infer_ane, infer_ane0 and per-operation *_ane symbols, where the same function loaded with .cpuOnly names none.
• Throughput: 20.4–20.6 ms/step (~48–49 tok/s) with .cpuAndNeuralEngine, against 38.5–39.1 ms/step (~26 tok/s) with .cpuOnly — a 1.9x difference. .cpuAndNeuralEngine has no GPU in its plan, so that gap can only be the Neural Engine. For reference, the same graph ran at 46.8 tok/s on macOS 26 before the regression, so this restores the previous behaviour rather than approximating it.
• Outputs are unchanged. Four prompts, greedy, compared token-for-token against the unmodified package under .cpuOnly: identical sequences on all four. The extra row is inert by construction.
• Cost: one row of KV state. For this model that is [64, 4, 1025, 128] instead of [64, 4, 1024, 128], i.e. +0.098% state memory, and no additional per-step work.

The unmodified package is the control and it fails exactly as reported: its decode function raises the same E5RT (11) under .cpuAndNeuralEngine, while its prefill function — unchanged in this respect — specializes for the ANE in the same session. The regression was always decode-only, and the pad fixes precisely the function that was broken.

3. A second, quieter failure mode worth knowing about.

Some state-write shapes do not crash at all. Two slice_updates writing into one state, or a write whose begin is data-dependent, load successfully — MLModel returns, predict() runs and produces output — but Core ML silently declines the Neural Engine and executes on the CPU/GPU. No error, no warning, nothing in the log. If you try one of those as a workaround, LOAD OK tells you nothing about where the model is running.

That is also why I would not describe .all as a workaround. It loads, but the decode graph lands on the GPU — 2529 of 6500 operations in the shipping package — and measures 37.9 ms/step, indistinguishable from .cpuAndGPU.

4. Two corrections to my previous reply.

The "anchoring" reading was wrong. I had inferred from a confounded experiment that a state write had to be origin-anchored. A single write starting at a non-zero row builds fine; that experiment had two writes into one state, which is a separate failure mode in its own right. The anchoring rule as I stated it was an artifact of not separating the two variables.

A methodology caution for anyone repeating this: do not use the presence of an H16C.bundle/<function>/<function>_ane directory in the E5RT bundle cache as the test for whether the Neural Engine was used. For a multifunction package that check returns a false negative — the ANE programs are compiled through ane_compile/multiprocedure/ and no per-function _ane directory is created, so a model fully resident on the ANE looks identical to one on the CPU. MLComputePlan is the reliable check. This cost me a wrong conclusion for a while.

5. What I would still like from Apple.

The E5RT diagnostic is still redacted: E5RT encountered an STL exception. msg = <private>. followed by E5RT: <private> (11). Nothing in the load path or the error points at the state extent, and the public error (functionName must be nil... / doesn't support the multi-function description syntax) points somewhere else entirely. Two concrete asks:

1. Un-redact the E5RT exception text, or map it to a documented MLModelError case. A compiler that rejects this graph should say so.
2. Accept a full-extent state write. If that is not possible, document the constraint. The workaround is one extra row and costs 0.098% state memory — reasonable once you know it, undiscoverable if you do not. Nothing in the ANE compiler output, coremlcompiler, or MLComputePlan distinguishes a package that will fail specialization from one that will succeed; both compile cleanly and only the load reveals the difference.

Feedback filed: FB24884635

I have now filed a Feedback Assistant report from the affected M4 Max, including system diagnostics and a self-contained matched-pair reproducer.

Additional cross-version and cross-hardware testing with the same original model produced the following matrix:

• M4 Max + macOS 26: works
• M1 Max + macOS 27: works
• M4 Max + macOS 27.0 (26A428): fails

This isolates the observed regression to the M4 Max + macOS 27 combination among the configurations tested.

The attached reproducer contains two synthetic single-function ML Programs with identical weights and the same [0, 256) state read/write window:

• State dimension 256: .cpuAndNeuralEngine fails with Core ML execution-plan error -14; unified logging reports E5RT (11).
• State dimension 257: .cpuAndNeuralEngine loads successfully.
• State dimension 256: .cpuOnly loads successfully.

Both packages pass xcrun coremlcompiler compile. The failure occurs when MLModel performs ANE execution-plan specialization at load time.

The reproducer was re-run immediately before filing and produced:

REPRODUCTION=CONFIRMED

This indicates an ANE compiler/runtime regression in the tested M4 Max + macOS 27 combination, rather than an invalid ML Program, multifunction packaging issue, or general macOS 27 incompatibility.

Feedback ID: FB24884635