← All tasks
cppml-explore/mlx #3586Not a task: not reproduced

Xcode 26.5 / metalfe-32023.883 mis-compiles NAX GEMM kernel

envgap__ml-explore__mlx-3586

01 / FAILURE SIGNATURE

As reported upstream

Given this is an 8-bit MiniCPM path, I’d look first at quantized matmul/GEMM kernels. A small numerical/codegen error in logits under greedy decoding would easily turn into the repeated yes111111111 pattern.
Not a benchmark task.
  • In a clean container the reported failure did not reproduce, or the known fix did not make the project run.

02 / ENVIRONMENT RECIPE

Base commit
657ccf79dc736e41dcd70f7fdc98d53b45744d7f
Manifest
CMakeLists.txt
Reproduce
Awaiting issue-specific recipe
Run under trace
Awaiting a meaningful runtime command

03 / ORIGINAL ISSUE TEXT

ml-explore/mlx #3586 · read the original issue
**Describe the bug**
Running `mlx-community/MiniCPM-V-4.6-8bit` with  `0.32.0.dev20260524+2165dc08` produces gibbering (`yes111111111`).    `mlx-0.31.2 mlx-metal-0.31.2` produces something which looks more expected in response to a prompt of `Hi`:  `Hello! How can I help you today?`

The culprit seems to be the locally built mlx.metallib, not the Python wheel and probably not the post-v0.31.2 source changes by themselves.

The checks:

* PyPI mlx + local mlx-metal: yes111111111
* local mlx + PyPI mlx-metal: Hello! How can I help you today?
* PyPI backend install, but with only mlx.metallib swapped to the local build: yes111111111
* restored PyPI mlx.metallib: Hello! How can I help you today?
So the bad output follows the Metal shader library specifically.

My bot's best guess: Xcode/Metal toolchain drift. The PyPI 0.31.2 backend artifact was built against SDK 26.4; the local artifact was built with Xcode 26.5 / SDK 26.5 / metalfe-32023.883. That smells like either a Metal compiler regression or an MLX Metal kernel that depends on behavior the newer compiler now handles differently.

Given this is an 8-bit MiniCPM path, I’d look first at quantized matmul/GEMM kernels. A small numerical/codegen error in logits under greedy decoding would easily turn into the repeated yes111111111 pattern.

The local mlx build follows mlx’s current dev install guidance, `pip install -e ".[dev]"`

**To Reproduce**

```zsh
% pip list | grep mlx
check_models              0.7.2                       /Users/jrp/Documents/AI/mlx/check_models/src
mlx                       0.32.0.dev20260524+2165dc08 /Users/jrp/Documents/AI/mlx/mlx
mlx-audio                 0.4.3
mlx-lm                    0.31.3                      /Users/jrp/Documents/AI/mlx/mlx-lm
mlx-vlm                   0.5.0                       /Users/jrp/Documents/AI/mlx/mlx-vlm
(mlx-vlm) jrp@Johns-MBP-2 src % python -m mlx_vlm generate \
  --model mlx-community/MiniCPM-V-4.6-8bit \
  --prompt "Hi" --max-tokens 10
Fetching 9 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 14064.36it/s]
Download complete: : 0.00B [00:00, ?B/s]                                                                                                                                    | 0/9 [00:00<?, ?it/s]
==========
Files: [] 

Prompt: <|im_start|>user
Hi<|im_end|>
<|im_start|>assistant
<think>

</think>


yes111111111
==========
Prompt: 13 tokens, 509.350 tokens-per-sec
Generation: 10 tokens, 293.992 tokens-per-sec
Peak memory: 2.378 GB
(mlx-vlm) src % pip uninstall mlx
Found existing installation: mlx 0.32.0.dev20260524+2165dc08
Uninstalling mlx-0.32.0.dev20260524+2165dc08:
  Would remove:
    /Users/jrp/miniconda3/envs/mlx-vlm/bin/mlx.distributed_config
    /Users/jrp/miniconda3/envs/mlx-vlm/bin/mlx.launch
    /Users/jrp/miniconda3/envs/mlx-vlm/lib/python3.13/site-packages/__editable__.mlx-0.32.0.dev20260524+2165dc08.pth
    /Users/jrp/miniconda3/envs/mlx-vlm/lib/python3.13/site-packages/mlx-0.32.0.dev20260524+2165dc08.dist-info/*
Proceed (Y/n)? y
  Successfully uninstalled mlx-0.32.0.dev20260524+2165dc08
% pip install mlx  
Collecting mlx
  Using cached mlx-0.31.2-cp313-cp313-macosx_26_0_arm64.whl.metadata (5.9 kB)
Collecting mlx-metal==0.31.2 (from mlx)
  Using cached mlx_metal-0.31.2-py3-none-macosx_26_0_arm64.whl.metadata (5.1 kB)
Using cached mlx-0.31.2-cp313-cp313-macosx_26_0_arm64.whl (584 kB)
Using cached mlx_metal-0.31.2-py3-none-macosx_26_0_arm64.whl (55.8 MB)
Installing collected packages: mlx-metal, mlx
Successfully installed mlx-0.31.2 mlx-metal-0.31.2
(mlx-vlm) src % python -m mlx_vlm generate \
  --model mlx-community/MiniCPM-V-4.6-8bit \
  --prompt "Hi" --max-tokens 10
Fetching 9 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 20729.67it/s]
Download complete: : 0.00B [00:00, ?B/s]                                                                                                                                    | 0/9 [00:00<?, ?it/s]
==========
Files: [] 

Prompt: <|im_start|>user
Hi<|im_end|>
<|im_start|>assistant
<think>

</think>


Hello! How can I help you today?
==========
Prompt: 13 tokens, 508.680 tokens-per-sec
Generation: 10 tokens, 290.371 tokens-per-sec
Peak memory: 2.383 GB
```

**Expected behavior**
The expected behaviour is shown in the trace with the released version of mlx, rather than the local build. 

**Desktop**
 - OS Version: MacOS 26.5

Continue on GitHub ↗

04 / LABELS

Labels from the report text only; not yet run

No supported category has been assigned.

Label rules and the text that matched
[]