LookAtMyQuant: Open-Source Tool Shrinks AI Models to Run on Apple Silicon
The open-source LookAtMyQuant project compresses large language models to fit a set memory budget while losing as little quality as possible. Its authors report shrinking the Inkling-2-Mini model from 18.89 GB to 5.38 GB.
An open-source project called LookAtMyQuant has been published on GitHub. It quantizes language models, meaning it stores their weights using fewer bits, so that large AI models fit into limited memory, for example on Apple Silicon devices. It is released under the MIT license, and its write-up drew notable attention on the Hacker News discussion site.
How it works
LookAtMyQuant is a quantization compiler. It measures how much compressing each stored piece of a model harms the output, then chooses a code for each piece so the whole model fits a size budget with the least damage. The authors say that on Gemma 4 E2B, its GGUF file has 12 times lower KL divergence than Q4_K_M at the same file size.
Results
Qwen3.5-4B was reduced to 1.23 GB (2.3 bits per weight). It solves 330 of 540 exact-answer math problems; the original solves 359, and Prism ML's Bonsai 2 Ternary 4B, at 1.20 GB, solves 328. Qwen3.8-27B was cut from 53.7 GB to 7.2 GB and solves 401 problems, as many as the original.
Inkling-2-Mini, whose text weights take 18.89 GB in the original, is offered in four versions: 12.76, 11.18, 8.87 and 5.38 GB. The largest matches the original on the GSM8K math benchmark (193 of 200) and the HumanEval coding benchmark (152 of 164). The smallest scores 88.0% on GSM8K and 70.7% on HumanEval.
On an Apple M5 Max, with a dedicated drafter model, the 5.4 GB version reaches a median of 124.4 tokens per second, and the 12.8 GB version 112.7.
Availability
The runtime installs with pip and can be used through the lamq command-line tool or a Python library. The repository also contains the code to reproduce the Qwen3.5-4B recipe, which the authors put at about $34 of GPU time.


