A preprint reports that a new way of assigning different numerical precisions to different parts of a language model can preserve benchmark quality at three-bit average quantization. In its tests, the method, called FAMPWQ, posted lower perplexity than several comparison methods and won pairwise judgments from an automated evaluator, although AWQ was faster on the tested hardware.
FAMPWQ is a weight-only, mixed-precision post-training quantization framework for deploying LLMs under a fixed memory budget. In practical terms, it changes the precision used for model weights after training and focuses on weights rather than joint weight-and-activation quantization. It has two stages: first, it estimates how much each layer is affected by the exact rounding change produced when its weights are quantized; second, a reinforcement-learning search called PPO uses those Fisher-based estimates to choose bit widths layer by layer.
At three bits, the gap widened
The evaluation covered seven LLMs, five benchmarks and seven baseline approaches. For fair comparison, all quantization methods used identical settings, including 128 randomly sampled C4 calibration sequences. PPL, short for perplexity, was the paper's text-prediction quality score; it was averaged across three independent runs with different calibration seeds, and lower PPL was treated as better.
On LLaMA-7B at four average bits, FAMPWQ recorded PPL of 5.81 on WikiText-2 and 10.34 on PTB, edging AWQ by 0.02 and 0.08 respectively. At three average bits, its average PPL was up to 2.37 lower than GPTQ and 0.40 lower than AWQ.
In an additional four-bit comparison, FAMPWQ had the best reported average PPL across three LLMs and three benchmarks: 10.81 for Qwen2.5-7B, 9.15 for Qwen2.5-14B and 8.04 for Mistral-7B-v0.1.
On Qwen2.5-7B under three-bit quantization, FAMPWQ's average zero-shot accuracy was 6.87 percentage points higher than RTN's and 6.69 points higher than GPTQ's.
GPT-3.5-turbo was used to judge 4-bit LLaMA2-13B-chat comparisons, with 80 questions and 160 trials per comparison. With ties excluded, FAMPWQ won 76% against RTN, 69% against OWQ, 64% against GPTQ, 72% against GPTQv2, 61% against AWQ and 54% against OmniQuant. The AWQ comparison met a p < 0.05 threshold in a two-sided binomial test.
Speed and setup told a different story
On an NVIDIA 4090, FAMPWQ reported throughput gains, meaning a higher processing rate, of up to 42% over FP16, 69% over OWQ and 28% over AMQ. AWQ nevertheless remained faster, posting speedups of 2.44 times on 7B and 2.10 times on 13B.
FAMPWQ's preprocessing cost was below one GPU-hour for every tested model, including 14B-scale models. The Fisher sensitivity calculation took 24 to 64 minutes, while the reinforcement-learning search took under five minutes on one GPU. Fisher estimation therefore dominated the reported preprocessing time.
The paper also compared its Fisher-information measure, or FIM, with oracle sensitivity to see whether the proposed allocation was tracking quantization degradation. Pearson correlations, a measure of how closely two measures move together, were 0.91 on LLaMA-7B and 0.88 on Qwen2.5-7B. At 3.5-bit average quantization, the reported PPL increase was 0.42 for FIM, versus 0.85 for weight magnitude and 1.19 for random allocation.
In the allocation ablation, the reinforcement-learning strategy had average PPL up to 1.50 lower than Greedy, 1.33 lower than Bayesian optimization, 1.90 lower than simulated annealing and 0.45 lower than the genetic algorithm.
The test still has limits
The experiments did not test Mixture-of-Experts models. FAMPWQ is limited to weight-only quantization, and the use of different bit widths can reduce throughput or compatibility with optimized uniform-precision kernels. The authors also identify Fisher sensitivity estimation as a dominant offline preprocessing cost.
The document is an arXiv preprint, version 2, dated 31 August 2026. The authors report partial support for Juncheng Jia from the Priority Academic Program Development of Jiangsu Higher Education Institutions and the Suzhou Frontier Science and Technology Program, Project SYG202310.
Paper data and sources
Original title: FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference
Authors: Gongwei Lee, Ji Liu, Juncheng Jia, Ji Wu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-24
DOI: Not available
Original paper · Full text