ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

arXiv cs.AI · 18 hours ago

ExTernD is a post-training quantization method that decomposes large language model weight matrices into ternary factors with an expanded rank beyond full rank, allowing it to progressively reduce quantization error. The method achieves 4-bit quantization performance (5.2-5.5 bits per weight) on Gemma and Qwen models, with a full Qwen3.5-4B conversion reaching 10.10 perplexity compared to 9.78 for unquantized models. This approach enables continuous scaling of accuracy versus memory trade-offs rather than being constrained to fixed bit-width levels.