While K-means quantization (Part 2) achieves impressive storage compression, it falls short on a critical dimension: computational efficiency. All inference operations remain in floating-point, pre...

With a solid understanding of numeric data types from Part 1, we’re ready to tackle our first practical quantization technique. K-means based quantization treats neural network weight compression a...

Neural network quantization has emerged as one of the most effective techniques for making deep learning models efficient enough to run on resource-constrained devices. But before diving into quant...

Rethinking How We Train Pruned Networks In Part 4, we mastered the practical art of pruning: Layer-wise pruning ratios for optimal compression Iterative pruning with fine-tuning for minimal a...

In Parts 1, 2A, and 2B, we explored the foundational parameters—temperature controls randomness, decoding strategies (greedy, beam search) determine how we search for sequences, and sampling method...

The Journey So Far Throughout this series, we’ve built a comprehensive understanding of neural network pruning: Part 1: Why pruning matters (model size explosion, energy costs) Part 2: What ...

In Part 2A, we explored foundational decoding strategies: greedy, beam search, and top-k sampling. We discovered that top-k’s fixed cutoff doesn’t adapt to the model’s confidence—sometimes includin...

Recap: The Pattern is Set, But Which Weights? In Part 2, we learned about different pruning granularities—from fine-grained to structured. Now we face the critical question: Which specific weight...

In Part 1, we learned how temperature controls the “shape” of probability distributions—making the model more focused or more exploratory. But temperature alone doesn’t answer a critical question: ...

Recap: Why Granularity Matters In Part 1, we learned that pruning can reduce model size by 3-12× without losing accuracy. But there’s a catch: not all pruning patterns are created equal. Imagine ...