A few predictions on Qwen4 HC and Quantization while we wait
I previously wrote about the shift to 4-way HC. Traditionally, optimization has focused primarily on FFN and Attention. With the HC architecture, however, residual stream mixing and information preservation must also be taken into account when optimizing for quality, model size, and speed.