Tencent Open-Sources Hy4 Preview: 770B MoE Foundation Model with 1M Context
On August 28, 2026, Chinese technology conglomerate Tencent officially open-sourced a preview checkpoint of Hy4, its next-generation flagship foundation model, releasing weights directly on Hugging Face.
Hy4 Preview utilizes a massive 770-billion parameter Mixture-of-Experts (MoE) architecture that routes each input token through 49 billion active parameters (770B-A49B). The release includes full-precision and native FP8 quantized checkpoints (tencent/Hy4-preview and tencent/Hy4-preview-FP8), designed to fit high-density multi-GPU clusters using standard vLLM and TensorRT-LLM serving frameworks. The model features a native 1-million-token context window, enabling repository-scale code analysis and multi-document synthesis without prompt truncation.
In early evaluation ledger scores tracked by BenchLM, Hy4 Preview recorded a 61.7% composite benchmark mark, positioning it ahead of recently previewed architectures like Qwen3.8-Flash-Next (55.7%) across reasoning and multi-turn instruction following. Tencent highlighted synthetic data distillation and self-correcting reinforcement learning loops as core drivers behind the model's 49B active efficiency.
The release intensifies competition among frontier open-weight providers in Asia, joining Z.AI's [GLM-5.3-Flash](/articles/glm-5-3-flash-release) and Alibaba's Qwen3.8 series in offering 1M-context foundation models for enterprise self-hosting. By providing immediate FP8 checkpoints, Tencent enables infrastructure engineers to deploy production-grade agent reasoning stacks while minimizing inference VRAM footprints.
