Google DeepMind Repurposes Gemma 4 as a Diffusion Model

Google DeepMind has developed 'DiffusionGemma,' which repurposes the existing language model 'Gemma 4' as a diffusion model. Rather than training a new model from scratch, it was built at less than 10% of the original training cost, making it a noteworthy effort in terms of development efficiency.

Text generation AI broadly employs two approaches. One is the 'autoregressive model' approach, which generates words one at a time in sequence. Most mainstream large language models today, such as ChatGPT and Gemini, adopt this method. The other is the 'diffusion model,' which applies a technology widely used in image generation AI to text. Diffusion models feature a mechanism that progressively restores information from noise, and are characterized by their ability to process multiple elements in parallel.

DiffusionGemma generates 256 tokens in parallel simultaneously. Through this, Google DeepMind demonstrates processing speeds of approximately 1,500 tokens per second. Tokens are the minimal units used by AI when processing text, roughly equivalent to 1-2 characters. Compared to the autoregressive approach, which outputs words one by one, the structure offers potential advantages in processing speed through parallelization.

However, challenges remain on the quality front. In benchmark evaluations, DiffusionGemma's performance falls short of the original autoregressive model, with particularly large gaps in tasks requiring reasoning. Reasoning tasks refer to problems where answers are derived through multiple steps, and this represents a current weakness of the diffusion model approach.

The significance of this effort can be attributed to the perspective of 'reusing existing models.' Since training large language models requires enormous costs and time, the ability to repurpose already-trained models for different architectures could fundamentally change research methodology. The fact that an alternative model was built at less than 10% of the cost represents one concrete demonstration of this possibility.

Research on text-based diffusion models has become a field that multiple research institutions have pursued in recent years. However, whether the performance gap between it and autoregressive models can be closed remains at a stage dependent on future research. The key question becomes whether inference capabilities can be improved while maintaining speed advantages.

Google DeepMind's efforts to diversify architectures by leveraging existing assets demonstrate a shift in text generation AI development from 'convergence on a single approach' to 'parallel exploration of multiple approaches.' Even if DiffusionGemma is not yet a finished product, demonstrating the potential for low-cost repurposing provides important insights for the future research community.