| | --- |
| | datasets: |
| | - d3LLM/trajectory_data_llada_32 |
| | pipeline_tag: text-generation |
| | tags: |
| | - diffusion |
| | - text-generation |
| | - fast-inference |
| | - d3llm |
| | license: apache-2.0 |
| | library_name: transformers |
| | base_model: GSAI-ML/LLaDA-8B-Instruct |
| | --- |
| | |
| | # d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation 🚀 |
| |
|
| | This repository contains **d3LLM-LLaDA**, an ultra-fast diffusion language model presented in the paper [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://huggingface.co/papers/2601.07568). |
| |
|
| | - 📄 **Paper:** [arXiv:2601.07568](https://huggingface.co/papers/2601.07568) |
| | - 💻 **Code:** [GitHub - hao-ai-lab/d3LLM](https://github.com/hao-ai-lab/d3LLM) |
| | - 🌐 **Blog:** [Ultra-Fast Diffusion LLMs](https://hao-ai-lab.github.io/blogs/text-diffusion/) |
| | - 🕹️ **Demo:** [d3LLM Demo](https://d3llm-team.github.io/) |
| |
|
| | ## Model Description |
| |
|
| | **d3LLM-LLaDA** is an ultra-fast diffusion language model that strikes a balance between accuracy and parallelism. It uses pseudo-trajectory distillation to teach the model which tokens can be decoded confidently at early steps, and employs an entropy-based multi-block decoding mechanism with KV-cache refresh during inference. |
| |
|
| | ## Key Features |
| |
|
| | - 🚀 **High throughput:** 5.0× faster than autoregressive models (Qwen-2.5-7B-it) on H100 GPU and 3.5× faster on A100 GPU. |
| | - 📊 **High AUP:** Achieves high Accuracy Under Parallelism scores across benchmarks. |
| | - 🔧 **Task Optimization:** Specifically optimized for coding and math reasoning tasks. |
| |
|
| | ## Installation |
| |
|
| | To use this model, it is recommended to clone the official repository and install the required dependencies: |
| |
|
| | ```bash |
| | # Clone the repository |
| | git clone https://github.com/hao-ai-lab/d3LLM.git |
| | cd d3LLM |
| | |
| | # Install dependencies |
| | pip install -r requirements.txt |
| | ``` |
| |
|
| | ## Citation |
| |
|
| | If you find d3LLM useful for your research, please cite the following work: |
| |
|
| | ```bibtex |
| | @article{arxiv'26:d3llm, |
| | title = {d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation}, |
| | author = {Yu-Yang Qian and Junda Su and Lanxiang Hu and Peiyuan Zhang and Zhijie Deng and Peng Zhao and Hao Zhang}, |
| | journal = {ArXiv preprint}, |
| | volume = {arXiv:2601.07568}, |
| | year = {2026} |
| | } |
| | ``` |
