TIDE: Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation

Diffusion Transformers (DiTs) are a powerful yet underexplored class of generative models compared to U-Net-based diffusion architectures. We propose TIDE—Temporal-aware sparse autoencoders for Interpretable Diffusion transformErs—a framework designed to extract sparse, interpretable activation features across timesteps in DiTs. TIDE effectively captures temporally-varying representations and reveals that DiTs naturally learn hierarchical semantics (e.g., 3D structure, object class, and fine-grained concepts) during large-scale pretraining. Experiments show that TIDE enhances interpretability and controllability while maintaining reasonable generation quality, enabling applications such as safe image editing and style transfer.

Authors

• Victor Shea-Jay Huang Multimedia Laboratory (MMLab), The Chinese University of Hong Kong Shanghai Artificial Intelligence Laboratory
• Le Zhuo
• Yi Xin National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
• Zhaokai Wang
• Fu-Yun Wang
• Yuchi Wang Peking University
• Renrui Zhang The Chinese University of Hong Kong
• Peng Gao
• Hongsheng Li Multimedia Laboratory The Chinese University of Hong Kong, Centre for Perceptual and Interactive Intelligence

Keywords

No keywords are indexed for this paper.