GLM-Omni Study Examines Why Image Generation Knowledge Fails to Transfer to Text Answers
GLM-5.5 要来了？唐杰最新论文，曝出智谱神秘模型
- Who
- Tsinghua University and Zhipu AI
- What
- conducted a study how image generation training transfers visual knowledge to language answers in unified multimodal models
- Quote
这篇研究由清华大学与智谱 AI 联合完成，唐杰担任通讯作者。
What happened
A joint Tsinghua University and Zhipu AI study introduces GLM-Omni and examines whether knowledge learned through image generation transfers to text answers. In controlled fictional-world tasks, GLM-Omni achieved 54.00% visual accuracy but 14.57% text-answer accuracy; interleaved image–text training raised text accuracy to 26.86%, and Visual CoT increased it to 41.14%. The paper does not establish a connection between GLM-Omni and GLM-5.5.
Read next
See the full storyTsinghua and Zhipu Study Finds Limited Transfer From Image Generation to Text AnswersThe story so far
Tsinghua University and Zhipu AI introduced GLM-Omni in a study that found knowledge learned through image generation transferred poorly to text answers, while interleaved image–text training and Visual CoT improved text-answer accuracy. In controlled fictional-world tasks, GLM-Omni achieved 54.00% visual accuracy but 14.57% text-answer accuracy. Interleaved image–text training raised text-answer accuracy to 26.86%, while Visual CoT increased it to 41.14%. The experiments examined knowledge transfer in a unified multimodal model using fictional worlds. Their results do not establish how well these approaches work in real, changing environments.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-10-27