Skip to content
K = 0.731

Tsinghua and Zhipu Study Finds Limited Transfer From Image Generation to Text Answers

1 reports 1 reporting sources 5 hours agoNew

Understand the event

AI overview

Tsinghua University and Zhipu AI introduced GLM-Omni in a study that found knowledge learned through image generation transferred poorly to text answers, while interleaved image–text training and Visual CoT improved text-answer accuracy. In controlled fictional-world tasks, GLM-Omni achieved 54.00% visual accuracy but 14.57% text-answer accuracy. Interleaved image–text training raised text-answer accuracy to 26.86%, while Visual CoT increased it to 41.14%. The experiments examined knowledge transfer in a unified multimodal model using fictional worlds. Their results do not establish how well these approaches work in real, changing environments.

Generated by AI from reports · Updated 2 hours ago

Latest development10/27 15:43 (UTC+8)
GLM-Omni Study Examines Why Image Generation Knowledge Fails to Transfer to Text Answers雷锋网

Read next

More changes in today’s briefingIntroducing GPT-6.1 Terra

Report timeline

Follow the reports to explore different sides of the event.

10/27
  1. 雷锋网
    GLM-Omni Study Examines Why Image Generation Knowledge Fails to Transfer to Text Answers

    A joint Tsinghua University and Zhipu AI study introduces GLM-Omni and examines whether knowledge learned through image generation transfers to text answers. In controlled fictional-world tasks, GLM-Omni achieved 54.00% visual accuracy but 14.57% text-answer accuracy; interleaved image–text training raised text accuracy to 26.86%, and Visual CoT increased it to 41.14%. The paper does not establish a connection between GLM-Omni and GLM-5.5.

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.