Skip to content
K = 0.731Subscribe

Yan Junchi Presented JTok and xHC Transformer Extensions

1 reports 1 reporting sources Updated 3 hours ago

Understand the event

AI overview

Shanghai Jiao Tong University professor Yan Junchi presented JTok and xHC, two approaches to expanding Transformer capacity, at CNCC 2026. The report says JTok-M reached the same loss with about 35% less training compute, while xHC improved downstream scores and used about 33% less training compute than a standard Transformer. JTok uses a learnable parameter table to modulate model updates for each token. xHC expands the residual stream, using dense reads and sparse writes. Both methods carry costs in inference latency, memory bandwidth, and throughput.

Generated by AI from reports · Updated 32 minutes ago

Latest development11/4 15:48 (UTC+8)
Yan Junchi’s Team Explores Two Ways to Extend Transformer ArchitecturesLeiphone

Read next

More changes in today’s briefingHugging Face Proposes Model Integration Contracts for Transformers

Report timeline

Follow the reports to explore different sides of the event.

11/4
  1. Leiphone
    Yan Junchi’s Team Explores Two Ways to Extend Transformer Architectures

    At CNCC 2026, Shanghai Jiao Tong University professor Yan Junchi presented JTok and xHC, two approaches to adding model capacity within the Transformer framework. The report says JTok-M reduced training compute by about 35% at the same loss, while xHC improved downstream scores and reduced training compute by about 33% versus a standard Transformer. It also describes inference latency, memory bandwidth, and throughput costs for both methods.

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.