Skip to content
Language Pretraining(Contemporary realistic technology illustration)

AccelerationFour-dimensional view8 years ago

2018-06-11

Language Pretraining

GPT and BERT first learn from large amounts of unlabeled text and are then adapted to specific tasks, turning language models from specialized tools into reusable foundations.

Art style · Contemporary realistic technology illustration

Origin of Life on EarthDinosaurs and MammalsAI Agents
Arrow keys step through pictures · Wheel to zoom · Drag or click to select time

Civilization metrics

Each sparkline shows the documented history of a metric on a logarithmic time axis. The dot marks the selected moment.

Kardashev index

0.726

Based on primary energy power · Type I = 1

World population

7.75Bpeople

1/1 of today

Total human primary energy power

1.8×10¹³W

1/1 of today

Compute per dollar

3.5×10¹⁰floating-point operations per second per dollar (FLOP/s per $, nominal dollars)

1/2 of today

Fastest delivery time for a message from London to New York

0.03 seconds

Comparable to today

Fastest speed reached by humans

39,897km/h

Comparable to today

Near this moment

The eight closest milestones. Select a title to jump there.

  1. 2017-06-12Intelligence

    The paper "Attention Is All You Need" introduces the Transformer architecture. Source

  2. 2018-02-06Universe

    The most powerful operational rocket makes its first flight, and its two side boosters land in sync. Source

  3. 2018-06-11Intelligence

    OpenAI pretrains a language model on unlabeled text, then fine-tunes it for specific tasks. Source

  4. 2018-10-11Intelligence

    Google's bidirectional pretrained model sets new records on many language understanding tasks. Source

  5. 2019-01-03Universe

    A spacecraft makes the first soft landing on the far side of the Moon. Source

  6. 2019-02-14Intelligence

    A 1.5-billion-parameter model writes coherent long-form text; OpenAI releases it in stages. Source

  7. 2019-04-10Universe

    The Event Horizon Telescope releases an image of the shadow of the M87 black hole. Source

  8. 2019-10-23Computing

    Google's Sycamore processor outperforms classical supercomputers on a specific sampling task. Source

Capabilities

Acquired93 / 121 items

Recently acquired

Not yet available

Scenes across eras

Each era illustrated in its own style.

Every timeline entry cites its source. Metrics interpolate logarithmically between sourced data points. Forecasts include institutional and community views, and attributed site scenarios; years are not guaranteed realization dates. Illustrations are AI-generated in each era's style.