1. 2012 — AlexNet. A deep convolutional network by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton wins the ImageNet challenge by a wide margin, showing the power of GPUs and large datasets.

  2. 2013 — word2vec. Researchers at Google show that words can be represented as vectors that capture meaning, making “king − man + woman ≈ queen” famous.

  3. 2014 — Generative adversarial networks (GANs). Ian Goodfellow and colleagues pit two networks against each other to generate realistic images.

  4. 2015 — ResNet. Residual connections let researchers train networks with over a hundred layers, surpassing estimated human-level error rates on ImageNet classification.

  5. 2016 — AlphaGo. DeepMind combines deep learning and reinforcement learning to beat Lee Sedol at Go.

  6. 2017 — The transformer. Attention Is All You Need introduces the architecture that will power nearly every large language model.

  7. 2018 — BERT. Google’s pre-trained transformer sets new records across language-understanding benchmarks and popularizes “pre-train, then fine-tune”.

  8. 2020 — GPT-3. OpenAI’s 175-billion-parameter model shows that scale enables few-shot learning from examples in the prompt.

  9. 2020–2021 — AlphaFold 2. DeepMind’s system predicts protein structures with accuracy competitive with experiments, a breakthrough for biology described in Nature in 2021.

  10. 2022 — Stable Diffusion and ChatGPT. Open image generation and a conversational assistant bring generative AI to hundreds of millions of people.

The common thread

More data, more compute and architectures that scale well. Each milestone made the next one easier — and the pattern continues with today’s multimodal and reasoning models.

Why these milestones mattered

Each step solved a specific bottleneck:

  • AlexNet proved that deep networks trained on GPUs could beat hand-engineered computer vision, redirecting the field toward deep learning.
  • word2vec showed that meaning could be captured in dense vectors — the idea behind embeddings used everywhere today, from search to retrieval-augmented generation.
  • GANs opened realistic image generation, later largely superseded by diffusion models for many uses.
  • ResNet’s residual connections made very deep networks trainable — and residual connections are a core part of Transformers.
  • AlphaGo demonstrated that combining deep learning with search and reinforcement learning could master domains requiring intuition.
  • The Transformer replaced recurrence with attention, enabling parallel training at enormous scale.
  • BERT made pre-training plus fine-tuning the standard recipe for language tasks.
  • GPT-3 revealed that scale alone produced new abilities like learning from examples in the prompt.
  • AlphaFold 2 showed deep learning could solve a fundamental scientific problem.
  • Stable Diffusion and ChatGPT turned research into tools used by hundreds of millions of people.

What came after

Since 2022, the frontier has moved toward models that handle text, images, audio and video together; reasoning models that spend more computation thinking before answering; open-weight models that rival closed ones; and agents that use tools and act on behalf of users. Each builds on the foundations above — especially the Transformer and the lesson that scale matters.

Honorable mentions

Other developments that shaped the decade include dropout (2012–2014) to reduce overfitting, batch normalization (2015) for stable training, sequence-to-sequence models with attention (2014–2015) for machine translation, AlphaZero (2017) learning games from self-play, CLIP (2021) connecting images and text, and InstructGPT (2022), which showed how human feedback makes models follow instructions.

Where to learn more

Each milestone has a landmark paper that is readable with some background; our research section explains several, including the famous ResNet and the Vision Transformer, with notes on why each paper mattered.

What to watch next

  • Which of today’s breakthroughs will look like milestones in ten years.
  • Progress in reasoning, agents and robotics.
  • New architectures that could complement or replace Transformers.
  • AI’s contributions to scientific discovery.

Key takeaways

  • AlexNet in 2012 started the deep learning revolution in computer vision.
  • Embeddings, residual connections and attention became core building blocks.
  • The Transformer (2017) underpins nearly all large language models.
  • GPT-3 showed that scale unlocks few-shot learning.
  • AlphaFold and ChatGPT showed deep learning’s scientific and mass-market impact.

Frequently asked questions

When did the deep learning revolution start?

Most accounts date it to 2012, when AlexNet won the ImageNet competition by a wide margin.

Why was the Transformer so important?

It used attention instead of recurrence, allowing much more parallel training and scaling to very large models.

Is deep learning the same as AI?

Deep learning is one approach within machine learning, which is itself part of AI. It dominates current AI systems.