The article discusses the advancements in natural language processing (NLP) in 2018. Large-scale pre-trained language models like OpenAI GPT and BERT have shown impressive performance on various tasks without requiring labeled data for pre-training. This approach is similar to ImageNet classification pre-training, but with even greater benefits. The author notes that this method allows for increased training scale, which can lead to better results. --- Why it matters: This matters because it shows how large-scale language models can be used to improve performance on various NLP tasks without requiring extensive labeled data. This has significant implications for researchers and engineers working in AI, as it opens up new possibilities for experimentation and improvement. Source: https://lilianweng.github.io/posts/2019-01-31-lm/