A common problem in machine learning is having too little training data. This article discusses two approaches to generating synthetic data for training models when there's not enough real data. The first method involves applying transformations and augmentations to existing data, such as changing the wording of text or altering images. The second approach uses powerful pre-trained models to generate new data points, particularly in language models where few-shot prompting has shown promising results. --- Why it matters: This matters because it provides researchers with two potential solutions for addressing a common challenge in AI development: limited training data. Being able to generate synthetic data can expand the possibilities for model training and deployment. Source: https://lilianweng.github.io/posts/2022-04-15-data-gen/