Researchers have developed a new method for training robots to perform dexterous tasks, such as picking and placing objects, using a combination of simulation data and real-world demonstrations. The approach, called Mask2Real-WM, uses two stages: first, it predicts future segmentation masks from past masks and action sequences; then, it maps these masks to photorealistic RGB images using a pre-trained model. This method allows for more accurate control over the robot's actions, particularly in fine-grained tasks such as per-joint movement. The authors claim that their approach can be trained on large amounts of synthetic data and then fine-tuned on smaller amounts of real-world data, reducing the need for extensive real-world training. --- Why it matters: This matters to engineers working on robotics and AI because it provides a more efficient way to train robots to perform complex tasks. By leveraging simulation data and real-world demonstrations, Mask2Real-WM can improve the accuracy and control of robotic actions, particularly in areas such as dexterous manipulation. Source: https://arxiv.org/abs/2607.04546