The Menteebot prototype unveiled today (Wednesday) incorporates artificial intelligence at all levels of its operation. The movement is based on an innovative approach of machine learning from simulation to reality (Sim2Real), in which the learning is carried out in a simulated version of the robot, which allows the use of an unlimited amount of data for training, so that it is then adapted to the real world but with minimal data requirements.
The mapping of the environment is performed in real time using algorithms based on NeRF (the latest family of technologies based on neural networks for representing three-dimensional scenes). The 3D models are enriched with semantic information and create cognitive maps that allow the robot to query in real time to find items and places. The robot locates its position within the 3D map and automatically plans dynamic routes, avoiding obstacles.
Large transformation-based language models (LLMs) are used to interpret commands and construct the steps required to complete the task. To perform complex tasks, emphasis is placed on the coordination of the robot and the ability to combine leg movement with manual work, for example, dynamic balancing of the robot when carrying weight or reaching out.
The prototype unveiled today, although not yet the final version of the robot, for the first time demonstrates end-to-end action – from a verbal command to the completion of a complex task that includes navigation, movement, understanding the environment, identifying and positioning objects, grasping and understanding natural language.
The robot is expected to go into serial production during the first quarter of 2025, and it will be operated by camera-based sensing only, electric motors developed in the company that enable unprecedented agility and fully integrated artificial intelligence. The artificial intelligence technologies embedded in it enable complex thought processes for completing tasks, managing conversations and continuously learning new tasks.
“The field of AI is just before a point of convergence of various technologies: computer vision, natural language understanding, powerful and detailed simulators, and methodologies that go from simulation to the real world,” said Prof. Amnon Shashua, founder and chairman of the company. “At Mentee Robotics we This combination is seen as the starting point for the design of walking human robots for any purpose, which will be able to move freely in a human environment and have the ability to perceive that enables the performance of household tasks and the learning of tasks for which they were not trained through imitation.”
