A robot that understands human speech and can learn quickly is presented - video

Mentee Robotics has unveiled its first prototype of a bipedal humanoid robot that understands human speech and can learn.

This was reported by SSPDaily.

The team is developing a humanoid robot that, according to them, will be able to understand commands in natural language using artificial intelligence.

The Menteebot prototype contains AI at every level of its operations. The robot's movement is based on a new machine learning method called simulated reality (Sim2Real). In this method, reinforcement learning takes place on a virtual version of the robot, which means it can use as much data as it needs to learn and then respond to the real world using very little data.

NeRF-based methods, which are the latest neural network-based technologies for representing 3D scenes, map the world on the fly. Semantic knowledge is stored in these cognitive maps that the computer can query to find things and places.

The Mentee robot can figure out where it is on a 3D map and then automatically plan dynamic paths to avoid obstacles.

The prototype demonstrated a full cycle of complex tasks, including navigation, locomotion, scene understanding, object detection and localization, grasping, and natural language understanding.

However, Mentee Robotics noted that this is not the final version. The company also reported that it initially targets two main markets for the Mentee humanoid. One of these markets is the home market, where a home assistant that can maneuver around households is capable of performing a range of tasks, including table setting, cleaning, laundry, and the ability to learn new tasks on the fly through verbal instructions and visual imitation. The second industrial market is warehousing, where the warehouse automation robot is designed to efficiently search for, retrieve, and transport items, and is capable of handling loads weighing up to 25 kg.

Mentee Robotics said it plans to release a production-ready prototype by Q1 2025. The system uses only vision-based cameras to perceive the world around it.

In addition, the company's engineering team has developed its own electric motors to provide the necessary dexterity for the robot.

"We are at the threshold of convergence of computer vision, natural language understanding, powerful and detailed simulations, and methodologies for moving from simulation to the real world," Prof. Amnon Shashua, Chairman of Mentee Robotics, said. "At Mentee Robotics, we see this convergence as a starting point for designing a future universal bipedal robot that can move everywhere - like a human - with a brain to perform everyday tasks and learn through simulation tasks for which it has not been trained before."