Key Advantages of Avsr Cognix

Self-Localization With Semantic Understanding

Localization is the most fundamental building block of autonomous robot. However, classical localization techniques only outputs positional data: robot pose, map point cloud, etc. What these techniques are missing are the context of the environment. We have designed Cognix to provide Semantic Scene Graph (SSG) that captures the types of objects and the relationships between the objects and the map. Imagine the first time your robot is deploying in any environment. Every object the robot has captured through the camera is labeled and stored in its memory. And this is utilized later for the robot’s understanding of the environment and path-planning.

Sensor data (3D lidar, 2D lidar, odometry, IMU or camera) is first piped into SLAM algorithm to create the map for localization and navigation. At the same time, camera images are fed into our proprietary segmentation model to box all the objects and the relationship between themselves and the robot.

Spatial Reasoning Navigation

Usually when we build and deploy robots, we need to collect data by creating a map. Once the map is created, humans use our semantic understanding to program the robot:

1. Setting Point of Interest (POI) like charging stations, loading docks (for warehouses), or elevators.
2. Set a pre-programmed path for the purposes of a guiding tour or full coverage cleaning

With LLM as the back end of spatial reasoning and navigation, the robot itself has the semantic understanding and spatial reasoning of the environment. The robot can now detect the POI in your environment. Without specifically programming the robot, it is capable of either going to POI or explore the environment until it finds the POI with natural language. This adds a layer of flexibility as well as one-shot localization because the robot can navigate in SLAM mode.

With natural language commands, our high level LLM planner fully utilizes the SSG and its reasoning capability to devise a plan. Our specialized Reinforcement Learning based low level planner execute the path. The high level planner and low level planner recursively devise a new plan and executing the plan until objective is fulfilled.

In a nut shell, Cognix takes in sensor data and outputs robot velocity. Any command can be issued via natural language for a simpler integration.

Others

• Out of the box functionality with Zero-shot localization: Perhaps most important need for dynamic real-world environments — no pre-mapping or configuration needed. Robot can move around on its own and build a map of the environment autonomously. This enables useful out of box functionality for the robot.
• Faster Search: As the graph captures semantic relations between objects, such as, “in front of”, “besides”, “to the left of”, it enables up to 30% faster search of objects in the environment.
• Multi-object search: Allows robots to search for multiple items in no particular order: much more difficult task than finding a specific item in one location. While, memory and common-sense allows the robot to move around the home without revisiting the same space.
• On Device or Hybrid Reasoning: The SSGs is represented in a proprietary low-dimensional representation of the robot’s environment that can be used to ground the LLMs. This allows the robot to reason and adapt in real-time, and perform tasks requiring long-horizon multi-step planning using a complete, accurate and updated model of the environment. A key advantage of this approach is that we can use much smaller, fine-tuned models to do complete or partial reasoning on-device instead of relying on large cloud based models, this enables cost-effective and scalable deployments.
• Situationally aware adaptive behavior