Skip to content

Conversation

@chant29
Copy link
Contributor

@chant29 chant29 commented Dec 9, 2025

Summary

Added a map analysis tool that allows the LLM to directly read and utilize Nav2 map data, and implemented read/write tools for semantic maps.

Added tools

Tool Purpose
get_verified_maps_list Returns the list of all available verified map specifications so the LLM can check which maps have predefined context before loading them.
get_verified_maps_info Retrieves the detailed specification and usage context for a pre-verified map after confirming it exists in the verified list.
write_map_location Adds or updates a semantic location entry (name, description, x, y, yaw) in the selected map’s YAML specification file.
draw_map_axes Adds world-aligned +X/+Y axes and grid lines to the saved OccupancyGrid map image (received_map.png), drawing the axes from the world-frame origin using the height, width, and resolution supplied by the Nav2 map server, resulting in an annotated map suitable for LLM-based spatial reasoning.
analyze_previously_received_map Loads the latest processed map overlay (received_map_overlay.png) with axes and grid already drawn, and returns it in ImageContent for LLM-based visual analysis.

Required Modifications for This Tool

Tool Modifications
subscribe_once&subscribe_duration Added msg_type as an additional parameter to parse_input.
function Modifications
parse_input If the input msg_type ends with OccupancyGrid, call the parse_map function.
parse_map The OccupancyGrid values (0 = free, −1 = unknown, 100 = occupied) are mapped to grayscale intensities (127, 255, and 0) and export the result as an image(received_map.png).

How to Use

When Nav2 is running, providing the LLM with scripts such as ‘retrieve the map image and draw the axes’ or ‘fetch the map image’ causes it to invoke the given tools. Through these tools, the LLM can analyze the map image data and understand the map’s structure and coordinate system.

Example 1 : Analyze the map image to determine what kind of space this is.

Screenshot from 2025-12-09 19-40-27

Example 2 : Interpret the user’s requested destination, calculate its coordinates, and navigate to that point.

good1_2_2

@chant29 chant29 marked this pull request as draft December 9, 2025 10:59
@chant29 chant29 marked this pull request as ready for review December 9, 2025 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant