Is there an example with an LLM mounted on an in-premise server?

[EmmanuelMess]

Is there support for mounting custom LLMs on in-premise servers? Can the LLM be brought up from a launch file? What are the requirements on the LLM?

[stex2005]

Great question! The nice thing about MCP is that it doesn’t really matter where your LLM is running, e.g., cloud, on-prem, or even on your laptop. The MCP server just exposes ROS topics, services, and actions. On the other side, it’s your agent/runtime (the MCP client) that needs to know how to talk to both the LLM API and the MCP server. So you can point it to vLLM, Ollama, LM Studio, or any local API you host yourself.

In terms of requirements: the LLM itself doesn’t have to be modified or “embedded” with MCP. The only real requirement is that the runtime you’re using supports structured tool/function calls (JSON in/out), since that’s how MCP describes ROS interfaces. That’s a much lower requirement than “re-writing the model,” but it does mean some LLM APIs are a better fit than others.

At the moment we don’t ship a launch file to start the LLM itself. Most people just run the LLM separately (via Docker, systemd, etc.) and then bring up ROS + the MCP server as usual. That said, a sample launch setup that ties it all together would be a great addition, something we’d be excited to see contributed.