AI Tools for API Companies: AI Needs Context, or How Our Auto-Generation Failed Spectacularly

The AI agent revolution is here, and with it, everyone's talking about agent tools. As a location API company, we've spent years perfecting location tools for human developers. So when the Model Context Protocol (MCP) promised to make our APIs accessible to AI agents, we figured it would be straightforward. Just auto-generate a MCP server with a healthy selection of tools like we do with SDKs, right?

Wrong. Our initial attempts failed spectacularly, and we learned some unintuitive lessons: the biggest of which is that the context in which AI consume APIs matters enormously.

What Are AI tools?

Fundamentally, tools are how language models interact with external systems. Normally, when a user asks an AI assistant to "find the best route from Seoul to Busan," the model can't give anything more than a vague summary. With the right tools, however, AI can use APIs to lookup addresses, provide real-time directions, and even map the resulting path.

The Model Context Protocol standardizes how AI models discover and interact with these tools. MCP servers act as bridges between AI systems and external systems, defining a consistent interface for tool discovery, parameter specification, and response handling. They're gaining traction because they solve a critical infrastructure problem: how to reliably connect AI agents to the vast ecosystem of existing APIs.

As engineers, this looked a lot like a pattern we already knew: SDKs for developers.

The "Obvious" Solution

Given we'd already spent years building solid SDKs generated from hand-crafted OpenAPI specifications, it seemed logical to start there. From this foundation, we auto-generated our first MCP server.

From start to finish, the whole process took a couple hours. We had a working MCP server, complete with tools for geocoding, routing, and creating maps. It seemed perfect.

Then we tried our prompt:

Find the best route from Seoul to Busan.

What Happened

Immediately, our quick win began showing its limits. The models were using our tools in subtly but fundamentally flawed ways, and that’s the worst kind of failure.

Location APIs, even in the simplest form, are complex. Most endpoints offer dozens of inputs and parameters. For humans, we solve this by offering clear documentation and guides. Unfortunately, this complexity quickly confused models. In the maze of options, they couldn't figure out which parameters mattered. When finding a route should it optimize for time or distance? What about tolls? Should it choose car, bike, truck, some other mode of transportation? This was further compounded by models not even knowing which questions to ask to feed the tool the right parameters.

Even when models did choose reasonable parameters, they struggled with complex responses. Our APIs return detailed JSON objects optimized for programmatic consumption, but AI models often missed crucial information or got overwhelmed by unnecessary details.

Token consumption quickly became problematic. Our mega-tools with extensive parameter lists consumed large amounts of context space, leaving less room for user input and driving up model costs and, more importantly, efficiency. A single tool description could easily eat 500+ tokens before the model even chose which tool it should use.

The pattern was clear across every endpoint we tested: APIs designed for human developers, no matter how well-crafted, don't automatically translate to effective AI tools. Our fundamental assumption—that well-built, well-documented APIs automatically make good AI tools—was completely wrong.

What We Were Actually Building

We had to take a step back and consider what we were actually trying to do.

When we started, we assumed we were building another SDK, but we weren't. We were building something fundamentally different: contextual tools specifically structured for AIs. The key insight we missed initially is that AI models consume APIs through a completely different lens than human developers.

Human developers read documentation, understand business logic, and make nuanced decisions about parameter selection. AI models need tools designed specifically for their consumption patterns: focused, well-described, and optimized for the constraints of language model reasoning.

What's Next

This failure taught us much of what we needed to know about building MCP servers that actually work. After our auto-generated approach failed, we had to retool our entire strategy.

In our next article, we'll share the design principles we discovered for creating AI-friendly tools, the unexpected ways building MCPs improved our underlying APIs, and the new capabilities this unlocked that we never had to build ourselves.

For now, if you're looking at MCPs for your own API company, remember this: the obvious solution rarely works when you're building for AI consumption. The patterns that make APIs great for human developers need fundamental rethinking for AI agents.

Want to stay ahead of the AI tooling curve? Subscribe to our newsletter for insights on building APIs that work seamlessly with AI agents, MCP server best practices, and the evolving landscape of AI-native development.

If you're impatient and want to see our principles in action right now, check out our MCP Server on GitHub. And join our Discord to discuss your own MCP experiences with other developers tackling these same challenges.