Wild times in the “old school industry” … everywhere agentic AI fantasies disruptions, which are supposed to be seamlessly integrated on the basis of holistic approaches, while keeping the human in the loop and yet still scaling. AI becomes a board-level topic – middle management executes and across the company new metrics are drawn up on a large scale, making some kind of success dependent on the use of AI. Processes and process knowledge quietly disappear (because the AI now takes over). Plenty of AI use cases are sought, found, and implemented. The content of management briefings is generated en masse in the prompt landscape. As a double check on the briefing content, AI is then used to generate a list of questions that the creators believe will also come from the briefing’s target audience. Welcome to the world of the perfect circle … an LLM interpolates the content of the management briefing from the operational-tactical level, and this is then in turn validated by a second AI, which generates a comprehensive (and mostly very naive) set of questions. AI talks to AI … with the human in the loop, even though at this stage he’s just an extra.
But actually this wasn’t supposed to be a rant on AI in general at all, but rather a little tour through the essential steps of AI deployment in companies. So …. here we go …
Let’s start a Band!
Commonly, the journey with your own AI begins with a line of code and a credit card. Whether it’s a first tentative business application or the casual “let me just play around with AI a bit” at home … your program sends text to a provider’s model, the model sends text back. Architecturally, this is about as exciting as a weather service from the 2000s: external service, stateless, API key … no witchcraft involved. In boardroom slides, this stage is typically called “AI strategy.” It’s the first of many stages, and things get complicated the moment the model takes over the process.
Knowledge comes at runtime
The model doesn’t know your company and, of course, is reluctant to admit it. So the first real layer in your AI ecosystem emerges: a system prompt for the role and a RAG for the knowledge. You break down documents, store them as embeddings in a vector database, and inject the relevant sections into the request at runtime. The knowledge lives with you, and the model only sees the excerpt it needs to generate an answer. Now you have some kind of knowledge base that wasn’t scraped together from the AI slop on the internet. You’ve successfully overcome this hurdle… Congratulations!
The model takes over the process
Up to this point, your code called the model and controlled it with instructions on what to do and how to do it. With tool calling, the game is turned around: the model now decides which function it calls. It then reads the result and decides anew, until the task is completed according to your goal description. MCP standardizes the connection to information resources, and suddenly the whole thing is called agent-driven AI. This step is the actual architectural break. The program or AI execution flow is no longer defined in the code itself, but instead emerges at runtime for the specific task. A system that could previously only talk incorrectly can now also act incorrectly, in practice.
Trust becomes architecture
Whoever acts needs rights, and rights need an identity. The user’s identity travels via OAuth through every tool call, with roles limiting what the agent is allowed to see. Then there’s prompt injection: a model cannot reliably separate data from instructions, so every email and every PDF is a potential command. Guardrails do filter inputs and outputs, but it’s more reliable to fundamentally grant the agent little permission and have humans approve risky actions. This is Zero Trust, this time against your own software.
From prototype to operation
Every request costs money. So we optimize: between the application and the models, we insert a gateway with cache and router: small models for simple tasks, large ones for heavy tasks. What was once a single model has thus become a fleet. That leaves the question of quality. Classic tests don’t work well with non-deterministic systems, so traces and logs show what the agent did, and evals check every change. Prompts must therefore be treated like code, with versioning, approval, and all the formal fuss that comes with it.
The Platform
The next problem is called scalability. Five departments build six agents, each with its own vector database and its own access credentials. So you centralize: a catalog for agents and tools, shared memory, unified identities, and a control layer for policies, costs, and auditing. Agents commission agents. The platform has its own team, its own roadmap, and its own budget. The model that started it all is now one of the most easily replaceable components.
What remains? If you didn’t duck your head fast enough when AI implementation responsibility was being handed out and ended up landing a nice future project, first check which tier it’s on and who in the hierarchy pays for the next one. The LLM provider’s price list only covers tier 1; the remaining five/six tiers appear further down the org chart.
BTW: The line of code from the beginning, by the way, still exists. It now has a Product Owner.
