API Orchestration and AI Agent Approach

Hello everyone, (Note: I’m intentionally not sharing the platform link here to respect community guidelines.) Lately, we’ve all been trying to integrate Large Language Models (LLMs) into our projects. Creating a simple chatbot is the easiest part. However, when it comes to the bot interacting with the outside world (e.g., retrieving data from a database and sending a POST request to another API), it turns into complete chaos. While libraries like LangChain are very comprehensive, they can sometimes be unnecessarily cumbersome for simple orchestration. Function call structures can break with the slightest command change or corrupt the expected JSON format. Those who struggle with “Invalid AI format” errors when you reach speed limits will understand exactly what I mean. To manage this process, I decided some time ago to try a completely different architecture and developed my own “Middleware Language” to work between APIs and LLMs. How Does the Architecture Work? (Application Wrapped in LLM) First, let me clarify that this is not a hardware invention or a new model trained from scratch. The system is actually an application “wrapped in LLM”. Here, the LLM only acts as a regulator. This middleware language I developed acts as a translator between the natural language and the backend (REST API). You only need to provide the system with your API documentation (or JSON schema). When a request comes from the user, the LLM uses this middleware language to decide which API endpoint needs to be triggered, matches the parameters, and returns a dynamic action (routing, UI update, etc.) to the user depending on the result. It can be successfully orchestrated regardless of how complex the API is. So we use the LLM not as an all-knowing repository of information, but as a logical conductor using the “API tools” at its disposal. This reduces token costs while minimizing the risk of hallucinations. What Can Be Developed with This Middleware Language? If you transform an LLM from just a text generator into an orchestrator, creating “Agent Workflows” becomes very practical: Autonomous E-Commerce Agents: Assistants that don’t just say “This is our return policy,” but also take the customer’s order number, trigger the shipping API in the background, check the shipping status, and then take action by saying, “If you’d like, I can also add these products, similar to your previous order, to your cart.” “Lead Generation” Bots: “Invisible salespeople” that analyze the visitor’s intent (tendency to buy) while chatting with visitors on corporate B2B sites, instantly extracting information such as email addresses, titles, or company names shared during the conversation, and saving it directly to the CRM API without any manual coding. Internal Tools: Agents that connect to internal databases or applications like Slack/Jira. Instead of employees having to use complex dashboards, this project enables chained workflows that simultaneously trigger multiple internal APIs, allowing them to say things like, “Get last week’s sales data and email it to the marketing team.” Do you think this project will translate into value? Note: I’m intentionally not sharing the platform link here to respect community guidelines, but happy to share more details if there’s interest.