Prompt Engineering - The Protocol of Intent: The Theoretical Foundation

narrowing the decision space is one of the things that my prompt engineering series is focused on. however… in order to understand why 99% accuracy my not be possible, its important to understand where those hallucinations come from. and the first step to doing that is understanding that there are 2 intended out put vectors for a conversation 1. definite responce 2. explored respnce definite response is something like if you want an email made, and you already know the target, the subject, the tone, the specified contents, etc. explored responce is something i deal with more often when you are exploring a topic of conversation and you dont know where it is going to go, like asking a question you dont know the answer to. most hallucinations seem to come in the second category from my observations. one might call that evidence, and it is, but i dont do empirical studies, so the evidence is still observational from me. but in one of my prompt engineering papers i point out that in exploritory conversation one what is known as conversational entropy. a useful way of understanding and measureing that entropy is imagining that before you start a conversation with an LLM that entropy is at 99%. one of the goals of prompting at all is narowing the entropy vector. bad promopting takes dozens of prompts to move from 99% to 70% most conversations want to be around or under 30%. the direct outputs happen around 10% for example. what does all this mean? the AI has access to ALOT of information. that is not a question. however, what it also has access to is every word in the english language (for example) and no built in direction for conversation. that is actually the minimum that the human in the conversation provides. how does this information tie together for this conversation. when the conversation is fresh, and the entropy is high, the halucinations are more likely to happen, then the halucinations are more likely. as the entropy lowers the likelyhood of halucination. and now we look at the most prominant reason halucinations happen is due to agents being trained for completion in the short term. what this looks like is - answer to question not - correct answer to question now, in the work i do, i had to develop guardrails, because strict rules turned it in to an execution agent and not a conversation/ exploration agent. and this matters for me because most my work is either creative (so i only know then intended direction, not necessarily the exact content) or im answering questions that have no exsisting answer (something else that does not have a predefined output). because of this, in most scenarios, a pure execution agent is of no use. in the back round a pure execution agent can handle handoffs and the like, but for pure exploration (like Protocol of intent) or new problem solving, i need an agent with room for exploration. if if there is no conversational exploration, that cannot happen. so those guardrails look something like - ‘done do this, do that; and here is 3 reasons why’ i have been useing this system successfully for months. and not with 1 agent. i have been coordinating single projects across 5 agents, 4 of which have completely different architectures and training regiments. so it works, and it works for me. does the process i have still have issues? yes, however those issues can be isolated and addressed. which is arguably more important, and what i have been building to on this responce. knowing that an agent can and will hallucinate is important, but what is of equal importance is being able to identify what type of hallucination it is, so that one can then mitigate those types of halucinations. a system that fails is almost guaranteed after a certain level of complexity. the real issue is whether it will fail in understandable ways that arent catastrophic.