{
  "path": "/posts/2025/developing-a-mental-model-for-using-models/index",
  "site": "at://did:plc:mracrip6qu3vw46nbewg44sm/site.standard.publication/self",
  "tags": [
    "language_models",
    "cursor",
    "voice_notes"
  ],
  "$type": "site.standard.document",
  "title": "Developing a Mental Model for using Models",
  "publishedAt": "2025-02-14T17:29:28.000Z",
  "textContent": "I had an interesting realization today while doing a demo building a web app with Cursor.\nI was debugging an issue with an MCP server, trying to connect it to Cursor's MCP integration.\nThe code I was using was buggy, and I'd never tried this before (attempting it live was probably a fool's errand to begin with).\n\nWhen I ran into issues, someone watching asked, \"Why don't you just ask the Cursor chat what's wrong?\"\nThis didn't occur to me because I instinctively figured that Cursor chat (and Claude, the model powering it) wouldn't know what was happening.\n\nThis experience crystallized something important for me: when using AI products and models, we develop mental models of what these systems have available to them about the state of our computing environment and the world.\n\nRaw Models vs. Product-Integrated Models\n\nTake gpt-4o as a \"raw\" model that you use via an API or OpenAI's Playground.\nIt has:\n\n- Its training data and the way OpenAI used that data to train weights in the model's architecture\n- The prompts you send as context\n- The model's own responses (in the case of a multi-turn conversation)\n- That's it (as far as I know)\n\nHowever, when you use AI products (like Perplexity, ChatGPT with search, or claude.ai), you're working with:\n\n- The base chat model\n- System prompts and instructions\n- Additional context (today's date, internet search results, search results from private knowledge bases or datastores)\n- A wide variety of non-standardized, product-specific features\n\nThis distinction matters because ChatGPT with search can give you real-time answers about the world, while a raw model like gpt-4o only \"knows\" what is in its training data.\n\nWhy This Matters\n\nThis distinction of what a model knows isn't obvious to many people using model-based products, especially those less familiar with how the models work.\nHaving your own mental model of what the language model has available to it is foundational to getting good at using these tools.\nIt helps you:\n\n- Discern what is within the model's or product's capabilities\n- Develop intuition for how to use these tools effectively\n- Know when a tool might be able to give you the right answer compared to when it lacks the context to do so (e.g. Claude doesn't know which team won the Super Bowl in 2025, Perplexity does)\n\n!Screenshot of Claude 3.5 Sonnet's response about its knowledge cutoff date, showing transparency about what it does and doesn't know\n\n!Screenshot of Perplexity's search results, showing the additional context gathered from internet search\n\nThis intuition is a soft skill and the ground is constantly shifting as these products are augmenting their capabilities.\nScaling this learning curve is what matters right now when it comes to augmenting your skills with AI.\n\nA Real-World Example\n\nIn my case with Cursor, I intuitively suspected (though wasn't 100% certain) that the Cursor Composer agent wouldn't know how the IDE was trying to make that MCP call to the local server I had running.\n\nI didn't explicitly think this through - it was just intuition developed through experience.\nWhen I went to check if Cursor knew about the MCP settings, briefly, a part of me then suspected that it _actually might_ and that I'd been wrong in my assumption.\n\nThis is where being \"up to date\" on the capabilities of these tools matters.\nThe challenge here is this a moving target and not realistic for people not spending tens of hours per week using \"AI tools\".\nI barely can keep up and I'm working with this stuff everyday.\n\nIt turns out Cursor can't check or modify its own configuration (yet), but this is realistically something that could be incorporated into the product in the near future.\nIn fact, it seems likely it will be.\n\nThe Challenge\n\nThe importance of understanding what different language-model-based products have available in their context window may not be obvious, but it's crucial for developing an intuition about when and how these tools can be effectively applied.\n\nThis is the first step to building your intuition for picking the right tools for your task.\n\nMaking headway\n\nWhen experimenting with a new tool or product, kick the tires.\nAsk the model about things that happened recently.\nAsk the model about itself.\nSometimes models will make things up, but once you start poking around, you begin the process of developing this critical intuition for getting the most of these tools - developing your mental model for using models.",
  "canonicalUrl": "https://www.danielcorin.com/posts/2025/developing-a-mental-model-for-using-models/index"
}