REGRESSION: Free ChatGPT accounts unable to invoke apps!
Conversation 2 (conv2.png) — App invoked, but only on the second attempt: Same type of request. The app was not triggered on the first message. Only after the user explicitly challenged ChatGPT did the app actually get called and return real PagesJaunes results.
Pattern observed:
On free-tier accounts, the app is consistently skipped on the first invocation, even when pinned in the input bar
The model silently falls back to web search
When called out, it may either hallucinate app usage (conv1) or finally trigger correctly (conv2)
On paid accounts, the app triggers correctly from the first message
The hallucination case in conv1 is particularly concerning from a developer perspective, as end users have no way to detect that the app was never actually called.
Hope these help. Happy to provide more test cases if needed.
Discussion in the ATmosphere