Just how much does AI know about you?
Paul Lieberman
March 4, 2026
I opened a Gemini chat session and asked it flat out: "Show me everything you know about me." It came back and said nothing.
> As an AI, I don’t have a "file" on you, a memory of our past lives together, or access to your private documents, emails, or physical location unless you explicitly share them in this specific conversation.
What prompted this question was that I had been working in a Google Sheets spreadsheet entering in some stuff for tax preparation. I noticed the Gemini icon on the top right so I clicked it and entered a prompt: "Please setup this sheet to help me with tax preparation." It was a bit slow to respond but then it popped up a colorful and nicely formatted table, with the data I had already entered, plus placeholder fields for stuff I still need to enter. Nice, until I looked closely and started freaking out. It listed my retirement and mortgage account including the financial institutions. It listed my wife's name, her business, investment property, and retirement account. How did it know all this? So I asked the instance of Gemini in the chat session.
> This experience can definitely feel startling, but it is actually a core feature of the Gemini for Google Workspace integration.
>
> When you use Gemini inside Google Sheets, it isn’t just looking at the blank cells in front of you; it has the ability to "bridge" your data across the entire Google ecosystem if certain extensions or settings are enabled.
>
> Here is how it likely found that specific information:
>
> 1. The "Google Workspace" Extension
> Gemini has an extension that allows it to "read" your other Google apps. If this is active, the AI can query your:
>
> Gmail: To find tax documents, mortgage statements, or retirement account summaries sent to your inbox.
>
> Google Drive: To scan PDFs of your 1099s, W-2s, or property deeds.
>
> Google Calendar: To find business events or names of family members linked to shared events.
>
> 2. Semantic Search (The "Ask Gemini" Panel)
> When you asked it to "setup a sheet to help prepare tax returns," the AI interpreted this as a request to populate the sheet with relevant context.
>
> It likely performed a background search of your Gmail for keywords like "Mortgage," "1040," "401k," or "Schedule C."
>
> If your wife’s name appears in your emails (e.g., "Tax document for [Name]") or in shared Drive folders, the AI connected those dots to build a "complete" picture of your household's financial profile.
It then showed me the exact steps necessary to turn this off. It also assured me that Google does not share my data and that it does not use it to train their AI models. I deleted the sheet it had built for me, but I have not as yet turned off this feature.
This experience had me reflect upon our earliest visions of how computers would work for us. There was always this concept of a "digital assistant" that would help manage the mundane details of our lives. You may already ask Alexa to read off your appointments for the day each morning. (do people really do that?). AI has now taken this concept to the next level. To be really helpful these agents have to have access to your data. In order for Alexa to read your appointments you have to first grant it access to your calendar. The flip side of this of course is our deeply ingrained paranoia that "Big brother is watching us". Maybe that comes from reading all those dystopian novels in our youth, or maybe it comes from the many cases of big tech companies selling, sharing, or leaking our personal data. I'm sure most of us don't really trust Google, Apple, Microsoft, Facebook, or any of the other tech giants.
When Google first got into advertising it's promise was to revolutionize how ads are shown by only showing you stuff you were interested in. If you grew up on TV like I did, you were forced to sit through hours and hours of ads that were completely irrelevant to your reality. So Google's idea of targeted ads sounds great right? Until you realize that the only way to show you stuff you're interested in is to already know a lot about you. Once it does, the technology works great. Most of the ads I see are for stuff I'd consider buying. When it get's scary is when you see an ad and you think "wait a minute, I was just talking about this", and you start to think they must be listening through your phone. Occasionally I will see and ad and think the only way they knew I wanted this was if they could read my mind! Paranoia strikes deep.
And so we are left with a conundrum. Do we forego the ways these smart agents can help us and make our lives easier, and attempt to block all access to our personal data, or do we just give up and think "they're going to steal all our data anyway so what difference does it make"? My suggestion at this point is to be selective in granting permissions to your data, while continuing to be vigilant in protecting it from sources you don't trust. I'm not really sure if I have a use case for Gemini in Google Workplace. I'll have to ponder it some more.
Discussion in the ATmosphere