Microsoft haven't even launched this stupid "ai" bullshit yet
and people have already found the very obvious No Shit Sherlock exploits
“Every few seconds, screenshots are taken. These are automatically OCR’d by Azure AI, running on your device, and written into an SQLite database in the user’s folder,”
The database is stored locally on a PC, but it’s accessible from the AppData folder if you’re an admin on a PC. Two Microsoft engineers demonstrated this at Build recently, and Beaumont claims the database is accessible even if you’re not an admin.
Microsoft is currently planning to enable Recall by default on Copilot Plus PCs. In my own testing on a prerelease version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you tick an option that then opens the Settings panel.
Everyone pushing for this needs to Be Made An Example Of, and I'm not joking
It's blindingly obvious that Recall was designed for one reason, and one reason alone: To Put More Laser Targeted Ads In Your Operating System
(well, that and selling new chips I guess)
This bit of it is so insane to me:
However, Recall doesn’t perform content moderation, so it won’t hide information like passwords or financial account numbers in its screenshots. “That data may be in snapshots that are stored on your device, especially when sites do not follow standard internet protocols like cloaking password entry,” warns Microsoft.
Holy shit dude, what are you doing. Do you employ any security experts? Could any of them get a job somewhere else?
Password cloaking is meant to stop a looking-over-your-shoulder attack when using a computer in public. Users constantly uncloak passwords when typing in a private setting – eg, to check how one has misspelled a long passphrase. This is normal and fine because one might assume that one's computer doesn't have a fucking KEYLOGGER INSTALLED IN IT BY DEFAULT FROM THE FACTORY.
Like, this needs to be said very clearly:
There is no secure way to implement this kind of feature.
You cannot be indiscriminately logging everything that passes through the output or input of a user's device. You cannot. Period. Doesn't matter if AI is involved or not, doesn't matter where you're storing it, this is not reasonable to do.
“We expect organisations to be transparent with users about how their data is being used and only process personal data to the extent that it is necessary to achieve a specific purpose. Industry must consider data protection from the outset and rigorously assess and mitigate risks to peoples' rights and freedoms before bringing products to market.
“We are making enquiries with Microsoft to understand the safeguards in place to protect user privacy.”