Microsoft haven't even launched this stupid "ai" bullshit yet
and people have already found the very obvious No Shit Sherlock exploits
“Every few seconds, screenshots are taken. These are automatically OCR’d by Azure AI, running on your device, and written into an SQLite database in the user’s folder,”
The database is stored locally on a PC, but it’s accessible from the AppData folder if you’re an admin on a PC. Two Microsoft engineers demonstrated this at Build recently, and Beaumont claims the database is accessible even if you’re not an admin.
Microsoft is currently planning to enable Recall by default on Copilot Plus PCs. In my own testing on a prerelease version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you tick an option that then opens the Settings panel.
Everyone pushing for this needs to Be Made An Example Of, and I'm not joking
It's blindingly obvious that Recall was designed for one reason, and one reason alone: To Put More Laser Targeted Ads In Your Operating System
(well, that and selling new chips I guess)
This bit of it is so insane to me:
However, Recall doesn’t perform content moderation, so it won’t hide information like passwords or financial account numbers in its screenshots. “That data may be in snapshots that are stored on your device, especially when sites do not follow standard internet protocols like cloaking password entry,” warns Microsoft.
Holy shit dude, what are you doing. Do you employ any security experts? Could any of them get a job somewhere else?
Password cloaking is meant to stop a looking-over-your-shoulder attack when using a computer in public. Users constantly uncloak passwords when typing in a private setting – eg, to check how one has misspelled a long passphrase. This is normal and fine because one might assume that one's computer doesn't have a fucking KEYLOGGER INSTALLED IN IT BY DEFAULT FROM THE FACTORY.
Like, this needs to be said very clearly:
There is no secure way to implement this kind of feature.
You cannot be indiscriminately logging everything that passes through the output or input of a user's device. You cannot. Period. Doesn't matter if AI is involved or not, doesn't matter where you're storing it, this is not reasonable to do.
Also I can't imagine this can actually comply with like, even half the various regulations around the handling/transmitting/storing of sensitive data. If you have this feature turned on in a computer in a doctor's office, are you unwittingly breaking a law? In a legal practice? In a government or military setting? Perhaps there will be some switch so you can turn it off but we know Microsoft is going to make it on by default and hide that behind layers of menus, is it going to be that easy for a misconfigured machine to turn into an actual liability issue?
