Microsoft haven't even launched this stupid "ai" bullshit yet
and people have already found the very obvious No Shit Sherlock exploits
“Every few seconds, screenshots are taken. These are automatically OCR’d by Azure AI, running on your device, and written into an SQLite database in the user’s folder,”
The database is stored locally on a PC, but it’s accessible from the AppData folder if you’re an admin on a PC. Two Microsoft engineers demonstrated this at Build recently, and Beaumont claims the database is accessible even if you’re not an admin.
Microsoft is currently planning to enable Recall by default on Copilot Plus PCs. In my own testing on a prerelease version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you tick an option that then opens the Settings panel.
Everyone pushing for this needs to Be Made An Example Of, and I'm not joking
It's blindingly obvious that Recall was designed for one reason, and one reason alone: To Put More Laser Targeted Ads In Your Operating System
(well, that and selling new chips I guess)
I believe this kind of features could be useful to people. I know I would DIE to be able to ask my computer "hey what was that one website with a red color scheme I saw the other day?" because my memory works very well with visuals but terribly with text.
BUT
Microsoft is not the OS vendor I would have trusted to implement it safely. And unsurprisingly they confirmed my suspicions.
IMO an OS vendor would need the following to do this as securely as possible (regardless of whether or not you think they should):
- encrypt that shit at rest. It sounds like this is what Microsoft intended to do but clearly they didn't but also it might be a perk of this feature being accessed earlier than the planned release date. I'm willing to bet the final version will properly encrypt that SQLite db and/or rely on BitLocker like they said BUT without my second point, it's still messy
- Apple has a very solid implementation of full-disk encryption with FileVault that is both effective and fully transparent as a user. It has been available for a very long time and in 99% of cases there's no performance/reactivity hit to enabling it. IMO every Mac user should enable it especially on laptops.
- Idk the state of full-disk encryption on Linux, especially when it comes to implementations that use on-device encryption chips but surely something is out there.
- Have systems that would let the OS say "no, actually, NOBODY but [feature] can access this and there's no exceptions". If you truly want to make that feature, then its artifacts should be treated with as much care as biometric data.
- Again, macOS/Apple have multiple systems in place to ensure that stuff would stay private. The Secure Enclave is an example of such a system, but it's a "write only" type thing so it would probably be impractical for a Recall-like feature. Something akin to System Integrity Protection would be a better fit and again, works pretty well on macOS for most (non-power) users.
- Idk the Linux ecosystem enough to know if such systems exist. I feel if any distro said "actually
/varis read-only by default now" there would be riots so perhaps this doesn't exist. I don't know.
Obviously even with that you're not safe from like, a flaw in the encryption that could get exploited by malware and such but yknow. That'd be better than whatever the fuck Microsoft is doing here. At least Windows 11 will soon enable BitLocker by default but it's baby steps.
My point I guess is that unfortunately due to a lot of reasons and also bad decisions they made with Windows for years, Microsoft is a terrible candidate to implement such a feature. Their saving grace here is that it'll be enabled (at first) only on "Copilot+ PCs" because of the requirement for powerful enough hardware to run their models on-device (which is a GOOD part of the feature, by the way, hitting the cloud would both be a privacy nightmare and also terrible user experience/cost Microsoft a ton of bandwidth/compute). It will be interesting/maybe worrying to see how the situation evolves when non-Qualcomm-ARM devices start having access to that feature.

































