rocket-propelled-shark

Rocketing his way into your heart.

Another funny internet artist



Turfster
@Turfster

Microsoft haven't even launched this stupid "ai" bullshit yet

and people have already found the very obvious No Shit Sherlock exploits

“Every few seconds, screenshots are taken. These are automatically OCR’d by Azure AI, running on your device, and written into an SQLite database in the user’s folder,”

The database is stored locally on a PC, but it’s accessible from the AppData folder if you’re an admin on a PC. Two Microsoft engineers demonstrated this at Build recently, and Beaumont claims the database is accessible even if you’re not an admin.

Microsoft is currently planning to enable Recall by default on Copilot Plus PCs. In my own testing on a prerelease version of Recall, the feature is enabled by default when you set up a new Copilot Plus PC, and there is no option to disable it during the setup process unless you tick an option that then opens the Settings panel.

Everyone pushing for this needs to Be Made An Example Of, and I'm not joking

It's blindingly obvious that Recall was designed for one reason, and one reason alone: To Put More Laser Targeted Ads In Your Operating System

(well, that and selling new chips I guess)


Osmose
@Osmose

Like the very concept of a log of everything you've seen on your computer is bad, straight up. This would be equally bad if it was doing OCR + text search without an LLM, but LLM hype has somehow disabled that part of the "this is a bad idea don't do it" instinct.

Any such log makes any compromise of a computer at any moment in time as bad as a compromise of the computer at the worst possible time. If I hack your computer and get your Recall database on a Saturday night while you're gaming, it's equivalent to me hacking your computer right when you were looking at your bank website or social security info or password manager.

(Actually password manager is an interesting case because they generally mask passwords, as do password fields. But you know what I mean.)

There's not much mitigation that makes it acceptable, either. Encryption at rest matters for, like, a breach of a remote server storing your data. It's less useful when all the plausible attack scenarios involve local access to the computer operating on the data, as either you're always entering a password to decrypt the data live OR all the info needed to decrypt the data is on that same system that is already compromised. This is why Yubikeys/TouchID are so useful—they require an interaction in the real world that users are actually willing to live with (usage of Recall would be extremely low if it constantly requires a password).

The only direction that might make something like this acceptable would be preventing saving of sensitive data in the first place, but determining what is sensitive data is so reliant on context that I don't see it as solvable—but I would not be at all surprised if the response to all this uproar would be Microsoft having an LLM scan the incoming screenshots and generate a response to the question "Does this contain sensitive data? (Giant list of sensitive data types)" and use that to determine what not to save. Which fuckin sucks.

Apps could have some sort of flag to indicate that they're in "private mode" and should not be recorded (mobile platforms already have some stuff like this, Firefox for Android prevents screenshots in private mode) but that wouldn't scale and has similar issues around determining what content is sensitive and what isn't.


Osmose
@Osmose

This is why Yubikeys/TouchID are so useful—they require an interaction in the real world that users are actually willing to live with (usage of Recall would be extremely low if it constantly requires a password).

And now today:

In addition to making Recall an opt-in feature, Microsoft’s Davuluri also writes that the company will make changes to better safeguard the data Recall collects and more closely police who can turn it on, requiring that users prove their identity via its Microsoft Hello authentication function any time they either enable Recall or access its data, which can require a PIN or biometric check of the user’s face or thumbprint.

Not that this makes it okay—PINs can be compelled or stolen, biometrics have unique vulnerabilities, etc. The idea of recording and saving everything shown on a screen is bad at it's very core in a way that can't be mitigated. But the opt-in bit at least makes it easier to avoid.


Turfster
@Turfster

Anyway, still not good enough of course.

The whole thing needs to be thrown in the fucking dumpster, where it belongs, along with everyone that pitched it in the first place and pushed the clusterfuck version to production.

If you think some of those fucksticks isn't going to sneakily turn it on in an update down the line, I've got a bridge to sell you.


xinjinmeng
@xinjinmeng

Recently I started using ZenDesk for website adminstration. It comes with an AI assistant. There's an option to turn it off, but if I try, I get a message that this cannot be deactivated because it's part of my "package". Presumably I would need to pay more to have it removed? 🤔 I'm guessing if I call customer service, I'll get a string of several people who will never tell me it can't be removed but also lack any way to remove it. 🙁

One of my phones is an Android that I purchased at a deep discount, using a student-rate. I've had it for years. It comes with Facebook and adverts pre-installed. I cannot remove these adverts, they are not an option in this build. I've called customer service about it, it is simply not an option to move the SIM card to a new phone. Either use it with this kind of invasiveness or don't. ☹️

I'm calling it now - Microsoft's response to make this feature 'opt-in' is because the next step is selling machines where the mere act of accepting the machine is opting-into being surveilled. That's what EULAs are for. 😠


You must log in to comment.

in reply to @Turfster's post: