audioerf

heard this was gonna be good 4 art

art goblin, sims addict. 30+ . professional tv animator, somehow. lol



surasshu
@surasshu

i am actually very sure netflix (and probably by extension the rest) is making up their numbers, like absolutely 100% would put money on it. like, they literally got into the news constantly by saying "our [random movie i have never heard anyone talk about. and i talk to movie people CONSTANTLY] is the most-streamed movie ever!" with no fact-checking required (or even possible). like literally "trust me bro". i didn't think about what would happen if they ever had to reveal that info for the sake of paying out royalties though...


Osmose
@Osmose

Analytics is really, really hard. You gotta come up with the right numbers to measure in the first place, measure them using code running on millions of devices with different operating systems and software, deal with internet connections of all different speeds and reliabilities, receive all these events and store them reliably, then fetch the data and aggregate it correctly, etc.

Then you still need to verify those measurements against a second independent metric to ensure that your measurement instrumentation isn't broken in some invisible way, and even then that only increases your confidence in the measurement and doesn't actually confirm it's true.

Most companies skimp out on all of this because it doesn't take very much effort to produce basic, flawed metrics, and the effect all the effort above has on the metric isn't terribly visible until you've already made decisions based on the metric and found that those decisions didn't have the effect you expected. Without intentional thoroughness, it can look like a waste of money and time since the minimal effort is producing numbers that look the same as the high-effort numbers.

Like I 100% believe Netflix had metrics telling them these stinkers had a bajillion views. I also 100% believe the metrics they based that off of was some shit like "count 1 view for every request made to the URL for this movie, regardless of how long they actually watched it, whether the request actually succeeded or not, or whether they were even logged in as a user".


You must log in to comment.

in reply to @surasshu's post:

I was going to make a goof reply with some extremely bullshit metric they probably count, but on further investigation I would not be surprised if they "Count every time a trailer autoplayed for a movie because you hovered over it for 2 seconds as a view" as a metric.

in reply to @Osmose's post:

Another thing is, you need different quality metrics for different purposes. Shitty-quality data is surprisingly robust for making business decisions as long as a) you're paying attention to directional shifts rather than absolutes b) everything is the same shitty.

I mean, it could also be a clownshoes shitshow over there and it's all on fire by accident. Very real possibility. I'm just saying that even in the best-case scenario data will still be crap because data quality may not be worth the engineering effort!

But accounting data must be precise and reliable to a far greater extent than business intelligence data. It's quite possible the infrastructure to produce that data doesn't exist, and potentially can't exist because internet data is inherently uncertain.

yeah good clarification--i don't necessarily think netflix just said "whatever, we'll just make it up" (although i also wouldn't put it past them) but i do think they put their self-funded movie in the auto-play at the top and then counted every auto-play as a full watch

Pinned Tags