Thing about Roko's Basilisk, The World Is A Simulation, and other seemingly-scientific thought experiments is that I believe we can criticize them the same way we criticize designs of perpetual motion machines: they're unbalanced. A car that dangles a magnet in front of itself to pull itself forward may seem to proceed from sound scientific principles: the magnet pulls the car forward, the car in turn pushes the magnet. The reason it doesn't actually work is because in real life the car is exerting just as much of a backwards attractive force on the magnet as the magnet is exerting a forward force on the car. The reason we think it should work is because of our intuition about donkeys and carrots.
Likewise the simulation hypothesis proceeds from the unbalanced assumptions that we are the purpose of the simulation, and that the creator is even remotely similar to us. And if we are not? If ourselves and the entire observable universe are merely an emergent property of a pattern-making machine designed to simulate something else? Then this is no different from simply having a universe that follows it's own natural laws, and the question "are we living in a simulation" becomes uninteresting. And yet these considerations are easy to overlook because the simulation hypothesis feels right, as it confirms the intuitions of many people's religious, specifically Christian, upbringing.
Same with Roko's Basilisk. Why, of all the possible superintelligences that could ever be built, assuming such a thing is even possible, why is it more likely than not that it would be the one who behaves like the Jealous God so many of us grew up fearing? There's no argument you can make that does not end up privileging those causative mechanisms that Just Sound Right over those that don't.
It really stands out to me, as someone who was never raised Christian, how often people assign very specifically Christian ideas to the general concept of divinity. (Or whatever higher power, if you want to call it a computer or a simulator or the will of Nature, that you use to mean the same thing. As soon as you start talking about what nature "intended", guess what, that's just God again)
"God exists" does not have to imply: that there is one God, that you can only follow one God at a time, that God created the universe, that God has always existed, that God is omniscient/omnipotent/omnibenevolent/omnipresent, that God can read your mind, that Heaven and Hell exist, that the afterlife is different for believers and nonbelievers, that sins are punished and good deeds rewarded, that being a nonbeliever is a sin, that there is a single "main" holy book, that congregations are hierarchically managed, that congregations meet weekly, that God is particularly concerned with proper marriage and childrearing, that there will be an apocalypse, that believers are expected to convert nonbelievers...
once you start noticing it you realize the "Hanukkah is Jewish Christmas, Mohammed is Muslim Jesus, Nirvana is Buddhist Heaven" mentality is everywhere.
