ok so i'm implementing this now cause it's piqued my interest again and is one of those things that's made SO much harder with rollback considerations that ya just wouldn't think about so i'm gonna talk about it!!
#1: we gotta handle multiple calls to play audio
so one of the big issues main issue for rollback and audio is unsurprisingly that the game state has to roll back and re-simulate the same game logic sometimes several times over. this includes situations like frame 5 saying "hey play this sound effect", then on frame 6 we get remote input for frame 3, and then re-simulate frames 3, 4, 5, and 6, where frame 5 still says "hey play this sound effect", and then frame 7 we get input for frame 4 and re-simulate frames 4, 5, 6, and 7, and you get the point. we've gotta have some kinda system in our sound player that knows when it's being asked to play a sound it's already started playing!
but surely we can just ignore calls to play sounds when we're re-simulating, right?
#2: nah we gotta handle playing sounds that we didn't know started playing earlier, too
say on frame 5 our opponent throws a punch and that plays a sound effect. when we get that punch input on frame 8 and roll back to frame 5 to re-simulate, we now know a sound effect should have been playing 3 frames ago, so now our sound player needs to know when a sound should have been played, and play it starting 3/60ths in (idk if i'll bother doing this part cause it might just sound weird at that small of a delay?)
#3: stop playing sounds we thought would exist but actually don't
another scenario is say you start a big attack that plays a sound effect for it's whole duration, so as soon as you start that attack, the sound effect starts. and THEN you get a rollback with old inputs that say that actually the opponent hit you before you could start that attack so you should have never been able to start it and it now needs to stop playing. you've gotta be checking that while you're in re-simulation that all the sounds you were playing in that time frame ARE requested to be played again, otherwise it means that they weren't actually supposed to play in the first place and need to be stopped. which sounds like no big deal, but
#4: sound playing instances can't be referenced by the game simulation cause everything's just a bunch of structs that get modified by systems but can't in themselves hold any persistence over time
yea :/
ok so how do u do it
this is me saying this before i've implemented it and worked it all out 100% but the way i'm going to approach it is by keeping track of all Play() requests with a struct containing the entity ID that called it, the frame it was requested to play on, and a reference to the sound's data. by default, any request to play a sound will just be played or stopped at the end of the frame, unless a rollback has happened, in which case we compile a list of all the sounds asked to play during any of the re-simulation frames, and as we get calls to play them again, remove them from the list. any sounds left in the list at the end will be stopped. any sounds asked to play that weren't part of that list will play anyways, taking into account the frame they were supposed to start playing vs the current frame (maybe, if this doesn't end up sounding weird)
and that's all i've been thinking about!! this is only part of the audio system as it relates to the rollback part of the game, so i've still gotta make the whole sound effect data scriptable object tools, and the audio source object pooling system, maybe dynamic left/right panning relative to the camera? idk sounds fun but i'm sleepy so see ya next time
