Oh god how do I keep track of all these open files and tabs
Work continues on OpenISACT, my source port/decompilation of Creative Labs ISACT. I wanted to write a bit about the tools at hand and my process and plans thus far.
I'll also be writing a second post today, after this one, about something I came across that was initially perplexing where I found answers in a very unexpected place. That was originally gonna be part of this post as well but it was unrelated enough that I thought it best to just keep it seperate.
Resources
For a project like this I really need all the help I can get and thankfully I've been able to piece together a jumbled collection of resources which I can use to get the job done.
Psychonauts Linux
The main and most important resource is the Linux release of Psychonauts by @icculus. This serves as the main source for the code which I'm using to reverse engineer ISACT. It has many odd quirks which make it perfect for the job.
The main "quirk" is that the game is built in debug mode. It has full symbol information which makes navigating the binary in Ghidra a breeze (well, relatively speaking1) because we get various names and other goodies. Instead of code that looks like this:
We get code that (after setting up variable names and such) looks like this:
It's a lot more immediately clear what's happening.
Interestingly, Icculus made the choice to forego the library seperation and merge ISACT in its entirety into Psychonauts itself. This does make it hard to untangle AudioDrv from isactwin but it isn't too bad and the seperation will become clear as we move forward.
This does have a caveat though - we don't really know the extent of the modifications Icculus might've made to ISACT are on his end (and he DID make them. IcculISACT links with SDL for fuck's sake). Only way to know for sure aside from obvious stuff would be to comb through the Windows ISACT DLLs simultaneously with the Linux binary and I don't really want to do that so OpenISACT is gonna end up specifically being a decompile of Icculus' ISACT as used in Psychonauts, with some blanks filled in if they arise and I can figure out what to fill them with. (I'm probably gonna try and get rid of the SDL dependency for instance, at least on Windows, to bring it closer to the original software)
THE SPECULATION UNDER THIS COLLAPSIBLE IS WRONG
The second quirk of the Linux port is one that's very lucky and quite specific to our purpose - the inclusion of ISACT *inside the game executable itself*. This one is a bit a weird. See, in the Windows version and ISACT SDK, ISACT has two external DLL libraries: - AudioDrv.dll: Provides an ISACT rendering interface. This is what ISACT uses to actually play and manage audio. The rendering interface provided in AudioDrv (used by Psychonauts) is stock to ISACT and handles creating the OpenAL context, managing flags and parameters passed to OAL and EAX and so on.[^2] - isactwin.dll: This is ISACT itself. This DLL contains the core of the ISACT engine itself and provides only a function to retrieve the ISACTInterface. You pass in an `ISACTRenderInterface` and it returns an `ISACTInterface` that uses that render interface and is then what you're primarily going to be using for everything else. Loading banks and so on all happen in here. This is ISACT's heart, so to speak.[^1]Now, these libraries are nowhere to be found in the Linux version. Why? Because their code is embedded inside the game itself now. Yeah, there's a bit of a confusing story going on here.
Over in this post, I partially reconstructed the source tree of Psychonauts' Linux port by making use of leftover asserts which included the path for the file the assert was contained in. This actually includes a couple of files from ISACT! In particular, we can see some ISACT code located in CommonLibs/NonDF/AudioDrv and CommonLibs/NonDF/ISACTEng. AudioDrv is self-explanatory while ISACTEng is likely all the core code for the ISACT engine and has pieces shared between AudioDrv and isactwin.
Now, I had some back and forth with @bekoha about this after making that post and we came to something of a realisation. I will note that I'm still trying to confirm if this is the case so this is still for all intents and purposes speculation but the evidence is quite strong. See how the ISACT stuff is in a folder called NonDF? Okay, so NonDF is used for third-party libraries that Double Fine didn't make, right? Well... not exactly?
Isn't it strange that these are in the NonDF folder, but DirectX isn't? In fact the only other thing in there that we can actually see (granted, there might be other files in there which don't have asserts in them or aren't compiled into the game) is MojoShader, Icculus' own library for handling D3D shaders across various APIs and platforms.
And why embed ISACT directly into the game on Linux instead of keeping them as libraries when porting them?
ISACT was seemingly free for use in PC games and in this case Double Fine seems to have leveraged it because it had the ability to quickly convert from Xbox XACT projects to ISACT. In the ISACT SDK we have, all we have are libraries and header files. It provides no source access to the internals and according to this it seems like it might've actually been necessary to contact creative and jump through special hoops to gain access to that stuff.
So, we don't know if Double Fine would have even had this code. It seems far more likely that they had exactly what we had - headers, lib files, ISACT Production Studio and some documentation. Icculus, meanwhile? I would not be shocked to learn that he has the real deal at hand and was willing to simply merge it directly into Psychonauts' codebase for the purpose of porting the game.
tl;dr2, ISACT is merged directly into Psychonauts' Linux binary and that means it too is built in debug and it too has all the debug information. This does complicate things because it means that both AudioDrv and isactwin are somewhat tangled together but it shouldn't be too difficult to sort them out as work progresses.
Note: The collapsible above contains a long, speculative ramble which I was quite confident in but it turns out was completely off the mark. I've decided to keep it here, hidden, for posterity but please keep in mind that I have confirmed it to be incorrect.
ISACT SDK
The original ISACT SDK naturally proves to be a useful resource, too. It ships with two headers - isacteng.h and isactri.h. They roughly correspond to isactwin and AudioDrv respectively. They're heavily stripped down for public eyes and seem to essentially be compilations of various bits and pieces of the ISACT codebase providing the minimum needed to interact with AudioDrv and isactwin. Still, these are incredibly useful because they fill in some of the gaps that we don't have from the debug info, mainly struct and other type information.
openal-soft
OpenAL Soft is an open-source implementation of the OpenAL spec and OpenISACT is specifically made with it in mind. Though, OpenAL Soft does work as a drop-in replacement for the proprietary OpenAL DLL and so the reverse will hopefully be true when it comes to OpenISACT - it should work with the proprietary OpenAL DLL as well as OpenAL Soft. This matters pretty much solely for reimplementing AudioDrv.
I'm not super familiar with OpenAL and so I've had to dig around OAL Soft a lot to get a grasp on things. Looking up what functions and such do inside it also helps me work backwards to infer type information inside ISACT's code. Since I'm working with compiled code, I'll usually see magic numbers passed to functions and need to quickly reference the OpenAL headers in order to check the correct original enum value for them.
Official OpenAL Documentation
Of course the official OpenAL docs are still useful for many of the same reasons - I'm not too familiar with OpenAL and the official docs tend to prove more helpful when looking up functions than searching in the openal-soft headers.
...Doom 3...?
Okay, this is cheating and I'm very much just including this here partially as a joke but I actually did use the Doom 3 source code today while working on this project so I'm counting it. It's not entirely out of left field - Doom 3 uses OpenAL and has some OpenAL headers inside it which served to solve a mystery I was dealing with. Even OpenAL Soft mentions the Doom 3 code being used as reference in some files, specifically ones related to what I was trying to figure out, so that's cool. I'll explain this properly in a future post.
Process
My working process for this is fairly straightforward. I run through the examples shown in the ISACT documentation and implement each function shown in them. I'll run down that function's entire call tree and implement as much as I can. I'll implement what I need of relevant types and classes along the way and if I happen to pass over anything that'd be simple/easy to do and save me some future work, I'll do that too. If there's any pieces of code I can't or don't want to work on at that time I'll label it clearly with a //TODO so it's easy to find later.
I've also taken a page of out of Icculus' own book and I've been using his STUBBED macro as seen in this presentation. Mainly because it's actually incredibly helpful but partially because I thought it'd be funny.
Progress
As it stands OpenISACT is in its infancy. Obviously. I only started it like a couple days ago. The project is an uncertain mess because I've yet to figure out how to organise and sort everything, the CMakeLists is a pile of duct tape because I don't know CMake, and I have no idea if it'll currently build on any configurations besides mine or what the process will be for that. I, at least, can successfully build it for both Linux and for Windows through MinGW.
I will say at least that AudioDrv is successfully at the point of doing some internal initialisation and acquires an OpenAL context and such, so that's good.
-
This stuff is still really hard and often tedious work. All the extra info goes A LONG WAY in just untangling what each part of the code is for in a general sense but we still don't get structure information and we're still dealing with some butchering of the code by the compiler here and there, so especially for more complex functions and particularly for figuring out the complete intricacies of any given class, there's still much to sift through. The debug information just removes a large majority of the pain and lets you focus on the more "fun" stuff.
-
If I can get confirmation and details on all this, I will likely turn this ramble into a seperate post and replace it here with a link to the post in question.