outrider

Super Mario Bros. Movie (1993) apologist

@outrider

rock sleep repeat

on the internet, nobody knows you're a dog

warning: will incessantly waffle at you about myst, kreator, and an array of fish bothering games

log in
outrider
@outrider
NireBryce
@NireBryce
love
@love

last week I realized that due to the requirements of this prototype's battle system having an AI system that looks ahead several turns, I now have a way of simulating game state without actually waiting on user inputs or animations or any other external state

in other words

for the first time in my entire game development career I can USE A TESTING FRAMEWORK!!! I can just drop in a little set of instructions of what should happen in a battle and I can press a button telling me if it behaves correctly or not! WHAT AMAZING TECHNOLOGY TO FINALLY BE ABLE TO TAKE ADVANTAGE OF. ONCE I GOT SOME BASIC TESTS INTEGRATED THEY IMMEDIATELY REVEALED A SUBTLE BUG IN THE SIMULATOR. THIS IS SO COOL I'VE NEVER BEEN SO EXCITED ABOUT A SINGLE BIT OF TECH IN MY LIFE AS THIS.

button that tells you if your code works or not. wow. wow. wow

outrider
@outrider

ever since leaving corpogames I've been wanting to do a talk on the differences between game and application dev

because this kind of stuff is so endemic in games

like even version control, because on the one hand many tools just aren't designed to work with things like that (eg Unity used to generate a big pile of files that hold eg workspace state and so change constantly) and on the other hand you don't really get some of the big advantages of version control systems when you're working with lots of binary files in highly proprietary formats

You must log in to comment.

in reply to @outrider's post:

love
@love

I would definitely LOVE to hear if you've had any experience with automated testing in AAA—it's not my space at all so maybe I've just missed it, but I've basically never heard anyone in games ever talk about having used it at all? And I'm sure there must be other design concerns related to making stuff work with tests other than just the basics of architecture that aren't immediately obvious.

login to reply
outrider
@outrider

I've only been at a single games studio and it was a weird middle ground - huge revenue and sales numbers but not really any mainstream big-name games, and they didn't even use a standard bug tracking tool until about 2010

I did my best to implement some automated testing tools inside the game's own scripting language for running in test builds of the game, but there wasn't anything like unit tests or CI builds or anything. Again it's difficult to do that kind of thing just because the tools usually simply weren't designed with that kind of thing given any thought at all

and like in most cases people will be using a pre-built game engine with its own scripting language that you can't really selectively compile or run only parts of, and obviously the big name engines don't have much capacity for things like tests either

hell I've recently tried to set up an automated, headless build for a Unity thing because I've been doing Linux distributions for free game stuff I like (because I bought a Steam Deck) and it's absolutely incredible how much Unity don't want you to do that kind of thing

if you want some thoughts or bounce off some ideas on how to get your tests running automatically whenever you make a commit to version control hit me up though, I've been doing that shiz for a living for the past few years and I just honestly love making people's lives a bit easier like that

login to reply
outrider
@outrider

it's just wild how these things have been common practice in application software for so long and meanwhile when we make games we're basically just banging sticks and rocks together

this is kind of what I'm thinking about wrt doing a talk, right - games are just all smoke and mirrors at the best of times so there's just incredible amounts of "fuck it, that'll do" in them and that extends from how they work even into how they're made

I like making software engineers with decades of experience in application dev cringe in horror when I tell them about how I know a games company of several hundred employees which had an entire person (and only one, so NO HOLIDAY FOR YOU EVER) doing a more-than-fulltime job manually merging a dozen or two developers' code changes and cherry picking individual changes line by line, from a code base of hundreds of thousands of lines across thousands of files, to go into new testing builds

a job that you may recognise is basically fully automated and integrated into everyday workflows by setting up an off the shelf version control system and using branches in it

login to reply
love
@love

Thankfully I don't particularly have any issues with source control, and also I'm in msbuild land rather than Unity, so having tests run automatically is actually pretty easy! Appreciate the offer, though.

I do think a big part of the problem is that "fuck it, that'll do" is the most important skill a game developer needs to actually ship—in other contexts, worrying about many kinds of technical debt in the same way a software engineer would is kinda foolish if a project only lives for five years, for instance. And I've seen people go from professional software engineering to game development struggle to get on the level of quick and dirty that games demand for fast iteration, or overengineer systems for types of safety that are necessary for APIs that other people have to work with, but deeply unnecessary for a videogame that acts as a black box.

...but you can definitely take that too far, and there are some things that doesn't apply to. Broad workflows are worth actually getting right! The calculus over "the amount of engineering overhead this would cost will never be offset by the amount of time you save later" can be easy to get wrong, especially if, as you say, the actual costs are simply offloaded onto another person. I think this is relevant for testing too, if you're at a studio with continual QA testing, that can... kind of more-or-less approximate the benefit of automated tests. Kinda. Albeit at a high cost. In my case, though, it's much more obvious because that burden is all on me either way!

login to reply
outrider
@outrider

and obviously corpogames has a very long history of just hiring a pile of people on temp contracts to have more bodies to throw into the QA dungeons in particular

anyway yeah having worked in both it's kind of fun to think about all the myriad ways in which games and application software dev are trying to achieve a similar goal ("ship a thing that works and is reasonably complete") but the priorities and methods are wildly different because of how they function

like automated testing is also difficult in games because you can unit test your stuff to hell and back but games specifically are made of so many discrete components that constantly interact, and often several of them at once or through multiple layers of systems, that it's just absurdly difficult to automate beyond a fairly basic level

by comparison even very complex application software setups are small and predictable and deterministic and that precision and exact reproducibility is much more important because who cares if the physics behave slightly wonkier at low framerates but you absolutely don't ever want that kind of thing in, say, airplane flight computers

peripherally relatedly, I saw a thing a while back where someone set up a github workflow to automatically test their Doom source port by running a few recorded demos and verifying that various statistics remain the same (damage taken/dealt, enemies killed, etc etc) which seems like a fun approach here but of course that again only tests the interactions of systems recorded in the demo

login to reply