Here's a fun quiz: take a look at these four game screenshots. What aspect ratios are they at?

Ready? ...sorry, this was a trick question. Actually, the answer is they're all the same aspect ratio. Each of these games is 4:3, and they definitely don't look like this on a TV. But how can these all be 4:3? How did it work to show them that way?

Why I'm writing this post

I'm not the kind of prescriptive person who says games have to be played the way they were made, but I do think it helps to know what games were designed for so you can make informed decisions. People who have played retro games on oldschool TVs have memories of seeing what older games looked like, but I think a lot of people who are newer to retro games haven't necessarily seen them in their original environment and don't always realize if they're seeing a slightly distorted image or not.

(By the way, this post isn't a precise technical guide. I'll be glossing over or simplifying some concepts to keep it understandable. I'll also note that I'm mainly talking about console games and things meant to be seen on a TV: computer monitors are very different. If you're interested in the considerations of computer monitors, dosbox-staging published a great blog post about the resolutions used by systems like IBM PCs and PC-98.)

What do these games actually look like?

Before we get further, how about some pretty pictures? Here's how the other three games look like when they're shown in 4:3.

Retro console games were made to be shown on a TV

Sounds obvious right? Yes, of course retro console games were made for TVs. But think about it: a TV is a squarish 4:3 rectangle. Most older games and consoles were designed to be played on a normal TV, and were intended to (mostly) fill the screen.

But like you can see at the top of the post, when we look at digital screenshots, what you get isn't always a 4:3 frame. The Sonic 2 screenshot is the only one that's actually 4:3—the other screenshots are too wide, too tall, or too thin. So how could those display properly?

Pixels are square... right?

Pixels aren't square! Not all the time, anyway. To understand how that works, there's a simple answer: older TVs are analogue, not digital. An analogue signal isn't defined in terms of pixels; there's a vaguer sense of "resolution". There's a rough maximum amount of detail you can cram into a signal, and that depends on what signal you're using to connect a console to a TV, but there isn't a true measurement of a TV's resolution in terms of so many pixels by so many pixels.

You could say that a TV doesn't have a "real", "native" resolution, but that's not quite true. A TV has a fixed vertical resolution, while its horizontal resolution¹ is variable.

Analogue TV is measured in terms of lines: on NTSC TVs (used in the Americas and parts of Asia), a signal is about 480² visible lines filling the screen, while on PAL TVs (used in Europe, Oceania, and parts of South America and Asia), a signal is about 576 lines. Your horizontal resolution, on the other hand, is what's variable: you can ask the TV to render as much or as little detail as you want within a line, so long as the signal can fit that much detail.

And that's how consoles handle this. They're always showing around 480 lines of signal, but they're generating a signal with varying amounts of detail within a line. If we think of it in terms of pixels, like the console generates internally (or like we see in an emulator), not all pixels are made equal: some are different sizes. Only Sonic, in our examples above, uses the square pixels we normally think of. Chrono Trigger is using thicker, shorter pixels that are spread out a bit horizontally; Gamera 2000 is using pixels exactly twice as long as they are tall; and Martial Masters is using very narrow pixels since it's packed so much detail into each line.

An aside: TV calibration is a mess

So while TVs are 4:3 and the image is meant to fill the screen, that's not... quite... how it works? Analogue TVs are analogue equipment, and you have to calibrate them for the image to actually be centred properly. Pretty much all CRTs came from the factory with the image maybe a bit off-centre, too big, with bits of the image on every side cut off. You could fix that yourself, but most people... didn't? So while most games knew they were designed for a 4:3 TV, they also knew their image would be mostly 4:3 instead of exactly 4:3... and that some of their image would be cut off.

Which sucks! But sometimes you could take advantage of that. Older consoles often had dead zones on the side of the screen; if you knew that most players wouldn't be able to see content at the edges, why not just not render it to save some processing room. A lot of earlier systems rendered a "224p" image, where they skipped a few lines on the top and bottom of the screen so they didn't have to render anything there. NES games also often rendered blank space on the sides of the screen so they could cut down on how much they needed to render, trusting that most players would never actually be able to see it.

This is a total tangent, but it also made things hard on developers too. If you knew that not everyone would be able to see the edges of the screens, you had to avoid putting important things there. Many devs didn't care, but many did, and so you had games putting things like health bars a little inward on the screen to keep them away from the edges. This is especially noticeable in home ports of arcade games: arcade games are designed for calibrated monitors that actually show most of the images, so the health bars sometimes have to get moved in the home ports. Take a look at these two Street Fighter Alpha 3 shots, for example: the PlayStation version squished the health and super bars inward a little to keep them away from the furthest edges of the screen. (First screenshot arcade, second screenshot PS1.)

1. In the analogue media era, people would estimate horizontal resolution in terms of lines, as in about how many distinct lines you can make out. It's not quite pixels, but it's a close approximation in the analogue realm.

2. What about 240p? you might ask. 240p is actually still a 480-line signal, just with a clever hack. Normally, a 480-line signal is divided into two alternating sets of 240 lines, which are drawn one after another. A 240p signal just keeps drawing to the same set of lines over and over, ignoring the other half of the TV. That's where the black scanline effect comes from: half of the lines on a 240p signal are blank.

But what about emulators?

What about emulators!

It used to be that the majority of emulators didn't correct for this at all: they showed whatever the game's internal resolution was, even if that's too thin or too tall. This has, IMO, slightly warped how people think NES and SNES games look like: most peoples' mental image of them is now a bit thinner than they were actually meant to look.

These days most emulators at least offer an option for aspect ratio correction, to stretch the game to show it at the size it would have been on an original TV. Some even use some pretty clever rescaling techniques to do it without making them look too blocky. Not every emulator has it on by default, however, so it's worth checking your options if you care about it.

Should you care about it? Well extremely centrist voice that's a personal decision. Personally, I think most games look better at their original aspect ratio instead of too thin or too wide, but maybe you feel different! But if you give it a try both ways, you can decide what you think looks better—on a game by game or system by system basis, if you want to.

Where Misty rambles slightly about game mods

The main place it bothers me really is when game mods for retro games don't take the aspect ratio into account. Awhile back, I saw a fan translation of a game announce how, aside from translating the game, they'd also improved it by increasing the resolution. After all, they couldn't imagine why the devs had left it running at 320x240 when it could have been 352x240. The problem, of course, being that on a TV those are the same size: instead of increasing the resolution, they've made all the sprites in the game thinner so they can cram a little extra view space to the left and right of the screen. Maybe that's just what they like, but I like to think they just aren't familiar with how it looks on real hardware and don't realize there's actually a side effect to making those changes.

In conclusion

Video games! Play them, or don't. The choice is yours.

Addendum: what about games meant to be shown on CRTs but where the console didn't support the intended resolution?

Why would that even happen? Even though you could display pretty much any horizontal resolution on a CRT, not every system supported every option. The video chip and video encoder in use controls what resolutions a system can output. Early consoles usually had a fixed aspect ratio, or maybe a couple of options; you weren't seeing every possible resolution from any given console. While later systems would often support higher horizontal resolutions, they might not always be able to do the lower resolution an older system could. If you were porting from one system to another with a mismatch in supported resolutions, what would you do?

I've got a few examples here from Saturn games. (This isn't by any means exclusively a Saturn problem, I just happen to be familiar with the library.) The Saturn's horizontal resolutions in low-res mode are 320 or 352, which was a problem for any game intended for a lower res like the 256 horizontal res common on 16-bit systems.

The most obvious approach, of course, is just to letterbox it. That's the default setting for the Saturn port of Sexy Parodius. The arcade game was designed for a 288x224 arcade board, so it just adds some black bars on the left and right sides. You get graphics displayed at the wrong aspect ratio - like in some emulators - but at least the picture's still there.

Sexy Parodius also offers another option though: extend the playfield left and right. It's still the wrong aspect ratio, but it fills the screen and lets you see little more of what's coming to you from the right side of the stage. (AFAICT it keeps the active screen size the same so it doesn't mess up the gameplay, it just lets you see more.)

What if the game screen is wider than your intended aspect ratio? Slightly more obscure, but it happens too. For example, the Capcom CPS2 games run at 384x224; on the Saturn, they crop it down slightly to 352x224 and reposition the life bars to accommodate for it. It's ever so slightly the wrong aspect ratio, but it works fairly well.

(Arcade)

(Saturn)

And, finally, there's the chaos approach. Castlevania: Symphony of the Night runs at 256x224 - I assume because of all the sprites they're recycling from Rondo of Blood, which also ran at that res. Did the Saturn version letterbox it? Give you a little more space? No - they stretched it. And this is before sprite filtering, so they couldn't make it look nice or anything; it's a jaggy nearest neighbour scale. Take a look at that staircase! Oof.

Compare to the PlayStation version, where the staircase clearly looks how it's meant to look: