Beta is on cohost dot org

All I know is that the last few times I’ve tried to write a windows iso to a flash drive from anything but a working windows install it’s ended in pain and then I remember this guide which is a nightmare https://alexlubbock.com/bootable-windows-usb-on-mac

As I understand it, a CDROM can have exactly one ISO data track, it has to be the first track on the disc, and an .iso is supposed to be a dump of just that track.

If only! You can have arbitrary numbers of ISO data tracks, and they can go anywhere on the disc. "Enhanced CD" discs usually place the data track after all the audio, and as you might imagine if you try to mount it as just a plain ISO, Windows gets very confused about the LBAs not starting at 150.

Speaking of starting at 150, that's because the ISO format (and almost every CD disc image format) completely skips over the leadin that contains the table of contents. The leadin is sectors 0 - 149, hence ISOs and every other disc image starting at 150 and not 0. It's great! Love it here.

(That's also why you need a table of contents like a standalone text cuesheet, or DiscJuggler's single-file .cdi that embeds the TOC with the disc data, etc.)

and now to look at how Linux images do persistent file systems while only writing a read-only one:
incoherent screaming, gnashing of teeth, hellfire
ah, well wasn't that informative.

This has addressed so many tiny mysteries from my days of disc burning and USB stick creation from, like, 1998-2005. So many things that were “it might work or it might not” or vague explanations of “to make a boot CD you need a floppy disk image” or whatever.

Thanks!

Tangentially related knowledge from hackintoshing: Modern installers for macOS come in "Offline" (has everything) and "Online" (you gotta have a connection to download the install files to the machine) varieties.

The Offline variety come with the predictable "ESP that kicks you into an installer on an HFS+ partition" format, but for the Online installer, you can just make the disk one huge fat32 partition with an EFI folder and a com.apple.recovery.boot folder.

This just ends up mounting a DMG file, from my understanding, so not that different from the Ubuntu installer in the end. But it feels fascinating that you can run a macOS installer from just a normal fat32 disk.

Rufus works for Windows because it's dumber than Balena: it copies the contents of the image to the disk. Like, the files. It literally does nothing more than make a FAT32 partition, then copy the contents of the Windows ISO to it, file by file. That's all.

This used to work with NTFS on BIOS, too. MBR partitions could contain a bootloader, and if they are marked as bootable, BIOS would boot it. Crucially, NTFS places there a short program that finds a Windows bootloader on that partition and loads it. CD images contained that bootloader, so if you create an NTFS partition on a flash drive, mark it bootable, and copy onto it the contents of an ISO, it'll boot!

This used to work with NTFS partitions created on Linux too, until fairly recently people decided distributing a binary of Microsoft's bootloader someone found somewhere as a GPL code is probably haram.

You can / could (?) still create a FAT32 Windows boot USB using the media creation tool - it will super-compress the WIM into an ESD (encrypted?) image. That seems to work well enough, and I think they also allow split images too.

It went the ESD route when you'd tell it to make a combo x86/x64 image. If you use RUFUS' NTFS driver, naturally it breaks secure boot. I was pretty chuffed at finding a way around that.

I can't say that I have a ton of direct experience in this subject but i believe that easy2boot has a lot of documentation that may help in making sense of all of this (this sort of distinction with iso files and booting is a key part of why easy2boot has options for renaming iso images)

so where does udf slot in for all this?

thanks very much for this very thorough write-up! it's a topic we'd been meaning to dig deeper into, too, because of all the hidden complexity.

as regards the presumed need to load a CD-ROM driver, remember that motherboard firmware used to be smaller, ROM chips used to be more expensive, and firmware vendors likely weren't in a hurry to take on the additional cost of writing drivers for CD drives. the strategy of loading the driver off the disc puts the cost on... well, whoever writes the burning tool.

you also mentioned that Linux-y stuff doesn't always load properly on Windows despite it being nominally a standard filesystem. there are several extensions to the filesystem itself, which offer various kinds of metadata; it's likely that one of them is in use and messing Windows up.

even spicier is what linux image builders had to do (still have to do?) to get things to boot on the mac implementation of EFI while also still functioning properly on UEFI. i think there's a tool called "bless" involved somehow

Possibly interesting tidbit: Gamecube games use a proprietary, non-ISO9660 filesystem, but the parts of it that the IPL cares about all happen to fit within the reserved area at the beginning of the ISO9660 spec, so it's possible to make an El-Torito-compatible Gamecube apploader to boot GC-Linux (or indeed, any other GC homebrew that benefits from having a filesystem)

this reminds me a lot of the eldritch madness matthew garrett inflicted on himself back in the day trying to make ubuntu CDs more reliable on Macs. sample post of the era: https://mjg59.dreamwidth.org/11235.html

So basically every Linux bootable images is the equivalent of those polyglot source files that are valid C, Bash, and PHP. Incredible.

I thought UDF was jiggered towards some awkward "sorta arbitrarily rewritable like a floppy, but not backwards-compatible and requires a very slow pre-format" CD-RW packet-mode. I tried it out once, it didn't take long to figure the whole thing wasn't worth the bother.

My theory is Linux images are still ISO9660 because the Linux ecosystem was heavily bring-your-own-media since at least the turn of the century. ISO images are simple and almost universally supported by burning software so distro devs supply them. A toolchain develops around authoring them for things live environments. DVDs and then USB come around and the toolchain evolves for end users to "burn" USB drives, but the limitations of ISO9660 images never become enough of a roadblock that that link in the toolchain gets swapped out. It's kinda like how rather than fixing Makefiles, they went out and invented Autotools and CMake.

Whereas windows didn't really officially dip its toes into BYOM for a lot later. DVD burners were already a thing you could expect the sorts of people who were downloading images off of MSDN to have access to. And multi-gigabyte thumbdrives were getting affordable too. And they don't have the momentum of an existing ISO ecosystem to deal with so hey, no reason not to use something more robust.