Coder, pun perpetrator
Grumpiness elemental
Hyperbole abuser
Tools programmer
Writer-wannabe
Did translations once upon a time
I contain multitudes
(TurfsterNTE off Twitter)
Trans rights
Black lives matter
Be excellent to each other
I probably have some part of this wrong but I can find almost no complete info about it online; I'm sure someone will have corrections, I will add them if you leave a comment.
I've been wondering for years: how is it possible that an ISO can be written to a USB drive and booted on a PC, just like that? First three guesses don't count, mine were all wrong.
All I know is that the last few times I’ve tried to write a windows iso to a flash drive from anything but a working windows install it’s ended in pain and then I remember this guide which is a nightmare https://alexlubbock.com/bootable-windows-usb-on-mac
oh interesting, i must have been using an outdated windows 10 installer. I'll have to revise that.
As I understand it, a CDROM can have exactly one ISO data track, it has to be the first track on the disc, and an .iso is supposed to be a dump of just that track.
If only! You can have arbitrary numbers of ISO data tracks, and they can go anywhere on the disc. "Enhanced CD" discs usually place the data track after all the audio, and as you might imagine if you try to mount it as just a plain ISO, Windows gets very confused about the LBAs not starting at 150.
Speaking of starting at 150, that's because the ISO format (and almost every CD disc image format) completely skips over the leadin that contains the table of contents. The leadin is sectors 0 - 149, hence ISOs and every other disc image starting at 150 and not 0. It's great! Love it here.
(That's also why you need a table of contents like a standalone text cuesheet, or DiscJuggler's single-file .cdi that embeds the TOC with the disc data, etc.)
oh weird! I could have sworn I'd read that it was limited to one. bad! can .ISO handle multiple data tracks?
Nope, an ISO represents a single data track. It can't represent more than one track.
Way back in the day when people used to split up and compress game rips to share on dialup, I remember PC Engine CD games that had a random set of .iso and .mp3s, because there were several games on the system that had more than one data track (and the data track is always track 2, not 1). CD-ROM kind of just lets you do whatever you like
my assumption is that there are essentially zero commercial windows tools that can understand a disc like this, they all process only the first data track and ignore any others unless you get special tooling
PowerISO detects PC/classic Mac hybrid discs at least, I haven’t seen much more than that. It also only shows the PC half IIRC.
Recently I was trying to boot Red Hat 7.3 on an HP i2000 (Itanium Merced) and ended up with this invocation to make a bootable CD out of the boot image which I believe was originally intended for an LS120 disk:
xorriso -as mkisofs -iso-level 3 -r -V "RHBOOT" -J -joliet-long -append_partition 2 0xef boot.img -partition_cyl_align all -o RHBOOT.iso . (very similar to https://wiki.debian.org/RepackBootableISO#arm64_release_9.4.0)
Why do I think it’s for an LS120? The size, and some later docs: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/installation_guide/ch-ia64-intro
Fun little bit of history!
and now to look at how Linux images do persistent file systems while only writing a read-only one:
incoherent screaming, gnashing of teeth, hellfire
ah, well wasn't that informative.
I looked a bit into that, it seems like they use a filesystem overlay with COW. it seems sensible enough in concept but the mount list sure looks fun and reasonable
Yeah the gnashing of teeth mostly comes from how (AFAIK) there isn't a unified liveusb-with-persistent storage layout beyond the bounds of any particular distro. some did FAT32 shenanigans, some made an ext4 partition after the iso ends on boot, some relied on USB flashers to create a partition and put it into the grub config.
This has addressed so many tiny mysteries from my days of disc burning and USB stick creation from, like, 1998-2005. So many things that were “it might work or it might not” or vague explanations of “to make a boot CD you need a floppy disk image” or whatever.
Thanks!
Tangentially related knowledge from hackintoshing: Modern installers for macOS come in "Offline" (has everything) and "Online" (you gotta have a connection to download the install files to the machine) varieties.
The Offline variety come with the predictable "ESP that kicks you into an installer on an HFS+ partition" format, but for the Online installer, you can just make the disk one huge fat32 partition with an EFI folder and a com.apple.recovery.boot folder.
This just ends up mounting a DMG file, from my understanding, so not that different from the Ubuntu installer in the end. But it feels fascinating that you can run a macOS installer from just a normal fat32 disk.
Rufus works for Windows because it's dumber than Balena: it copies the contents of the image to the disk. Like, the files. It literally does nothing more than make a FAT32 partition, then copy the contents of the Windows ISO to it, file by file. That's all.
This used to work with NTFS on BIOS, too. MBR partitions could contain a bootloader, and if they are marked as bootable, BIOS would boot it. Crucially, NTFS places there a short program that finds a Windows bootloader on that partition and loads it. CD images contained that bootloader, so if you create an NTFS partition on a flash drive, mark it bootable, and copy onto it the contents of an ISO, it'll boot!
This used to work with NTFS partitions created on Linux too, until fairly recently people decided distributing a binary of Microsoft's bootloader someone found somewhere as a GPL code is probably haram.
I believe for this reason one of the manual methods for making Windows ISOs into USB remains "Just copy the files on there, then run a command that copies over a bootsector"
You can / could (?) still create a FAT32 Windows boot USB using the media creation tool - it will super-compress the WIM into an ESD (encrypted?) image. That seems to work well enough, and I think they also allow split images too.
It went the ESD route when you'd tell it to make a combo x86/x64 image. If you use RUFUS' NTFS driver, naturally it breaks secure boot. I was pretty chuffed at finding a way around that.
I can't say that I have a ton of direct experience in this subject but i believe that easy2boot has a lot of documentation that may help in making sense of all of this (this sort of distinction with iso files and booting is a key part of why easy2boot has options for renaming iso images)
so where does udf slot in for all this?
actually i thought this entire longpost was going to be about how iso is just udf now and iso9660 proper is ancient history
I'm not sure that actually happened, in short. The Ubuntu ISO for instance shows up in Linux mount as iso9660; if it was udf it would say udfs afaik. I had been under an impression that DVDs required udf but this does not appear to be true. I haven't looked into it in depth but I suspect the features udf grants are just not that important on read only media compared to the increased complexity and decreased compatibility, but that's a pure guess.
Oh and the other part of the answer is that, as far as I know, UDF doesn't affect it at all because the disc header structure is identical for backwards compatibility purposes. Apparently you can make a dual FS disc that reads as both ISO and UDF, so the boot sector has to be there still and that means you can still do this trick with it.
thanks very much for this very thorough write-up! it's a topic we'd been meaning to dig deeper into, too, because of all the hidden complexity.
as regards the presumed need to load a CD-ROM driver, remember that motherboard firmware used to be smaller, ROM chips used to be more expensive, and firmware vendors likely weren't in a hurry to take on the additional cost of writing drivers for CD drives. the strategy of loading the driver off the disc puts the cost on... well, whoever writes the burning tool.
you also mentioned that Linux-y stuff doesn't always load properly on Windows despite it being nominally a standard filesystem. there are several extensions to the filesystem itself, which offer various kinds of metadata; it's likely that one of them is in use and messing Windows up.
even spicier is what linux image builders had to do (still have to do?) to get things to boot on the mac implementation of EFI while also still functioning properly on UEFI. i think there's a tool called "bless" involved somehow
Possibly interesting tidbit: Gamecube games use a proprietary, non-ISO9660 filesystem, but the parts of it that the IPL cares about all happen to fit within the reserved area at the beginning of the ISO9660 spec, so it's possible to make an El-Torito-compatible Gamecube apploader to boot GC-Linux (or indeed, any other GC homebrew that benefits from having a filesystem)
this reminds me a lot of the eldritch madness matthew garrett inflicted on himself back in the day trying to make ubuntu CDs more reliable on Macs. sample post of the era: https://mjg59.dreamwidth.org/11235.html
https://mjg59.dreamwidth.org/11285.html is a nice overview of the fully put-together madness at play here, including pieces of a partition map being executed as code
