Beta is on cohost dot org

<< Chapter 1 < Chapter 4

I picked QMK because there was already code for the Ergodox Infinity. There were also already code for the class of LED Drivers I’d chosen for my RGB backlighting. Surely it would just be a matter of combining those two things together!

Programming

There are many things that make Arduino popular and easy to use. But the most important thing is the ecosystem: You write their language (mostly C) in their editor, and you can easily program their board over USB. No extra tooling required, and the community has libraries to do almost anything you could want.

The part that makes that USB magic work is called the Bootloader, and just about every board has one. Under certain conditions (usually immediately after a reset), the board will enter “programming mode.” It has enough code to boot up the USB driver and listen for a certain set of commands from the computer. When it receives a binary, it overwrites a region of the chip’s built-in flash memory with the provided machine code. Importantly, you can’t command a bootloader to overwrite itself, thus preventing people from turning the board into an ineffective paperweight.

But the bootloader is just code, there's nothing magic about the chip. How do you get the code on there the first time? You use a programmer.

The original Ergodox Infinity has contacts on it for a Needle Adapter. These make sense for a “mass” production run. But the cables are very expensive for a one-off like mine, so I replaced it with a standard 2x5 SWD header. A ribbon cable connects to the end of the programmer.

In addition to flashing a binary, you can use a programmer as a debugger, stepping through line by line using GDB.

This will be important later.

QMK

QMK is a very popular firmware for mechanical keyboards. There are a lot of keyboards in the repository, including some very small-run boards. The online configurator simplifies customising the functionality of your keyboard, the bread-and-butter of mechanical keyboards. The internal API supports a large number of features including RGB matrices and LCD displays, and before I built my keyboard I made sure everything was in the supported list. I expected to simply modify the existing Ergodox Infinity firmware files for my own keyboard, and initially that’s what I did. But as last entry’s cliffhanger mentioned: It didn’t work. Time to debug.

The Internals

Firmware like QMK doesn’t have an “operating system” in the way you might normally think. There’s no Linux kernel churning away, usually no kernel mode/user mode difference at all. But there are some very operating system-like components that it does have.

One is the Hardware Abstraction Layer, or HAL. This is a lot of what modern operating systems do: your computer doesn’t care if your keyboard is USB, Bluetooth, or PS2 once you’ve got it connected. The parts that care about that connectivity are insulated from the parts that care what key you’re pressing. Likewise, you don’t want to have to rewrite your display control code from device to device just because they picked different pins on the microcontroller. The HAL provides a way to write a driver just once and use it accordingly.

We also need a real-time operating system, or RTOS. A normal OS like Linux generally just tries to make sure every task (browser, shell, backup daemon, Magical Diary: Horse Hall) is run as much as it needs. But sometimes you need better guarantees than that. For a keyboard, you absolutely need to scan the key matrix every so often. Otherwise you don’t have a keyboard, you have a fun stim toy that lights up. An RTOS handles constraints like “This must run every 500 milliseconds” and “this task is always more important than others.”

QMK uses a project called ChibiOS, “a complete development environment for embedded applications.” I have learned a lot about this, because I spent a lot of time digging through QMK firmware.

Things I Tried Which Didn’t Work

Serial Debugging

There are lots of ways to “debug” an application when something is going wrong.

One of the most straightforward ways is “print” debugging, where you add a bunch of “here I am!” “this worked!” “no problems here :)” statements to your application.

Unfortunately this application doesn’t have a screen I can use, so I can’t just stare at a console the way I usually can. (Yes technically it does have a screen. But it wasn’t working yet, and anyway it could only show a few characters.

Fortunately there’s a solution to this: Serial debugging! That’s right, that protocol that hasn’t really changed since the seventies is the hero we need.

Now we just need to connect the spare serial line and...

Ah.

Oh well. There are two other serial lines, designed for connecting to the other half of the board. I don’t have one of those, I can just use the lines that talk to that. A few changes in the QMK configuration and we’re good to go.

Now usually these would be connected to your computer through a USB-to-serial adapter. I don’t have one of those, but I do have a Raspberry PI...

Unfortunately, after all of that work, the logs didn’t tell me anything particularly interesting. The instructions I thought were executing did seem to be, and when they weren’t the printf bisect search wasn’t particularly forthcoming.

Resoldering Chips

Of the two chips, the LED driver is a QFN-40. That means it’s a Quad-sided Flat package with No leads. Of particular note, it’s very difficult to tell if solder joints are actually... soldered, since they’re entirely out of view. The solder pads are clearly visible, but the actual connections to the chip are less obvious.

An improper connection could result in outright failure, but that’s the best case scenario. It could also result in weird, inconsistent behaviour or fry a chip altogether.

The easy part is removing a chip. The same hot-air reflow station that melted our solder paste the first time can melt it again (or “reflow” it). Once all the solder on the pads is liquid, the solder can be soaked up with a wick or the chip simply removed entirely. Be careful and quick though! Chips are only really designed for one high-temperature soldering session, and further sessions could damage them.

Unfortunately, while the first soldering job can make use of our fancy stainless steel solder mask, subsequent soldering can’t. The mask wants to cover the entire board, but now most of the board is covered in components and the mask can’t sit flat.

Complicated SMD components that repair shops might want to replace actually have stand-alone stencils available. But even if they did exist for little QFN-40 chips, there’s too many components nearby to use one.

o

This means, instead of nice aligned strips of solder paste, I have to kind of improvise with a combination of a toothpick, tweezers, a USB microscope, and hope. And of course, flooding pads with too much improperly placed solder paste makes bridging even more likely. As with many things embedded development, the solution is to simply not make any mistakes.

Assembling a Second Board

I originally bought enough components for at least two boards, more if they didn’t get all their connectors and accoutrements. Using what I learned from my first attempt at assembly, I performed the exact same actions and expected different results.

I did not get different results.

Test Points

I neglected to break out test points for any of the LEDs, so I manually soldered wires to a few places where LEDs would theoretically go.

My initial thought was to test these with a multimeter. Unfortunately they all use PWM, so even when “working” the voltage is going to be zero most of the time. What I really would’ve needed to check this was an oscilloscope, which I don’t have.

Step-through Debugging

“Debugging” is a general process, but there’s a specific kind of program called a debugger. Generally, it allows you to pause and step through execution by instruction, set breakpoints, and examine variables.

ARM chips have an interface called SWD, or Single Wire Debug to allow this kind of interaction. It, naturally, requires at least two wires. Those are the pins the programmer uses to flash programs, and now we can use it to plumb the depths of this microcontroller.

The debugger I was taught to use originally was `gdb`, a command-line application that’s older than I am. And in typical computer fashion, that’s actually still what I use.

A tool called OpenOCD (no relation to the disorder) uses the programmer, but of course it’s not plug-and-play. First, you have to run a command that flashes OpenOCD onto the programmer itself. That command will start up an in-between program that runs on your real computer, and exposes a telnet port that gdb can connect to.

And of course, since the NXP chip is Arm, you need a special multi-arch version of gdb.

I also eventually got this working in shudder Eclipse.

From there, I had to figure out the bootloader written for the original Ergodox Infinity. And a bunch of weird messages about code optimization.

And in the end, I didn’t really learn anything new. Well, I learned how to use a debugger on an ARM chip. But the code I thought was executing was, and I was no closer to a solution.

Disabling EzPort

The chip I was using had an “EzPort,” a sort of proprietary SWD equivalent. At one point a random forum post suggested that this mode could be inadvertently activated, which would prevent any normal functioning from occurring. There was a way to disable this in software, but of course that’s not useful if you can’t get it to boot.

Luckily, there was a pin I could pull down to disable EzPort altogether. Unfortunately I hadn’t broken it out, so I had to do a really fiddly soldering job:

This did not make a difference.

Teensy

At a certain point I stopped being able to communicate with the microcontroller on the board altogether. In an attempt to eliminate as many variables as possible, I bought a Teensy, which has the exact same chip as the board I was using.

Unfortunately I wanted to do step-through debugging, and the Teensy doesn’t break out those pins. So I did a bunch of soldering nonsense:

And I wired up the I2C, hoping to see different results. But for whatever reason this never really worked. Perhaps my soldering skills were inadequate, or I damaged a chip, or I just misunderstood what I was doing. Regardless, this was ineffective.

And of course, all of this didn't work. Next time, I'll talk about something that didn't work but I put so much work I gave it its own chapter: I²C.

<< Chapter 1 < Chapter 3

The boards came!

Now it’s time to solder.

When you think of soldering, if you think of anything at all, you probably think of what is now called “through-hole.” A component has long leads, they go through holes drilled in the board, leads get soldered then trimmed.

This can be automated on an industrial scale, but it’s expensive and time consuming. Instead, almost all modern microelectronics is done with surface-mount components. As the name suggests, these components are soldered only to the top of the board - no holes to drill, leads to trim, or voids to fill.

Introduction to Surface-mount Devices

Surface-mount soldering is usually accomplished by using Solder Paste, an incredibly expensive combination of solder and flux paste. The flux serves to hold components in place, while the solder serves to attach them when heated up. Then when heat is applied, the flux melts away and the solder attaches components firmly to the pads.

One board can have hundreds of components on even more pads, so the point-by-point application of paste is cumbersome and inefficient. Instead, most assembly houses will use a “solder mask,” literally a thin piece of metal with holes cut out where the pads are. That way, you can just squeegee on exactly as much paste as you need, and it will all go in the right spots.

The board manufacturing house I picked will sell you a solder mask to go with your board for a small extra fee. I didn’t intend to build a lot, but I thought the solder mask would make assembly for the few I did easier.

Kailh Hot-swap Sockets

You might notice there’s some pieces of tape on that solder mask. It did not come that way! But take a look at the back of the board. Those large squares are intended to hold Kailh Hot-swap Sockets:

Now, these are technically surface-mount components. But I was advised that solder paste connections may not be mechanically sturdy enough to stand up to repeated replacements of keyswitches. They would also require a lot of solder paste, and as I mentioned earlier, it’s very expensive. So I elected to tape over those holes. Once everything was assembled, I would go through and hand-solder them with the through-hole diodes I selected.

If you’re very observant, you might see the solder mask in the picture is backwards from the images of the board I posted. Is this some clever geometry play? No. After I took this picture I realised I’d taped the wrong side, and had to rip it up and start over.

Pick-and-Place

After the solder mask goes on, all the tiny little SMD components have to be physically placed on the board, a process called “population.” Professional board houses use automated tools called “pick-and-place machines” to place all the individual components on the board in the right places. These are mesmerising, extremely accurate, and cost (on the absolute lowest hobby-grade end) several thousand dollars.

So instead, I used this Pick-and-place:

It’s a time-consuming, extremely fiddly process to populate all of these boards. It is also the time when you realise that it’s all well and good to pick a 1005 diode in your extremely zoomed-in IDE, but 1.0x0.5mm is very, very small, and this will not be an easy process.

The components mostly come in paper tape, intended for the less organic pick and place machines. To make assembly easier, I placed pieces of tape in these small boxes Adafruit sells for this purpose. At one point I tried removing some of the passive components from their tape and placing them loose in the boxes. Do not do this, you will never see half of those objects again.

It took me a while to get into a rhythm. I made a spreadsheet that listed the component numbers by part type, so I could first populate “every 33Ω resistor,” then “every 5.6kΩ resistor,” etc. It is important to go carefully and in order - these components are too small to have labels. Several times I dropped a component on my assembly mat, and because I wasn’t sure which one it was I just had to discard it. Diodes are directional too - they have a tiny white spot that lines up (hopefully) with the dot on the PCB.

There’s also some time pressure here - as soon as you get the solder paste applies it starts to dessicate. I would set aside an evening to do the whole process, from applying solder paste to populating to reflow.

Reflow

“Reflow” is the process of melting the flux the solder paste is suspended in, liquifying the solder and bonding components to the PCB. There are two primary ways to do this.

The classic and commercial way is in an oven. You slowly and carefully raise the entire board to a given temperature, leave it there just long enough to melt the solder, then cool it again. There are commercial ovens that do this, but hobbyists often repurpose old toaster ovens.

I elected to go the other main route: I bought a Chinese hot air rework station off Ebay. These are usually intended for doing repairs, but for small-scale board assembly like I was doing they would (hopefully) work just fine.

Once I got everything populated and double-checked, I set up the hot air station and started from the top right, working my way down. Sometimes I got the fan a little too high and I blew some diodes straight off the board. I was constantly tweaking the temperature settings - it wasn’t clear to me how the hot air temperature corresponded with contact-soldering temperatures. Also, the project mat I was doing wasn’t as heat-resistant as I thought and I badly warped it.

But after about an hour of frustrating work, I finally had one properly assembled, beautiful-looking board.

No you didn’t.

If only it were that easy. Instead, I found things like this:

I’ve seen tons of videos of almost magical SMD assembly - a combination of surface tension and heat that seats chips with dozens of pins perfectly every time. But the instructions were very much of the “draw the rest of the owl.”

I ended up doing a bunch of trial-and-error work. I used a combination of desoldering wick, a soldering iron, flux, rubbing alcohol, and more solder paste.

In retrospect I was probably storing my solder paste improperly (I keep it in the fridge now.) But after a lot of trial and error, and melting one of the SWD connectors I got everything mostly working. I even managed to connect my Segger J-Link to the chip and program it. Excited, I flashed an image of QMK, hoping to see the RGB blinkenlights I’d spent so much time routing.

They did not blinken.

Next time on Aleph Build Log: One of Ten Thousand Things Is Wrong And I Have To Figure Out What

<< Chapter 1 < Chapter 2

Most keyboards have a case of some kind. Electronics are fragile, and the last thing you want is to short important pins or tear a tiny resistor off the bottom. Even minimal “bare circuit”-vibe keyboards tend to use at least a bottom case:

Now, this leads to a bit of a problem for me. There are plenty of existing 3D printable cases for the original Ergodox. It’s a very common form factor, and it has a commensurate amount of support.

However, because my design was based on the Ergodox Infinity, these wouldn’t work. The cables would be on the wrong side, in the wrong form factor, and there’d be no hole for the LEDs. Perhaps an existing one could be modified by someone more skilled, but not by me.

The Ergodox Infinity, too, has open-source case designs, but these aren’t 3D-printable. Instead, they’re made of laser-cut acrylic. At least at the time I was working on this I didn’t have easy access to a laser cutter. But even if I did, there was another problem: My keyboard wasn’t quite the same size.

I did my very best to keep the dimensions exactly the same as the original Infinity. Unfortunately, my addition of stabilisers and Kailh hotswap sockets meant one of the edges was just a little too small. I brought it out, probably less than a centimetre, but the damage was done. I couldn’t use an off-the-shelf case. Obviously, then, I had to design my own from scratch.

Selecting a CAD Program

Unlike EDAs, all of the most popular CAD programs (Solidworks, Fusion 360, 3DS Max) are proprietary. But there is a very popular 3D modelling program: Blender. Surely those are basically the same thing, right?

They are not! Ask me how I know. CAD is designed with a strong focus on real-world dimensions, on adjustable parameters, and a bunch of tooling and views that make it better suited. It turns out trying to do CAD in blender is a bit like trying to edit a PDF in Photoshop: technically possible, but far from ideal.

There are a couple of free-as-in-freedom CAD suites available. The first one I encountered was OpenSCAD, which I’d used to build 3d-printable keycaps.

After a bit of time messing around with it, I realised this was a very different kind of program from the Autodesks of the world. Instead of clicking, dragging, and scrolling, objects were built entirely up in a custom scripting language based on tunable parameters. This works great for things that you’ll constantly want to tweak, like keycaps. But for my design, which would only ever really need to be one size, that wasn’t particularly helpful.

Also, the learning curve seemed steep even by 3D modelling standards.

Instead, I settled on FreeCAD. This had a much more traditional user interface: you can make squares, extrude them, carve them out, the works. Armed with documentation and the knowledge that 3D modelling is notoriously complicated and difficult, I set off to build my 3d-printable custom case.

How hard could it be?

KiCad 3D Models

The best way to ensure that my case design lined up with my PCB, especially when I didn’t have said PCB in hand yet, was to work based on the same plans. Fortunately, that was doable! There’s a set of scripts called StepUp that focus on integrating KiCad designs into Freecad shapes.

To do this, though, first KiCad needs to actually know what the board will look like. It understands the geometry of the board itself, as we’ve already seen, but there’s a lot more that will need to fit into the case than just the PCB.

The basic way KiCad works is by matching up IDs with symbols on the schematic side and footprints on the layout side. The first resistor you lay down will be R1, Capacitor C1, microcontroller U1, voltage regulator Q1 (in order of most to least comprehensible) and so on.

On the schematic side, you might say you want a 33Ω resistor, which would render as a box

You also pick a footprint there. There any number of different sizes of resistor that could meet the 33Ω mark, from the classical cylinder to nearly microscopic. In this case, we’ve chosen a 1005 metric / 0402 imperial, which means 1.0mm by 0.5mm or 0.039” by 0.020”. When we sync our changes to the schematic, the program will look for R1, make sure it has the right Value (33Ω) and Footprint (1005). From there, it will render the footprint of the pads to be soldered to.

While you’re doing this, you can optionally specify a 3D model. This allows KiCad to render not just the pads, but the entire component as well.

Unfortunately, this is largely a manual process. For a common size like 1005 it’ll be easy to find a component, but something like a keyswitch or an LED might require scrounging around SnapEDA, Mouser/Digikey, and other places to find a footprint that matches. You need to be a little clever, too. Nobody will have a models for a NXP MK20DX256VLH7, but there’s plenty of LQFP64 models because many chips use the same form factor.

After a lot of painstaking work, you can render a fully populated 3D model of your board for beauty shots.

StepUp

This is, once again, not a tutorial, because I don’t actually know what I’m doing. Go find a real guide if you need help!

The basic process for StepUp is that you import a PCB, it produces a bunch of error messages, you spend a bunch of time staring at them, futzing with settings, and tearing your hair out. If by divine providence you manage to get it all imported, you’ll be left with a 3D model that badly lags your computer.

Oh at this point I realised I probably should use a beefier computer than my several-year-old ultrabook. But I pressed on.

CAD modelling is all about precision. You don’t want to just eyeball your keyboard grid, you want to measure it. So at this point I “projected” the schematic onto a plane.

Then you can convert that sketch into guidelines, and you can draw based on them.

FreeCad is a “parametric” cad program, which means you can set tunable parameters. Here’s some of mine from an earlier draft:

You build up operations like “draw lines that are always [HorizontalKeyMargin away from the outer edge of keys],” and slowly build them up, extruding surfaces and cutting holes.

I even printed out some prototypes of the key stabiliser plate:

But I still haven’t really finished. Because pretty soon, I got a fedex package from Shenzhen.

Next time on Aleph Build Log: How Hard Can Soldering Be

< Chapter 1

The first thing any good project needs is a name. I certainly couldn’t call it the Ergodox Infinity 2. For one, it was hardly an authorised sequel. I’ve spoken to the original board designer a few times on Discord, so it’s not a complete vacuum, but certainly I don’t plan on sending a pull request when I’m done.

For another, you can’t have infinity times two. That doesn’t make any sense. If you need bigger infinities, you use Aleph numbers.

(Please don’t ask me what that means. I’m many things but none of them are “math understander”).

Midway through the design process I had a brief crisis of faith - was it just ego, giving my board a new name to a mostly-rehashed product? I talked to a number of people in some Keyboard discords, and they assured me that “adding addressable RGB LEDs” is more than enough difference to justify my moniker.

The Ergodox Aleph was a go.

Circuit Boards

I’ve been curious about building my own circuit boards for a while now. When I lived in Toronto, I was a member of a hackerspace where it seemed like everyone was putting in Digikey orders, and the space was covered with purple OSHPark boards. At that point I barely knew how to solder, and was also trying to finish a degree in computer science, so I never got the chance to try. Now one had landed in my lap

In theory, you can build a circuit board mostly by hand, and that’s how it was done in days of yore. You buy a piece of fibreglass covered in copper, mask it with something like a sharpie, and etch it. “Etch” could mean physically, and some people do actually build simple PCBs with CNC routers. But usually it involves a bunch of unpleasant chemicals and an extremely well ventilated space. These produce beautiful, almost organic-looking circuit boards.

EDA Basics

Modern boards, like almost anything modern, are no longer made by hand. Instead, you design your board in a specialised piece of software called an (TODO) EDA. At my old hackerspace EAGLE (now owned by Autodesk) was the tool of choice. But just as popular is an entirely open-source tool called KiCAD.

Luckily for me, the Ergodox Infinity was built with KiCAD, so I didn’t need to “acquire” EAGLE or deal with importing. I could get straight to work.

EDAs are usually used in two steps. First, you lay out a circuit diagram symbolically in a schematic.

This was mostly done for me, but I wanted to add LEDs, so I built out a sample circuit for those two.

Then, you switch to the PCB Editor. You lay out all your components, which was again mostly already done. But I had to carefully align my LEDs such that they fell directly under the keyboard footprints.

Then, it’s time for either the best or the worst part. You’ve just arranged your components on the board, but they’re not connected to each other.

Each component has one or more “pads,” and “ surfaces that components will eventually be soldered to the board. The EDA will tell you which pads need

It’s up to you to solve this several-hundred-utilities problem. Your tools? You can draw traces, and drill holes. Traces can be on either side of the board, and holes connect one side to the other.

This can be a sort of zen experience, a puzzle game with a purpose. Broadly speaking, I try to put horizontal lines on one side and vertical on the other, which reduces conflicts. But they certainly aren’t eliminated. KiCad has a little counter in the bottom with how many unrouted lines you’ve got, and I spent two days bringing that number down to zero.

All of the components already on the board were connected already, but I ended up needing to rip most of them up.

It gets harder as you go, too. The more lines you connect, the more board space you’ve taken up, and the less space there is for new traces. High-performance circuit boards like a modern motherboard have many more than the front-and-back layers I have access to, but those are much more expensive to produce. It’s definitely half art half skill. And some of the results are extremely satisfying.

Board Clearances

After giving the board a very careful once, twice, thrice-over, I was just about ready to send it off to be manufactured. I used my friend sen’s final checklist, then I exported the files my chosen fabricator needed, hit upload, and sat back to wait.

Fortunately, I got an email back almost immediately. Unfortunately, it was not good news..

As shown below, some hole to trace clearance, slots to trace clearance is too narrow.

So some trace might be cut. And there is no space to move the trace.

Is that OK if proceed per gerber?

Or shall we cancel it for you to replace file?

You see, there’s a very important settings menu in KiCAD I’d neglected to check. Not every PCB manufacturer has the exact same manufacturing capabilities, so it’s important to check their specifications page, and set up the design constraints accordingly.

I had not done this. And because I hadn’t done this, all my traces were to close to holes, and they might get cut off. All these companies, even the very cheap ones I was using, use semi-automated, semi-manual checks to make sure a board can be manufactured. Mine might work, but it also might just be missing critical pieces.

And of course, since I’d designed my boards wrong from the start, I had unacceptably small tolerances all over the board. But that was okay, because I wanted to make another change.

LEDs

When you think of addressable RGBs, you might think of Adafruit Neopixels or any number of clones. Those strips are mostly based on a specific component called "WS2812B,” which really rolls of the tongue.

These are a bit of a clever piece of kit. LEDs themselves are not very smart, they can just turn on and off. But these have a little chip inside them, one that makes them a little smarter. In essence, you can wire up a long chain of them and they will all talk to each other. You simply wire their data pins together in series, and like a bucket brigade they pass signals down the chain.

I decided I didn’t want to do this. I was worried about power consumption - those built in chips didn’t use much power, but they did have some constant current draw, even when the LEDs themselves were off.

I also didn’t really understand how to calculate duty cycles, so I didn’t know how many LEDs would be on at any given time.

Instead, I decided to use a dedicated LED controller chip. This would allow me to use cheaper dumb LEDs, and it would give me centralised control over them. Lower power consumption overall, good control.

There’s just one problem: Because the LEDs aren’t smart, you need to wire every single one back to the controller. Three times, one for each of Red, Green, and Blue. They form a matrix, and in software you write “row 6 column 5 turn on,” and the LED at that intersection turns on.

This is an insane thing to do.

But I did it anyway.

It took me many more days to route, but I eventually got into a groove, and I routed the entire board.

Conclusion

I ended up making another couple changes. I added holes for stabilisers, added my hot-swap switches, and switched out the tiny surface diodes for big through-hole ones I hope would be easier to install.

But after a few days, there was no more procrastination. I sent my board back to the fabricators, it passed QA, and then it was time to wait.

But I didn’t just twiddle my thumbs.

Next time on the aleph build log: we make a case.