• they/them

@daboross-pottery

For future reference: https://daboross.net/. Blog & RSS feed not yet built as of 2024-09-13.


post-cohost newsletter
buttondown.com/daboross/

GFD
@GFD

given how much effort i put into optimizing my input workflows (we use keyboards so much!! it’s so much), it’s a pretty significant event now for me to change to a new software Android keyboard, so i’m gonna type a bunch of words about doing that, if you’re interested to read about it. stick around to also hear about the open‐source, on‐device voice typing app i found!

previously: i found that the “hackable” Android keyboard i’d used for years, Multiling O Keyboard, didn’t work properly on my new 64‐bit‐only phone, because its dictionary plugins were all uploaded to Google Play before Android even had 64‐bit support. this meant it lost all autocorrect abilities, rendering it very impractical to use. the app is closed‐source.

i’ve spent several months now suffering with no autocorrect while waiting for something better to come along. i chose this option over trying to figure out how to decompile and recompile ancient Android apps (no thanks), recompiling modern Android apps and hacking their layouts, or using a keyboard with autocorrect but without a customizable layout and my preferred characters. now apparently the English language plugin for Multiling O Keyboard (plus a few others) was updated 2 months ago, so it does actually work on my new phone now, but it wasn’t possible for me to have known this because i obviously can’t get update notifications for an app i don’t have installed in the first place, and the dev never directly replied to the email i’d sent them. in the meantime, i’ve begun living a new life with HeliBoard.

HeliBoard is a relative newcomer to the open‐source Android keyboard space that’s recently become adequately feature‐rich and competitively customizable. Multiling O Keyboard is still unmatched in terms of raw capability, but even with autocorrect it’s still clearly showing its age and lacking in UX polish. HeliBoard, meanwhile, is based on the AOSP keyboard (rather than being fully bespoke), which is much more clean and modern if perhaps light on features.

i’m very accustomed to my heavily‐customized, high‐density, ortholinear Colemak layout. idk how other people live their lives without top‐level hyphen and apostrophe keys honestly. the bottom row of keys, besides housing a variety of handy functions including quick cursor and clipboard controls, also pushes the spacebar and other keys further up, which is much more comfortable for me. i also keep an absurd amount of Unicode characters handy because i’m the kind of asshole who will actually use U+2014 EM DASH — with hair spaces! — instead of just putting spaces around a hyphen‐minus like a plebian. there are various other practical applications, but i also just like being able to call someone a ♛ or pull out my custom 〜⁠⁎✧𝆯❈ ⁠sarcasm glitter ⁠❈𝆯✧⁎〜.

in adapting this layout for HeliBoard, i also came up with a new colour scheme for it, one that’s much less saturated. it’s possible i was influenced by seeing @platinumtulip’s choice warm, muted tones in their artwork and desktop themeing so much recently, but it could also just be a natural consequence of me trying to make something that feels more at home in a Material Design 3 environment. (HeliBoard’s maintainer will not be adding MD3 styling directly because they don’t consider it to be worth the extra 1.5 MB it’d add to the app size. i respect this stance mostly because not nearly enough devs take application size into strong consideration anymore. on that note though, one of Multiling O Keyboard’s most impressive achievements is its app size of less than a megabyte!)

while HeliBoard supports enough of my eccentric modifications now to be useful (its most recent update finally added word left/right keys), it’s definitely still not perfect, and some of my changes i had to implement in very roundabout ways or weren’t easy to figure out. but it’s still a definite improvement, and actually maintained. there are a bunch of features missing compared to Multiling O Keyboard though, which will take time to adjust to not having:

  • pressing shift to make cursor controls select text;
  • slide keys (slide from a key in up to 8 directions for alternate characters/functions);
  • multi‐tap keys;
  • key modifying rules (press one key to temporarily change the function of another key as specified);
  • text transformations, including custom ones (HeliBoard transforms selected text between lowercase, UPPERCASE, and Sentence Case when pressing shift, but that’s it);
  • 4+ word suggestions (Multiling O Keyboard just stuffs as many as it can into the toolbar);
  • total theme customization (far more options for size, gaps, roundness…);
  • calculation suggestions (e.g. type “4+3” to receive “=7” as a word suggestion);
  • ⌦ forward delete key (normally i keep one where the 🤚︎ one‐handed toggle key is right now);
  • dedicated word delete key, plus an option to hold ⌫ backspace for word delete. (HeliBoard instead has a swipe gesture on the backspace key for large variable deletions, which i am finding to be useful, though i’m not sure i prefer it.)

the other big thing HeliBoard doesn’t technically have that Multiling O Keyboard does is swipe/glide typing. as it turns out, there are zero open‐source swipe/glide typing libraries, so HeliBoard instead supports some cockamamie method of importing an extracted library from a closed‐source app like Gboard using dynamic code loading. the author acknowledges that this is naturally a fairly substantial security risk, and so provides builds without support for this; i’m opting to use those.

i think i’m ready to move on from Multiling O Keyboard though. it’s always had some bugs and oddities that were a constant source of annoyance, like errors when typing too quickly, poor support for astral plane characters, and documentation that’s so scattered and poorly‐maintained that i can only access some critical parts of it via the Internet Archive now. i’m also assuming HeliBoard will get attention much more consistently given that it’s open‐source. now i just need to get around to finishing my rewrite of my PC keyboard’s firmware…


i use GrapheneOS on my Pixel 8. this OS bundles its own “app store”, which normally just hosts their own bundled apps and some mirrors of Google software (including their sandboxed Google Play Services). but recently, i noticed GrapheneOS added something called Accrescent to their app store. turns out it’s an app store.
huh?
it seems the eventual plan was for GrapheneOS to expand its in‐house app store to let developers upload their own apps. given the dire state of open‐source app repositories, this would’ve been the only solution to meet their stringent security requirements. (in a similar vein, they maintain their own camera and PDF viewer apps. while these are bundled with GrapheneOS, you can also get them on their Github and Google Play.) but now they’re delegating this to Accrescent, an in‐development app store that meets their own expectations for security and privacy. this seems very sensible, as such a project needs a lot more unique resources than a simple offline app — developer platform, app reviewers, servers… all this can be easily managed by an external team without taxing the GrapheneOS project directly. including Accrescent in the GrapheneOS app store is significant though, because it allows GrapheneOS users to maintain the chain of trust established by their OS, rather than having to trust another source to obtain Accrescent from. there’s a lot of cool work being done here!
well, it’s certainly a much better experience than F‐Droid, even if it is pretty barren because of its pre‐stable state. i spotted Aves in there (the great gallery app i started using after Simple Mobile Tools got bought by a company that infested those apps with ads and trackers), so i figured the other apps in there must be pretty good too.

this is how i was led to an open‐source, offline voice‐to‐text app, something else i’ve been passively interested in for a while. for some godforsaken reason it’s called “Transcribro” but whatever. it works! quite well!

in fact it works so well that it wraps around to being a bit hard to use, somehow. see, it automatically does punctuation based on the tone of the speech, which is hard for me to adjust to. i’m very used to speaking differently when using voice‐to‐text, using my speech more deliberately as a convenient mechanism for producing words and punctuation rather than a highly‐nuanced form of social communication. this means not using tone and pauses as normal (and so speaking punctuation instead, which doesn’t work here), speaking very slowly as i form sentences more deliberately in my head, and not using filler words. it’s not like i have to talk to my phone exactly like how i would talk to a person now, but it’s still much closer than i want it to be. i get that this method is probably preferable for quite a few people, but given that i can already struggle with real‐time verbal conversations…

there were no misheard words in my initial tests though, which was quite surprising to me. naturally, i had to look up this app to see how it worked. this led me to discover that it’s basically a wrapper around a wrapper around an OpenAI model. i’ve… had to grapple with that one a bit. of course, speech‐to‐text is one of the proper uses for neural network models, where there is correct, data‐light output to pair with input for training. this also seems to be using one of the models OpenAI created before they started completely ignoring the “open” in their name for all practical purposes: the Whisper family of models is made available under an MIT license, and afaict all the data used to train them are plainly listed. i’m also using this exclusively for personal purposes, and OpenAI isn’t directly making any money off of anyone here.

you’d think that just storing an entire model directly on‐device means that Transcribro takes up a ton of space, but at most it seems to be only twice as large as Gboard. which of course means that Gboard takes up a truly absurd amount of space, considering that the model being used here is ⁓70 MiB (which is like, 99.9% of Transcribro’s app size).

one quirk of how this all works is that, in contrast to the online speech‐to‐text services i used a long time ago, it’s not real‐time at all. that is, after i’m done speaking, it takes a few seconds before all the text produced from my speech is suddenly dumped into the text field all at once, rather than typing out every word as it’s said. i think the model technically always processes 30 seconds of audio at a time, making it fairly rigid that way. that’s fine by me, though; most of the situations where i’m using speech‐to‐text are ones where i’m already not looking closely at my phone screen, nor terribly concerned about immediacy.

idk i’m just really happy that there’s a lot more just, really good open‐source Android software now. stuff that just works! and has standard, non‐eccentric UIs. it’s nice. i talked about Auxio a while ago too. maybe i should talk about more apps that i like using?


You must log in to comment.
Pinned Tags