rykarn
@rykarn

Major distros are soon switching to versions of desktop environments that use Wayland instead of X11. This is a bad state of affairs for accessibility.


I use Talon Voice. It is a program that runs in the background that listens to spoken commands in your microphone, and performs certain actions on your computer based on what you say. I started using it earlier this year when I got RSI in my wrist which made it hard for me to type on a keyboard and use a mouse.
My wrist has healed up again and I am once more capable of typing on a computer (this text for instance). But my main source of typing is when I am programming, something I do for a living, and for that I still use Talon and the Cursorless extension. I could probably move back to using emacs with vim keybindings, the way I used to before my wrist issues, but I would prefer not to. Both because I would be worried to injure my wrist once more, and also because I have grown really fond of programming with Cursorless. I am as fast if not faster with voice coding compared to my ability to program with a keyboard (with a typing speed of about 85 WPM). Still, I am lucky that for me, Talon Voice is an option and not a necessity for interacting with a computer. Many others in the Talon Voice community do not have that privilege.

The Wayland protocol lacks (by design) a whole bunch of features needed for Talon and similar accessibility programs to function, things that are fully possible on Linux with X11, OSX and Windows. Things such as querying what windows are available, their titles, what window is focused, emulating keyboard input, querying for the mouse position. At the moment, these APIs would need to be provided by whatever Wayland compositor happens to be in use on a user's system. Since no common accessibility interface for these tasks exists between the currently available compositors you then run into the problem of getting the development teams behind the different compositors to agree on a standard. That, or require the single developer behind Talon to write his own compositor in order to support Linux.

This really fucking bums me out. For my home computer I'm good for a while because I use a distro where I can decide whether to use X11 or Wayland. At my current work we use a LTS release of a distro still on X11 with the intention of staying on it for as long as it is supported. So I'm good for now, but I do not look forward to potentially having to choose between ditching Talon Voice or ditching Linux altogether.

I'm sure that Wayland is a necessary step forward to get rid of the technical debt of X11, but moving forward now will leave the state of accessibility on Linux, and thus the people depending on it, behind. There are ongoing efforts to compile the issues faced by Talon Voice and contacting the teams behind the big distros and desktop environments, but any change that hopefully might occur from that is still a long way out.

(Note that I have no real insight into the nature of Wayland/X11 and how compositors work and so on, so my description of the technical challenges should be taken with a grain of salt. This is mostly based on what I have pieced together from a high level view of the Wayland architecture and the things mentioned by the Talon developer in the Talon community Slack channel.)


You must log in to comment.

in reply to @rykarn's post:

i think in 5 or 10 years when the dust starts to settle, we're going to see one wayland compositor stack win: wlroots. I don't see a way that multiple flavours of wayland survives in the long term, and wlroots has the most support in terms of 1. number of compositors, 2. number of ancillary tools, and 3. breadth of protocols for things like accessibility and more (no clue what the state of that kind of thing is currently)

i'm probably going to stick with xorg as long as i can too.

I hate how every big distro seems to charge forward on changing stacks like this when Wayland breaks most accessibility tools like, i don't need the desktop to look fancier that bad. Get this shit ready for prime time FIRST and then switch.

Agreed, and the tragedy of Wayland is that none of the problems you're experiencing were necessary to create a replacement to X11. These problems were designed in, deliberately, by zealots. They want to be as *un-*X11 as possible regardless of the consequences.

Note, the following should be taken with a huge grain of salt due to my very superficial understanding of wayland and the needs of Talon Voice and a11y interfaces in general:

I have not kept up much at all with developments save for what people sometimes post in the Linux channel in the Talon Voice slack channel. One development is Newton. Skimming it gives me the impression that it is focused on GTK and AT-SPI which is a way for a11y tools to interact with GUIs of individual apps. This is good, but Talon also needs access to the entire desktop state (what windows are present, interacting with the DE itself, emulating keyboard and mouse). Some of these things are brought up in the blog post also so maybe some of this work would lead to features that Talon needs. However, this is directly implemented in the Gnome compositor at the moment, it is not yet promoted to official wayland protocols or dbus interfaces that would allow a compositor-agnostic approach to a11y.

For reference, here are the requirements which I posted in another comment where the needs of Talon are outlined.

Based on what I can gather from the Talon developer, it is unlikely he will implement wayland support unless it can be done via official protocols and interfaces. He will not want to implement it for one specific compositor. I am well aware that he is not a fan of having to support Linux in general, it being the smallest user segment of Talon yet the biggest support burden, due to it being such a moving target (different display servers, certain DEs dropping system tray support, just so many different combinations of distros and window managers/DEs).

Pinned Tags