[BRLTTY] Footsteps towards better accessibility in Linux

[Aura Kelloniemi]

Hi Samuel et al.,

On 2025-05-10 at 19:02 +0200, Samuel Thibault <samuel.thibault at ens-lyon.org> wrote:
 > Aura Kelloniemi, le mer. 07 mai 2025 22:30:29 +0300, a ecrit:
 > > I mean the logic of mapping a (possibly very complex) combination of keycodes
 > > to commands. BRLTTYdoes it very well and it is far from simple to duplicate.

 > Do you mean e.g. the management of combinations of key presses and
 > keytable definition? Such as

 > bind LeftJoystickLeft+RightJoystickPress TOP

Yes, but also add to this all specialities BRLTTY supports, such as contexts,
immediate keys (with !), long presses, key event
repetition and commands being activated when a key is pressed and the same
command deactivated when the key is released.

These cannot be supported just by recording key press/release events and
tracking the time between them, because long key presses and key repeat require
some events to trigger mid-keypress. Implementing this requires precise timing
and good async/threading support, which might not be available for scripting
languages.

There are probably already libraries that can be used to implement all this in
a way that is completely agnostic to keyboard type (i.e. is not bound to
BRLTTY or braille displays). I remember seeing such a library for Rust, but
cannot find it with simple searching any more.

BRLTTY also supports mapping regular PC keyboard events to BRLTTY commands. If
BRLTTY somehow supported client-local key tables, the user could possibly bind
regular keyboard events to application commands.

Finally, BRLTTYs key mapping support is best in class – certainly better than
what Linux console offers, I don't know about xkb and it is non-trivial to
duplicate the logic in applications. Also BRLTTYs approach is so powerful that
it is likely suitable for almost every application out there.

 > > Or allow the application to tell BRLTTYwhich BRLTTY commands it wants to handle
 > > – commands like LNUP, FWINRT, HOME, etc.

 > Which is already the case. But apparently for the commands that you
 > don't want to handle, you want to receive the underlying raw keycodes,
 > instead of just drop them?

Yes, but I don't think it would be trivial to implement this so that BRLAPI
sends some key presses as BRLTTY commands and some as raw key codes, because
it is non-trivial to determine, whether one key press belongs to a command,
because it largely depends on what presses/releases will follow.

 > > Additionally the application could supply BRLTTY a model-specific key table
 > > with application specific commands.

 > Again, taking the xkb example, it'd seem to me simpler to let the
 > application perform the translation by itself, possibly using a library
 > helper for this, rather than having to tell the server the translation,
 > only for the server to perform the translation on behalf of the client.

Maybe, but then we need to give up support for mapping PC keyboard events
to commands. I don't know if this is crucial – depends on how much people
generally map regular keyboard keys to BRLTTY commands. I don't do this as I
have enough keys in my braille display, but for displays with less keys, this
might be necessary.

 > > This would allow BRLTTYs key press->command translation machinery to be reused
 > > by applications, would not require tremendous changes to BRLTTY

 > The difficult part is how to tell BRLTTY the desired translations. Which
 > essentially is the same kind of difficulty as defining a library helper
 > that does this.

It chould be done with new brlapi commands or using parameters.

We would need to export BRLTTY's key table management data structures to
client applications, and there would need to be a brlapi call for transferring
key tables from the client to the server and the other way around.

Some associated utility functions for binding/unbinding/listing commands would
go with this.

 > It's thus probably simpler to just do the translation on
 > the client side. That can e.g. make it simpler to have different binding
 > depending on application state etc.

Yes, if there is a library that the application can utilize to do this, as the
translation process is non-trivial as I pointed out.

But now that I think of it, providing a library for doing translation is not
easy, if the library needs to be usable from many different languages, because
the translation process requires async/threading capabilities, and writing
language and platform-agnostic portable concurrent code is not trivial.

I think easiest would be to do translation on server side (if the client
application is willing to provide key tables) or let the client do everything
by itself in whatever way it likes.

If the server translates the key codes to commands, the application needs to
register new command names (probably strings, some may need additional int
parameters or flags). The client would also need to provide new key tables.
There would also be a way to combine already existing tables from BRLTTY with
the new tables from the client application. Admittedly this is complex.

But if the client does the translation, all the same problems need to be
solved. Now the dut is left for the client though. BRLTTY already has code for
doing the translation and data structures for handling the mappings, but they
are more difficult to utilize from the client side, because the client might
need to run another thread to do the translation.

 > For a start, the piece that is missing is the combination of receiving
 > the desired commands *and* receiving keycodes that are not bound to a
 > desired command.

I don't know if there is any manageable way to accomplish this given he
complexity of key mappings.

 > And application will already be able to use
 > BRLAPI_PARAM_DEFINED_DRIVER_KEYCODES and BRLAPI_PARAM_DRIVER_KEYCODE_NAME
 > to get the set of raw keycodes and their names.

BRLAPI_PARAM_DRIVER_KEYCODE_NAME cannot tell anything more about the key than
its name. if I remember correctly, it does not work for routing keys at all –
for the first routing key it says the name is is "routing key" or similar, and
for all the rest it gives no name.

-- 
Aura