[BRLTTY] Footsteps towards better accessibility in Linux

[Aura Kelloniemi]

Hi,

On 2025-04-03 at 15:42 -0400, Nicolas Pitre <nico at fluxnic.net> wrote:

First of all, thank you for your comprehensive reply!

 > On Tue, 1 Apr 2025, Aura Kelloniemi wrote:
 > > Most of us, I believe, are using the Linux console daily. It works quite
 > > nicely for most of us, I think. There are some issues though: most
 > > importantly, sighted people generally don't use Linux console nowadays.
 > > Linux's console infrastructure is pretty much deprecated and the code receives
 > > very little maintenance.

 > I didn't see any deprecation notice yet. And the code receives little 
 > maintenance because for the most part it just works!

Yes it has not been officially deprecated, the situation is more like that the
Linux console is being abandoned by users.

20 years ago people still actively used Linux console in daily practices and
at that time limited Unicode support was added. Also the console saw porting
from the VGA to kernel mode setting, which was a big project.

Nowadays people mostly don't use Linux console except in emergency situations
when the graphical environment does not work.

New services generally don't take Linux console's existence into account
(upower for example). Most terminal programs are not tested on Linux console
and many developers even refuse to try their applications on the Linux
console, even though it is just one keystroke away.

In addition to the Unicode issues (which I discuss below) and the already
mentioned bracketed paste, there is no support for the alternate screen. Many
new terminal applications simply expect that it is available, and garble the
display in Linux console.

 > > There are cricital bugs in the Linux console affecting us. One of them is that
 > > many Unicode double-width and zero-width characters are not recognized as such
 > > which leads to serious rendering issues with all applications when these
 > > characters (like emojis) are printed to the console.

 > When I became annoyed enough with not being able to benefit from Unicode 
 > support while using BRLTTY on the Linux console, I just rolled up my 
 > sleeves and implemented the needed functionality:

I am very grateful for that, I guess it is very important for many of us.

 > Now if you have specific examples of double-width Unicode values that 
 > don't display properly then please share them and I'll be happy to have 
 > a look. That's what I'm willing to contribute.

All new double-width and zero-width Unicode code points which have been
introduced since Unicode 5.0. That is where the console driver is at the
moment. I don't know where I could get the list of these code points.

Also, Linux console does not support combining characters (which I need), nor
BiDi text (which I don't need, but many do). Proper Unicode rendering (without
alignment issues/terminal garbling) requires supporting tehm.

In Unicode a code point and a grapheme cluster are a very different thing, and
there would be a lot to do in order to get proper Unicode support to the
console.

If you try to edit files containing a lot of emojis, you will see the
rendering issues. When there are a lot of zero-width characters (like
"variation selectors"), long lines get wrapped, but any Unicode-aware editor
thinks that the content was rendered properly and its rendering logic starts
to work in very bad ways. Combine this with tmux or screen, and there is a
huge mess going on in the terminal.

Also, text which uses combining diacritics has the same effect as text with
zero-width characters as programs expect the characters to take less columns
than what they actually do.

To solve these issues, Emacs does not render any characters on Linux console
for which there is no glyph in the console font. This of course is just a
partial fix and it does not work inside terminal multiplexers.

 > There is also the issue of BRLTTY itself trying to remove the space 
 > filler used to double the width of such characters messing up alignment 
 > with other elements on the screen, in which case this would be BRLTTY 
 > itself the culprit. Are you sure this isn't your case?

There is a brltty.conf option to remove this feature of BRLTTY, and I have
enabled it consistently.

 > > The other problem important to many of us is that Linux console does not
 > > support extended keyboard input, like shifted cursor keys. This is very
 > > limiting especially in newer termianl applications. Using Org-mode for example
 > > without shifted/CTRLed/METAed cursor keys is unpleasant. As people who mostly
 > > do not use mouse, good keyboard input support is a top priority.

 > Just try this. Create a file called, say, custom.map and add those two 
 > lines to it:

I have implemented a partial solution that kinda works for myself, but it is
far from being complete. Someone had a more polished solution on the internet,
but I can't find it anymore.

The main problem is that there are not enough function key entries in the
kernel to have a binding for all possible combinations of regular keys and
modifiers. And because people have different needs (some may want to have
Shift-F1, while others would prefer Shift-Ctrl-Alt-Tab) there will be no
universal solution until this limitation is lifted.

Even better would be to add an escape sequence for all possible key/modifier
combinations. But this would take more memory, and because there is a clear
logic for how the modifiers effect the produced escape sequence, programming
that logic would be better.

 > So there you have it: a makeshift bracketed paste that takes only 30 
 > seconds to implement. Works directly in the Linux console.

True. The only problem is that it leaves the bracketing state management to
you and thus the implementation is not very end-user friendly.

 > If you do test those hacks and turn them into proper proposals, and 
 > advocate for those to be part of a distribution to the right maintainer, 
 > then eventually they'll be part of a distribution. It is that easy.

If I am right, distributions generally don't bind extra function keys in
console keymaps, but leave them for user configuration. All of these hacks
would use up all the function key entries and leave no room for customization.

 > > Linux console experience for sighted users is even worse than for us. The
 > > Linux console font can have at most 512 different glyphs (which is not enough
 > > nowadays). Also the colour support is limited and buggy. Linux console does
 > > not support text attributes, such as underline, bold, standout, etc.

 > True, although the 512 limit comes from the old VGA support. Nowadays the 
 > Linux VT console is mostly using a graphical framebuffer. So this limit 
 > is pretty much arbitrary.

Lifting this limit would be non-trivial though. It would require
reimplementing the /dev/vcsa devices, and would probably require changes to
kernel's internal glyph representation, font management IOCTLs, font
management utilities and to the console font file format.

 > At some point there were talks about simply moving console support to 
 > user space entirely. There is in fact many possibilities out there but 
 > according to https://bylr.info/articles/2022/10/29/til-textual-fb-term/ 
 > this isn't gaining much traction.

Could this be because people are using terminal emulators within their
graphical environments? At least the Linux userland is strongly pushing people
to desktops.

 > BRLTTY itself has a built-in console emulator which can be easily 
 > extended to keep track of whatever text attributes you want for direct 
 > consumption for braille output. Certainly much much easier than trying 
 > to interface with whatever other terminal emulators out there. It simply 
 > needs someone to volunteer for the work.

Yes, it is easier, but it has open questions:

1) How to handle the menus/dialogs of the terminal emulator?
2) What is the performance/memory consumption impact of doing the terminal
rendering twice (once by brltty-term, once by the GUI emulator)?
3) What terminal types does brltty-term support?
4) What is the performance/memory usage impact of running a separate BRLTTY
instance for every terminal window/tab? Even if this is tolerable on PC, is it
tolerable on lower-end ARMs.

 > [ skipping over the GUI stuff ]

There are also things related to the user session functionality of desktops,
which would be very useful, and which work only partly in console. I once
tried to have a working notification system in the console, but could not get
even the D-bus messaging part working.

 > > More important question of course is, is there something we can do about it.

 > Why not both?

Of course both, if we have resources.

Of course adding a few new code points to kernel's console driver table is not
that big a project. But projects like accessing terminal emulator dialogs with
braille (without orca) or adding proper Unicode support to Linux console would
be much more intense.

I was asking about the funding options because it might be quite an optimal
way to proceed with big tasks.

 > Sponsorship usually comes with expectation of some benefits in return 
 > (like visibility, publicity). Otherwise this is called charity.
 > Personally, I know of none.

How were AT-SPI, ATK and GAIL implemented? They required a lot of coding.

 > > Does somebody know, if the funding options have been thoroughly evaluated and
 > > how easy/difficult it would be to get even one developer a long-term payment
 > > for working with accessibility?

 > Such evaluation, too, comes with someone dedicating time to do it.

Yes, of course, and it might be suitable work for someone who perhaps does not
do coding, and especially someone who has a good relationship with a
JavaScript-enabled web browser.

 > What are you eager to volunteer for?

Mostly the things I hav been doing so far: discussing, designing features,
brainstorming, reporting bugs, testing, debugging, coding some minor things.

The amount of time I have available for this (unfortunately) is very limited.

-- 
Aura