[BRLTTY] Possible Tactile Graphics Support in the BRLAPI Protocol

[kperry at blinksoft.com]

You might want to talk to Orbit they have a protocoal they give out if you
want it.  The monarch does not have a public protocol yet. We really should
get an over all tactile graphics protocol so we don't end up in the braille
display hell we are in.  The problem is there is no one size fits all
protocol yet.  I have the Orbit one and it is a pretty good start that could
be added to the Brialle USB HID api.

-----Original Message-----
From: BRLTTY <brltty-bounces at brltty.app> On Behalf Of Elijah Massey
Sent: Sunday, June 15, 2025 5:13 PM
To: brltty at brltty.app
Subject: [BRLTTY] Possible Tactile Graphics Support in the BRLAPI Protocol

Hello,
I have started writing an implementation of the brlapi protocol in pure
Rust, that directly parses and writes the packets instead of using
libbrlapi. One of the things I want to use it for is writing a brlapi server
for the Monarch, since I figured out how to display any dot matrix from
within an Android app, and capture all key events from the Braille keyboard
and panning, arrow, and zoom keys, and how to capture the locations of
double taps (when you hold your finger somewhere on the display and press
the circle button twice). So after my protocol library is done, I can write
a server that manages connections from clients, keeps track of the state of
the display, converts key presses to brltty commands, etc, and hook that up
to the display and keys of the Monarch. Then BRLTTY could connect to it over
WiFi with the ba driver and display multi-line Braille, or Orca or NVDA
could connect to it directly and display Braille.

However, this wouldn't expose all the capabilities of the Monarch, because
the Monarch can display more than Braille dots. To display multi-line
Braille, my server would calculate the amount of spacing needed around the
Braille, and leave those dots unused, like the driver for the Dot Pad does.
But when displaying tactile graphics, those unused dots are very useful, and
there doesn't seem to be a way to expose that over brlapi. I could use raw
mode with a custom packet format, but then only one application could use
the display at a time, because raw mode gives exclusive control to one
client ignoring the priority system and the TTY paths. This solution would
also mean that displaying tactile graphics on the Dot Pad would be done
differently than displaying them on the Monarch, unless I just implement the
Dot Pad protocol for my raw mode. But then if the Cadence or something else
has tactile graphics support, then there would be the same problem.

Do you think it would be a good idea to add packets to the brlapi protocol
for treating supported displays as arrays of equidistant pins? Maybe a
second getDisplaySize packet that returns its width and height in pins, and
a writeDots packet that takes a starting x value, a starting y value, a
width, a height, and an array of 0s or 1s, which sets a section of the
display pins. It might also be good to be able to get the parameters of the
Braille spacing somehow, so an application could display both text and
graphics letting brltty or my brlapi server handle Braille spacing, and
still know enough so they don't overlap with each other. This would allow
people to write graphics viewers as brlapi clients that work with all
current and future tactile graphics displays, or add image display support
to screen readers like Orca or NVDA, or add image support to brltty on
Android.

Sent from my iPad
_______________________________________________
This message was sent via the BRLTTY mailing list.
To post a message, send an e-mail to: BRLTTY at brltty.app For general
information, go to: http://brltty.app/mailman/listinfo/brltty