Add keyboard_layout_base and switch keyboard_layout_us to it

[Neradoc]

This is the KeyboardLayoutBase class used in the layout bundle.

• ASCII_TO_KEYCODE works the same, with the shift-key bit.
• NEED_ALTGR lists characters that require the altgr key in addition.
• HIGHER_ASCII is a dictionary that associates the ord() int value of high ascii and utf8 characters of one or more bytes to the keycode.
• COMBINED_KEYS is a table (dictionary) of characters like Ñ that can be accessed by typing first a dead key (like ~) followed by a regular key (like N) - on a Windows French keyboard for example.
  • Indexes are ord() int values of each character.
  • The value is an int encoding 2 bytes.
  • The upper byte is the keycode of the first key, including the shift bit.
  • The lower byte is the ascii code of the second key (a-z, A-Z or space).
  • The lower byte's high bit also encodes whether or not the first key needs altgr, so as not to pack NEEDS_ALTGR with a bunch of characters.

With the Ñ example:

• ~ is altgr + é, é is keycode 0x1F
• N is ascii code 0x4E, with the altgr bit becomes 0xCE
• Ñ is therefore 0xD1: 0x1FCE,
• When typing, "N" is looked up: ASCII_TO_KEYCODE[0x4E] == 0x91 (shift+0x11)
• So write("Ñ") sends the keycodes: (0xE6, 0x1F) (altgr+é) then (0xE1, 0x11) (shift+n)

In the Keyboard Layout bundle, every layout class is called KeyboardLayout, to make changing layout easier without having a different class name in every file, and have to guess or lookup the class name (is it CamelCase of the module name ? Is it US or Us ?). For compatibility, I added KeyboardLayout = KeyboardLayoutUS in the US keyboard layout. Importing multiple layouts is possible by using the full module.KeyboardLayout name, or import KeyboardLayout as ...

I'm not quite sure if I did the documentation correctly, I would love guidance on that. KeyboardLayoutBase is not for end-user use, but it's now the class that contains the method calls of the layout and therefore their documentation.

[dhalbert]

Thanks for tying this all together with your new layouts!

[dhalbert]

Could you add a docstring after each these, explaining what they do, perhaps with a simple example? The description you gave in the original PR post is a good starting point, but could be expanded.

Why is there not a problem with SHIFT and ALTGR using the same bit?

[Neradoc]

The shift bit is added to keycodes (in ASCII_TO_KEYCODE, HIGHER_ASCII and the high byte of COMBINED_KEYS), the altgr bit is added do the the Combined Keys codes. Being 0x80 it fits in the low byte (which is a low ascii character). So they are the same bit but don't go in the same place.

It's a little tricky, since the altgr modifier is for the first key (the dead key) but is packed in the byte for the second key. It's all a trick to avoid adding all those letters to NEEDS_ALTGR or having a third byte for the altgr flag, by packing all that into existing bytes.

The combined-key codes bits are: 0b SDDD DDDD AKKK KKKK

• S is the shift bit for the first key,
• DDD DDDD is the keycode for the first key,
• A is the altgr flag for the first key,
• KKK KKKK is the (low) ASCII code for the second character.

[dhalbert]

Thanks for the great explanation! Could you add what you wrote to a docstring (more or less), or an extended comment?

[Neradoc]

I've pushed an update with a bunch more docs and rebased, and the type annotations.

[dhalbert]

I tested this on a CPX, by freezing your branch instead of the main-line module.

I see one issue, of sending control characters. In 7.0.0, with the regular frozenadafruit_hid, this works:

...
>>> l = KeyboardLayoutUS(k)
>>> l.write("abc\ndef")
>>> abc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
>>> def

But with your branch, the \n does not appear:

>>> l.write("abc\ndef")
>>> abcdef

Interestingly, "abc\bdef" does work, and shows abdef.

Could you try this?

[dhalbert]

It is the interior \n that is disappearing:
with this change:

Adafruit CircuitPython 7.0.0-579-g9da541ed2-dirty on 2021-10-26; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_hid.keyboard_layout_us import KeyboardLayoutUS
>>> from adafruit_hid.keyboard import Keyboard
>>> import usb_hid
>>> k = Keyboard(usb_hid.devices)
>>> l = KeyboardLayoutUS(k)
>>> l.write("abc\ndef\n")
>>> abcdef
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abcdef' is not defined
>>> 

Latest stock adafruit/main:

Adafruit CircuitPython 7.0.0-579-g9da541ed2 on 2021-10-26; Adafruit CircuitPlayground Express with samd21g18
>>> from adafruit_hid.keyboard_layout_us import KeyboardLayoutUS
>>> from adafruit_hid.keyboard import Keyboard
>>> import usb_hid
>>> k = Keyboard(usb_hid.devices)
>>> l = KeyboardLayoutUS(k)
>>> l.write("abc\ndef\n")
>>> abc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
>>> def
... 

[dhalbert]

This is really odd. Now I cannot reproduce this.

[dhalbert]

This is really odd. Now I cannot reproduce this.

It's intermittent; still trying.

[dhalbert]

There's some kind of race condition, unrelated to this PR. I'm seeing this kind of thing intermittently, on both stock 7.0.0 and with this PR. So nothing to do with the PR.

>>> l.write("abc\ndeg\n")
>>> abcdeg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abcdeg' is not defined
>>> time.sleep(1);l.write("abc\ndeg\n")
>>> abc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
>>> deg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'deg' is not defined
>>> 

[dhalbert]

Thank you for all your work on this!