Esp32s2: implement ParallelImageCapture

[jepler]

This is tested as working with

• The ESP32-S2-Kaluga devkit, version 1.3, with an ili9341 display
• An unmodified OV7670 camera plugged into the camera header (Update for incompatible upstream changes Adafruit_CircuitPython_OV7670#8)
• A modified version of adafruit_ov7670 (this incompatibly changes the API of ParallelImageCapture)
• some scripts on CIRCUITPY

import time
import board
import busio
import displayio
import adafruit_ili9341

displayio.release_displays()

#spi = busio.SPI(MISO=board.LCD_MISO, MOSI=board.LCD_MOSI, clock=board.LCD_CLK)
spi = busio.SPI(MOSI=board.LCD_MOSI, clock=board.LCD_CLK)

display_bus = displayio.FourWire(spi, command=board.LCD_D_C, chip_select=board.LCD_CS)

kw = dict(width=240, height=320)
def init_with_madctl(madctl):
    return adafruit_ili9341._INIT_SEQUENCE.replace(b'\x36\x01\x38',
            bytes([0x36, 1, madctl]))

display = displayio.Display(display_bus,
    init_with_madctl(0b010_110_00), width=320, height=240)

from adafruit_ov7670 import (
    OV7670,
    OV7670_SIZE_DIV1,
    OV7670_SIZE_DIV16,
    OV7670_TEST_PATTERN_COLOR_BAR,
    OV7670_TEST_PATTERN_SHIFTING_1,
    OV7670_TEST_PATTERN_COLOR_BAR_FADE,
)
from displayio import (
    Bitmap,
    Group,
    TileGrid,
    FourWire,
    release_displays,
    ColorConverter,
    Colorspace,
)


bus = busio.I2C(scl=board.CAMERA_SIOC, sda=board.CAMERA_SIOD)

# Set up the camera (you must customize this for your board!)
cam = OV7670(
    bus,
    data_pins=board.CAMERA_DATA,
    clock=board.CAMERA_PCLK,
    vsync=board.CAMERA_VSYNC,
    href=board.CAMERA_HREF,
    mclk=board.CAMERA_XCLK,
    mclk_frequency = 20_000_000,
)
#cam.test_pattern = OV7670_TEST_PATTERN_COLOR_BAR
width = display.width
height = display.height

bitmap = None
# Select the biggest size for which we can allocate a bitmap successfully, and
# which is not bigger than the display
for size in range(OV7670_SIZE_DIV1, OV7670_SIZE_DIV16 + 1):
    cam.size = size
    if cam.width > width:
        continue
    if cam.height > height:
        continue
    try:
        bitmap = Bitmap(cam.width, cam.height, 65535)
        break
    except MemoryError:
        continue

print(width, height, cam.width, cam.height)
if bitmap is None:
    raise SystemExit("Could not allocate a bitmap")

g = Group(scale=1, x=(width - cam.width) // 2, y=(height - cam.height) // 2)
tg = TileGrid(
    bitmap, pixel_shader=ColorConverter(input_colorspace=Colorspace.RGB565_SWAPPED)
)
g.append(tg)
display.show(g)

t0 = time.monotonic_ns()
display.auto_refresh = False
while True:
    cam.capture(bitmap)
    t1 = time.monotonic_ns()
    bitmap.dirty()
    display.refresh(minimum_frames_per_second=0)
    t1 = time.monotonic_ns()
    print("fps", 1e9 / (t1 - t0), end="\r")
    t0 = t1

Note that with a DEBUG=1 build and the Kaluga's pinout, CircuitPython will just crash when the camera module drives its "D4" pin, since this pin is shared with the JTAG peripheral! You just have to NOT use debug builds 🤕

[jepler]

@ladyada this is ready for testing if you're inclined & have the hardware. I also need to re-test the other boards, since the constructor was changed and I shouldn't be too confident I didn't introduce problems there.

[ladyada]

ok ill get a kaluga 1.3

[jerryneedell]

@jepler I tried this code with the updated ov7670 library on my Kaluga 1.3 -- It "works" but the display is a bit confused.
I wonder if my display board is the same as yours. I am using the board that came with the Kaluga.
Here is the funky image

[jerryneedell]

hmm -- If I swap the height/width setting , then I get a normal image, but the camera size is reduced to 160 x 120 -- progress
FWIW -- my display board is the ESP-LyraP-LCD32 v1.2

Adafruit CircuitPython 7.0.0-alpha.3-55-g2cbdd18a2 on 2021-06-10; Kaluga 1 with ESP32S2
>>> 
>>> 
>>> import ov7670_test
240 320 160 120
fps 6.25582

[jerryneedell]

Going back to the 320x240 (as opposed to 240x320) but adding a 90 degree rotation to the displayio init gives me a full screen. I guess it is just something about this display.

Adafruit CircuitPython 7.0.0-alpha.3-55-g2cbdd18a2 on 2021-06-10; Kaluga 1 with ESP32S2
>>> 
>>> 
>>> import ov7670_test
320 240 320 240
fps 2.50424

full screen picture

[jerryneedell]

@jepler On my display, I found that if I use the "standard" ILI9341 Init sequence then I get the full display filling the screen similar to what I get with the 90 degree rotation with your init sequence.
In your modification to in the init sequence, you are changing the MX and MV bits of the Memory Access control register from the defaults. Was that just trial and error for your display? I now see that by manipulating the My.MX,MV bits I can get my image to fill the display but it is rotated with respect to the way the camera is mounted. If I change the bits, I can reorient the display, but it does not fill the screen... so many controls .... ;-) I added a flexible cable so I can reorient the camera. Nor I have the full screen in the proper orientation! At least this seems to make some sense the actual image capture seems to be working fine. I just need to understand the display and camera configurations better.

Also, just to satisfy my curiosity why is the color depth in your example set to 65535 and not 65536? It does not seem to make any difference, but if it is a 16 bit value, shouldn't it be 65536?

[jepler]

OK, I tested with rp2040 and samd51 and they both work after updating the lib & example!

[ladyada]

tested with great success on esp32s2