-
Notifications
You must be signed in to change notification settings - Fork 1.3k
ParallelImageCapture: Add continuous capture on espressif #5540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
By having a pair of buffers, the capture hardware can fill one buffer while Python code (including displayio, etc) operates on the other buffer. This increases the responsiveness of camera-using code. On the Kaluga it makes the following improvements: * 320x240 viewfinder at 30fps instead of 15fps using directio * 240x240 animated gif capture at 10fps instead of 7.5fps As discussed at length on Discord, the "usual end user" code will look like this: camera = ... with camera.continuous_capture(buffer1, buffer2) as capture: for frame in capture: # Do something with frame However, rather than presenting a context manager, the core code consists of three new functions to start & stop continuous capture, and to get the next frame. The reason is twofold. First, it's simply easier to implement the context manager object in pure Python. Second, for more advanced usage, the context manager may be too limiting, and it's easier to iterate on the right design in Python code. In particular, I noticed that adapting the JPEG-capturing programs to use continuous capture mode needed a change in program structure. The camera app was structured as ```python while True: if shutter button was just pressed: capture a jpeg frame else: update the viewfinder ``` However, "capture a jpeg frame" needs to (A) switch the camera settings and (B) capture into a different, larger buffer then (C) return to the earlier settings. This can't be done during continuous capture mode. So just restructuring it as follows isn't going to work: ```python with camera.continuous_capture(buffer1, buffer2) as capture: for frame in capture: if shutter button was just pressed: capture a jpeg frame, without disturbing continuous capture mode else: update the viewfinder ``` The continuous mode is only implemented in the espressif port; others will throw an exception if the associated methods are invoked. It's not impossible to implement there, just not a priority, since these micros don't have enough RAM for two framebuffer copies at any resonable sizes. The capture code, including single-shot capture, now take mp_obj_t in the common-hal layer, instead of a buffer & length. This was done for the continuous capture mode because it has to identify & return to the user the proper Python object representing the original buffer. In the Espressif port, it was convenient to implement single capture in terms of a multi-capture, which is why I changed the singleshot routine's signature too.
554e323
to
a8614a6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor suggestion. Up to you if you want to add more.
|
||
|
||
//| def continuous_capture_stop(self) -> None: | ||
//| """Stop continuous capture""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably mention what happens to the buffers as well. I'd assume you'd delete their reference so they may be garbage collected after this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I added text to cover this. Needs a fresh review now!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
By having a pair of buffers, the capture hardware can fill one buffer while
Python code (including displayio, etc) operates on the other buffer. This
increases the responsiveness of camera-using code.
On the Kaluga it makes the following improvements:
As discussed at length on Discord, the "usual end user" code will look like
this:
However, rather than presenting a context manager, the core code consists of
three new functions to start & stop continuous capture, and to get the next
frame. The reason is twofold. First, it's simply easier to implement the
context manager object in pure Python. Second, for more advanced usage, the
context manager may be too limiting, and it's easier to iterate on the right
design in Python code. In particular, I noticed that adapting the
JPEG-capturing programs to use continuous capture mode needed a change in
program structure.
The camera app was structured as
However, "capture a jpeg frame" needs to (A) switch the camera settings and (B)
capture into a different, larger buffer then (C) return to the earlier
settings. This can't be done during continuous capture mode. So just
restructuring it as follows isn't going to work:
The continuous mode is only implemented in the espressif port; others
will throw an exception if the associated methods are invoked. It's not
impossible to implement there, just not a priority, since these micros don't
have enough RAM for two framebuffer copies at any resonable sizes.
The capture code, including single-shot capture, now take mp_obj_t in the
common-hal layer, instead of a buffer & length. This was done for the
continuous capture mode because it has to identify & return to the user the
proper Python object representing the original buffer. In the Espressif port,
it was convenient to implement single capture in terms of a multi-capture,
which is why I changed the singleshot routine's signature too.