Retry send if only some bytes sent

[tannewt]

Fixes #6654 and fixes #6689

[dhalbert]

This looks good. But if sent ==0, then i think the connection has been closed? Should this handle that somehow?

[tannewt]

This looks good. But if sent ==0, then i think the connection has been closed? Should this handle that somehow?

I don't think sent can be 0. It'll return an error instead.

[RetiredWizard]

Looks like you nailed this one. I've done many refreshes from both a solid network connection (right next to the router) and a less solid connection and the file listing has showed up every time. I did get a couple net::ERR_CONNECTION_RESET messages in the developer console screen but the file manager seemed to operate fine regardless of that message being displayed.

[dhalbert]

I was thinking about received count being zero, sorry! :forehead-slap:

[RetiredWizard]

While testing this on the ESP32-C3 and ESP32-S3 I noticed a possible issue. When I run the following code:

 from supervisor import runtime
 i=0
 print("Press any key to stop output")
 while runtime.serial_bytes_available == 0:
     i+=1
     print(str(i)+"012345678"*5+"abcdefghijklmpqrstuvwxyz")

in the Web Workflow serial terminal it will run for a short time (how long varies from run to run and depends on the particular ESP chip) and then stop. The code will properly exit if a key is pressed. The behavior varies somewhat depending on whether a usb serial terminal is connected or not. I believe that there's a display buffer that's being filled at which point Circuit Python attempts to pause the program until the Web Workflow serial terminal can catch up. My guess is that while the serial terminal is catching up, web workflow ends up yielding program control to background operations and control isn't returned until a key press is detected.

I modified the web_workflow_send_raw function as follows:

    if (sent > 0) {
        total_sent += sent;
        if (total_sent < len) {
            // Yield so that network code can run.
            port_idle_until_interrupt();
        }
    } else if (sent == -EAGAIN) {
        port_idle_until_interrupt();
    }

This resulted in the display stopping sooner and much more consistently than the original version.

[RetiredWizard]

Moving the C3 close to the router and running the test script resulted in the program running much longer than it was with the weaker network connection. This is making me think my original guess that there was a display buffer being filled may have been off and network errors may actually be the triggers causing Web Workflow to yield control.

[RetiredWizard]

I removed the extra call to port_idle_until_interrupt and added an ESP_LOGE print just before the original call to port_idle_until_interrupt.

The output of the program stops after the second call to port_idle_until_interrupt. The number of lines that print varies from a few hundred to a few thousand but there are always two calls to the idle function and the program appears to stops just after the second call is made.

[RetiredWizard]

When the output has stopped on the serial terminal, any Web Workflow activity on another open browser window (i.e. clicking on a link in the Web Workflow file manager) will restart the output until two more idle calls are made.

[tannewt]

When the output has stopped on the serial terminal, any Web Workflow activity on another open browser window (i.e. clicking on a link in the Web Workflow file manager) will restart the output until two more idle calls are made.

Ah, I have an idea. The idle call causes FreeRTOS to yield. The socket select task should wake CP but I don't think I have it set to do that when a socket is writable. I'll look into it.

[RetiredWizard]

Looks like you got it! The output seems to be running indefinitely now.

[dhalbert]

Thanks again @RetiredWizard !