Skip to content

DNS tests modification for ESP8266-specific scenario #11601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

michalpasztamobica
Copy link
Contributor

Description

The ASYNCHRONOUS_DNS_TIMEOUTS test floods the device with UDP requests (it skips the 100 ms delay to simulate instant timeout). ESP8266 starts responding with "busy p..." message. It needs more time to process the requests and recover for subsequent tests.

I have also increased the total suite timeout, not only to cover the delay I've just added, but also to allow larger time buffer as we've seen the tests have come really close to the borderline and they have a large variation, dependent on the network status.

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[x] Test update
[ ] Breaking change

Reviewers

@AnttiKauppila
@AriParkkila
@tymoteuszblochmobica
@SeppoTakalo
@VeijoPesonen

@ciarmcom
Copy link
Member

@michalpasztamobica, thank you for your changes.
@tymoteuszblochmobica @AnttiKauppila @AriParkkila @VeijoPesonen @SeppoTakalo @ARMmbed/mbed-os-ipcore @ARMmbed/mbed-os-test @ARMmbed/mbed-os-maintainers please review.

Copy link
Contributor

@VeijoPesonen VeijoPesonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't add device specific settings to the test code. In my opinion it would be better to increase the sleep time for all devices.

@michalpasztamobica
Copy link
Contributor Author

@VeijoPesonen , ok, I see your point here.
Do yo think it would be justified to leave a device-specific comment? Or should this information be only available through the commit log, after one runs git blame on the line containing the delay?

@AnttiKauppila
Copy link

AnttiKauppila commented Oct 1, 2019

Should the whole test be thought again because we know exactly how many callback calls we want/need. Something like,

static int counter = 0; //incremented in event_queue_call()

void ASYNCHRONOUS_DNS_TIMEOUTS()
{
... //as is
    // Give event queue time to finalise before destructors
    // ThisThread::sleep_for(2000);
    while (counter < (MBED_CONF_APP_DNS_SIMULT_QUERIES + 1)) { //timeout is also handled in callback
        ThisThread::sleep_for(1000);
    }
... //as is
}

This would work with all targets and shields

@michalpasztamobica
Copy link
Contributor Author

@AnttiKauppila , I thought the same, but actually this callback gets called more than a 1000 times throughout this test case. It is a callback for all possible queue events, not the final timeout event - a replacement for whatever is installed in the query.

The test floods the device with UDP requests (it skips the 100 ms delay to simulate instant timeout). ESP8266 starts responding with "busy p..." message. It needs more time to process the data and recover for subsequent tests.
@michalpasztamobica michalpasztamobica force-pushed the tests_dns_timeouts_esp8266 branch from 42a4bd0 to 3297445 Compare October 1, 2019 12:15
@michalpasztamobica
Copy link
Contributor Author

@VeijoPesonen , I changed the code as you suggested, leaving the reason behind the large number only in the commit message. I hope this is ok?

@AnttiKauppila , I also had another thought: what if we add a small delay in the "fake" event queue handler, to decrease the rate at which ESP is flooded with messages, but I noticed, that even without the delay we sometimes pass with only 1 timed out request (and sometimes with 5 of them) and I thought this is too narrow to failure...

Copy link
Contributor

@VeijoPesonen VeijoPesonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@AnttiKauppila
Copy link

@michalpasztamobica Checked the code again and it seems the real callback functionality you would need is hidden in main.cpp's hostbyname_cb() function. This would require a bigger task to fix, so let's put this PR in first. And we should fix things properly instead of tweaking/hacking/faking something to work.

@adbridge
Copy link
Contributor

adbridge commented Oct 7, 2019

CI started

@mbed-ci
Copy link

mbed-ci commented Oct 7, 2019

Test run: SUCCESS

Summary: 5 of 5 test jobs passed
Build number : 1
Build artifacts

@adbridge adbridge merged commit 51680cb into ARMmbed:master Oct 7, 2019
@michalpasztamobica michalpasztamobica deleted the tests_dns_timeouts_esp8266 branch October 23, 2019 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants