Skip to content

Adds CONTRIBUTING info for tests. #212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 24, 2015
Merged

Adds CONTRIBUTING info for tests. #212

merged 7 commits into from
Jul 24, 2015

Conversation

tcr3dr
Copy link
Contributor

@tcr3dr tcr3dr commented Jul 17, 2015

Needs work.

@tcr3dr tcr3dr mentioned this pull request Jul 17, 2015
8 tasks
For example, the addition of a new API item for retrieving a vehicle attribute would need test code for
getting, setting (if applicable) and printing the attribute. An easy way to create this would be to
update and run :ref:`example-vehicle-state`.
Web client tests use nose. First export a DRONEAPI_KEY in the format `<id>.<key>`. The run:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is nose? Should be a link. Similarly, the DRONEAPI_KEY should link to our cloud docs.

What are the web client tests? Worth a line explaining what these tests cover.

@hamishwillee
Copy link
Contributor

Need to add information about the different classes of tests and how to write for each of them as per #223

@tcr3dr tcr3dr added this to the v1.4.0 milestone Jul 21, 2015
@tcr3dr tcr3dr force-pushed the tcr-contrib branch 3 times, most recently from f1ad946 to 26adfaa Compare July 22, 2015 20:08
@tcr3dr tcr3dr changed the title [WIP] Adds CONTRIBUTING info for tests. Adds CONTRIBUTING info for tests. Jul 22, 2015
@tcr3dr
Copy link
Contributor Author

tcr3dr commented Jul 22, 2015

@hamishwillee I updated this doc with all suites of tests and addressed your comments. Feel free to review again.

These docs will accompany the 1.4.0 release (which is mostly just the addition of the test infrastructure).

@hamishwillee
Copy link
Contributor

Hi @tcr3dr

Look pretty good. I pushed some minor subedits for style structure. Please check you're happy with those.

Comments:

  1. This branch has no unit tests so I can't assess how useful the unit test advice is
  2. FYI, the theme I'm testing against doesn't differentiate 4th level headings, which is one reason I changed the heading levels (the other being I think it improves the structure).
  3. The web client is not actually part of the Python public API - as you will see in the API reference it is not documented. It would be, but too much was not implemented so we hid it. We might want to remove this altogether from docs until next version of the cloud API.
  4. The integration test advice says ""Avoiding printing any data from your test.". However test_12.py file we supply does have prints. I think this is an old version of test_simple.py and probably should be removed
  5. We advise to use asserts. a) should we state in text that you can pull asserts_equals from testlib? b) That is the only assert I see - do we need to provide (and document) others or is this enough?
  6. Should we be further recommend need for meaningful names for the files. test_simple.py is not as useful for searching as test_vehicle_parameters.py (and I think we should change this name too unless this is just considered a "demo" file - in which case perhaps "test_simple_demo" might be even better.)
  7. Should we also recommend that each file and method added be given brief documentatio? I think this a good idea.
  8. We might want to move the example of setting environment variables (or copy it) up to the integration test section, since here we also state that you need to set them.

@tcr3dr
Copy link
Contributor Author

tcr3dr commented Jul 23, 2015

@hamishwillee I address most of your concerns.

  • The branch is now rebased so you can test if unit tests work.
  • I added a warning to, but didn't remove, the Web API test section (for completeness)
  • The recommendations for test_simple and test_12 are good and I'll open an issue for those.

If this passes CI and you are happy with it, let me know!

@hamishwillee
Copy link
Contributor

@tcr3dr Really good fix/response! I created #233 (like #232) and added a couple of minor typo fixes to highlighting of assert_equals.

My only concern is that we refer to mock but there is no useful real-world example (e.g. test_api.py) that shows enough to know "correct use".

Consider the example:

import mock
from mock import MagicMock
import droneapi
from droneapi.module.api import APIModule
from nose.tools import assert_equals

def test_mode():
    api = APIModule(MagicMock())
    res = api.get_connection()
    assert_equals(len(res.get_vehicles()), 1)

From a user perspective trying to work out how to use Mock with dronekit I wonder what APIModule is (since it isn't public). Why not use the local_connection?

Reading the mock docs I might assume that here you are providing a "Mock" connection to use, and that guarantees that get_vehicles() returns 1, but I can't see where the setup of such an object occurs? ie it looks like black magic.

The use of Mock implies that these tests never run against a real device - correct?

Upshot, a little more explanation of mock or a better example which we link to as an example would help.

I would be happy for what we have here now to go out and this additional material to be provided in a separate issue if you preferred.

@tcr3dr
Copy link
Contributor Author

tcr3dr commented Jul 24, 2015

@hamishwillee Good points. I'll split that into a separate issue. Mostly because I'm not super familiar with Mock myself yet, though I've seen it used extensively by @eliao to good effect. As the tests improve, so shall the docs.

tcr3dr added a commit that referenced this pull request Jul 24, 2015
Adds CONTRIBUTING info for tests.
@tcr3dr tcr3dr merged commit 2943d82 into master Jul 24, 2015
@tcr3dr tcr3dr deleted the tcr-contrib branch July 24, 2015 20:08
@eliao
Copy link

eliao commented Jul 24, 2015

I'm relatively new to Mock myself, but it seems quite handy. It's very useful for isolating tests so that they test only exactly what they are meant to. If the tests include networking, etc, then they can fail/succeed due to things that are outside their control. Unit testing vs. functional testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants