Skip to content

Commit 4722d32

Browse files
committed
Results always docs
Signed-off-by: Conor MacBride <[email protected]>
1 parent 743701a commit 4722d32

File tree

1 file changed

+25
-7
lines changed

1 file changed

+25
-7
lines changed

README.rst

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -86,19 +86,19 @@ The hash library can be generated with
8686
``--mpl-generate-hash-library=path_to_file.json``. The hash library to be used
8787
can either be specified via the ``--mpl-hash-library=`` command line argument,
8888
or via the ``hash_library=`` keyword argument to the
89-
``@pytest.mark.mpl_image_comapre`` decorator.
89+
``@pytest.mark.mpl_image_compare`` decorator.
9090

9191

9292
Hybrid Mode: Hashes and Images
9393
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
9494

9595
It is possible to configure both hashes and baseline images. In this scenario
96-
the hashes will be compared first, and the baseline images used if the hash
96+
the hashes will be compared first, with the baseline images only used if the hash
9797
comparison fails.
9898

9999
This is especially useful if the baseline images are external to the repository
100-
with the tests in, and can be accessed remotely. In this situation if the hashes
101-
match the baseline images wont be retrieved, saving time and bandwidth. Also it
100+
containing the tests, and are accessed via HTTP. In this situation, if the hashes
101+
match, the baseline images won't be retrieved, saving time and bandwidth. Also, it
102102
allows the tests to be modified and the hashes updated to reflect the changes
103103
without having to modify the external images.
104104

@@ -113,16 +113,16 @@ against, the tests can be run with::
113113

114114
and the tests will pass if the images are the same. If you omit the
115115
``--mpl`` option, the tests will run but will only check that the code
116-
runs without checking the output images.
116+
runs, without checking the output images.
117117

118118

119119
Generating a Failure Summary
120120
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
121121

122-
By specifying the ``--mpl-generate-summary=html`` CLI argument a HTML summary
122+
By specifying the ``--mpl-generate-summary=html`` CLI argument, a HTML summary
123123
page will be generated showing the baseline, diff and result image for each
124124
failing test. If no baseline images are configured, just the result images will
125-
be displayed.
125+
be displayed. (See also, the **Results always** section below.)
126126

127127
Options
128128
-------
@@ -184,6 +184,24 @@ are run. In addition, if both this option and the ``baseline_dir``
184184
option in the ``mpl_image_compare`` decorator are used, the one in the
185185
decorator takes precedence.
186186

187+
Results always
188+
^^^^^^^^^^^^^^
189+
190+
By default, result images are only generated for tests that fail.
191+
Passing ``--mpl-results-always`` to pytest will force result images
192+
to be generated for all tests, even for tests that pass.
193+
If a baseline image exists, a diff image will also be generated.
194+
All of these images will be shown in the summary (if requested).
195+
196+
This option is useful for always *comparing* the result images again
197+
the baseline images, while only *assessing* the tests against the
198+
hash library.
199+
If you only update your baseline images after merging a PR, this
200+
option means that the generated summary will always show how the
201+
PR affects the baseline images, with the success status of each
202+
test (based on the hash library) also shown in the generated
203+
summary.
204+
187205
Base style
188206
^^^^^^^^^^
189207

0 commit comments

Comments
 (0)