PHPLIB-616 Continuous Matrix Testing for 4.2 era drivers #812
Conversation
What is this referring to? Is the PR incomplete as-is? |
The PR works, but it does not fix all failing tests (which is not bad per se, but makes it harder to spot new failures if a server patch release changes something). The failing tests are hard to skip, as they are single tests from a suite of spec tests where we have no way of applying groups selectively. I have yet to try and figure out if I can get PHPUnit to apply groups to single tests from a data provider, but for the time being we can either leave failing tests in place, or skip the entire failing test suite. |
alcaeus
force-pushed
the
v1.5-matrix-testing
branch
from
9cc586b to
e130249
Compare
|
@jmikola upon further investigation, there's no way (currently) to add a group to a single data provider test, so I've skipped the entire retryable reads spec tests for matrix testing. I'll revisit this separately. I've also backported a fix to the spec test runner that avoids dropping the admin database, as that caused additional failures. Tests now pass: https://spruce.mongodb.com/version/603d13a8d1fe075a9b7cf65c/tasks |
| * @group matrix-testing-server-5.0-driver-4.0 | ||
| * @group matrix-testing-server-5.0-driver-4.2 |
There was a problem hiding this comment.
I realize this format dates back to https://github.com/mongodb/mongo-php-library/pull/810/files#r579294457 (already merged in v1.4), but seeing this again I wonder if it makes sense to add something to this name that communicates the tests are actually skipped for this server/driver combination.
My first impression reading this after not thinking about it for some time is that these are tests for this particular version, not incompatible tests intended to be skipped.
Maybe just prefix "exclude-" to these?
There was a problem hiding this comment.
Adding a prefix sounds good. I'll make the change in 1.4 and merge up, fixing things along the way.
* v1.5: Rename matrix testing exclusion groups PHPLIB-616 Continuous Matrix Testing for 4.2 era drivers (#812) PHPLIB-617 Continuous Matrix Testing for 4.0 era drivers (#810)
* v1.6: Rename matrix testing exclusion groups PHPLIB-616 Continuous Matrix Testing for 4.2 era drivers (#812) PHPLIB-617 Continuous Matrix Testing for 4.0 era drivers (#810)
* v1.7: PHPLIB-615 Matrix testing for 4.4 era drivers (#813) Rename matrix testing exclusion groups PHPLIB-616 Continuous Matrix Testing for 4.2 era drivers (#812) PHPLIB-617 Continuous Matrix Testing for 4.0 era drivers (#810)
* v1.8: Fix test failures due to wrong merge PHPLIB-615 Matrix testing for 4.4 era drivers (#813) Rename matrix testing exclusion groups PHPLIB-616 Continuous Matrix Testing for 4.2 era drivers (#812) PHPLIB-617 Continuous Matrix Testing for 4.0 era drivers (#810)
PHPLIB-616
This branch skips tests that fail when running the 4.2 driver against newer servers. The patch build has been updated to test 4.2 as well: https://spruce.mongodb.com/version/602f72fc9ccd4e172a60e6e8/tasks
The failing tests are tricky: these are single tests in some of the spec tests which we can't skip due to how data providers work. Skipping the entire test suite seems overkill, so I'll have to figure out another way to fix this.