Skip to content

Test case name in JUnit XML not consistent when using scenario retries #4983

Open
@nikzupancic

Description

@nikzupancic

When using Codecept with mocha-junit-reporter I noticed that testcase name is not always consistent and changes depending on whether retry is used on scenario or not.

For a test without any retries, its testcase always follows the pattern: <suite name>: <scenario name>. Including trimmed down XML for easier reading:

  <testsuite name="JUnit reporting" timestamp="2025-04-30T12:34:22" tests="1" file="/path/to/test.js" time="10.966" failures="1">
    <testcase name="JUnit reporting: Test 1" time="5.060" classname="Test 1">

If I use either Feature.retry(1) or Scenario.retry(1) the name of testcase omits the colon : character between suite and scenario name.

  <testsuite name="JUnit reporting" timestamp="2025-04-30T12:34:22" tests="1" file="/path/to/test.js" time="10.966" failures="1">
    <testcase name="JUnit reporting Test 1" time="5.060" classname="Test 1">

If the test uses retries but passes on 1st try the colon is still there:

  <testsuite name="JUnit reporting" timestamp="2025-04-30T12:34:22" tests="1" file="/path/to/test.js" time="10.966" failures="0">
    <testcase name="JUnit reporting: Test 1" time="5.060" classname="Test 1">

The full title seems to get modified somewhere during the retry but I haven't been able to identify where. It's missing no matter if the test passes or fails on the retry. I know it looks like a small difference but it's an inconvenience if you want to do any kind of automatic processing on output XML files.

package.json dependencies:

  "dependencies": {
    "codeceptjs": "^3.7.3",
    "mocha-junit-reporter": "^2.2.1",
    "playwright": "^1.52.0"
  },

This is probably not affected by the type of helper used but I reproduced it using both Playwright and Webdriver.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions