Skip to content

Improve coverage reports #1548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 19, 2022
Merged

Conversation

sevanspowell
Copy link
Contributor

@sevanspowell sevanspowell commented Jul 15, 2022

After a request from the ouroboros-network team to add coverage reports to their repository, I realized the coverage reports provided by haskell.nix have a few deficiencies:

  • If a package has no coverage of any of its modules, no coverage report is generated - leading to broken links in the project coverage report index page.
  • There is no link to the unioned project coverage report on the project coverage report index page.
  • HPC won't generate any coverage output if a module has 0% coverage.
  • If a test-suite depends on a module via "other-modules", the module is addressed differently in the mix file than usual. This caused the coverage report to fail to find the module.
    • Fixed this in e01f641 by expanding the search directories.
  • Refactored the interface for generating coverage reports:
    • Remove the concept of "package boundaries" from the "coverageReport" function.
    • The "coverageReport" is now a function of:
      • arbitrary checks generating tix files
      • arbitrary mix modules
    • This more closely reflects the model of hpc, which doesn't care about package boundaries.
    • Use this new "coverageReport" function to simplify the "projectCoverageReport" implementation.

- When covering a project with multiple packages, the project coverage report
  will produce a table of individual coverage reports for each package. The
  links in this table don't go anywhere when a package is uncovered (they are
  broken links).
  - By producing an empty coverage report when a package has no coverage, we fix
    the broken links, and provide a tad more information about the coverage
    state of that package.
- Provide a link to the union/all coverage report.
- Format the reports as a list instead of a table of one column.
- Provide better explanation of what each report means.
- Modules that have no coverage at all are simply not included in the HTML
  reports generated by HPC.
- Add a warning to the project coverage report index page so users are aware of
  this limitation of HPC.
@hamishmack
Copy link
Collaborator

bors try

iohk-bors bot added a commit that referenced this pull request Jul 15, 2022
@sevanspowell sevanspowell marked this pull request as draft July 15, 2022 09:52
@sevanspowell
Copy link
Contributor Author

Converting to draft while I figure out a few niggling issues I discovered - sorry @hamishmack !

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 15, 2022

try

Build failed:

@michaelpj
Copy link
Collaborator

Nice!

@sevanspowell sevanspowell force-pushed the sevanspowell/improve-coverage-reports branch 2 times, most recently from 2c0f423 to 521c657 Compare July 19, 2022 04:32
@sevanspowell sevanspowell marked this pull request as ready for review July 19, 2022 06:36
@sevanspowell
Copy link
Contributor Author

bors try

iohk-bors bot added a commit that referenced this pull request Jul 19, 2022
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 19, 2022

- Remove the concept of "package boundaries" from the "coverageReport" function.
- The "coverageReport" is now a function of:
  - arbitrary checks generating tix files
  - arbitrary mix modules
- This more closely reflects the usage of hpc, which doesn't care about package
  boundaries.
- Use this new "coverageReport" function to simplify the "projectCoverageReport"
  implementation. A project coverage report now simply:
  - copies out constituent coverage reports.
  - writes out an "all coverage report" using all checks in the project and all
    mix modules..
  - writes out a summary index page.
@sevanspowell sevanspowell force-pushed the sevanspowell/improve-coverage-reports branch from 521c657 to e01f641 Compare July 19, 2022 07:00
@sevanspowell
Copy link
Contributor Author

bors try

iohk-bors bot added a commit that referenced this pull request Jul 19, 2022
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Jul 19, 2022

@hamishmack hamishmack merged commit 0dca71e into master Jul 19, 2022
@iohk-bors iohk-bors bot deleted the sevanspowell/improve-coverage-reports branch July 19, 2022 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants