Use coverage from previous commit(s)

Hello!

We use semantic-release to publish new releases of our software, but the way Codacy handles coverage does not play nice with it.

After each commit that should trigger a release (fix, feat, etc.), a workflow runs, creating a new commit, which increments version numbers and updates CHANGELOG.md, and then publishes a release on GitHub. The commit created in this process has [skip ci] in the message, so this commit will not trigger any GitHub Actions builds, and thus not have a coverage report associated with it.

Here’s an example where you can see what our repositories’ commits look like:

I’ve already added a workaround to this repository for coverage disappearing after committing something that should not trigger tests by caching coverage files, skipping tests and then uploading the old coverage again. This does not work for the [skip ci] commits, though, as the workflow does not run at all. I really don’t want to remove the [skip ci] tag from those commits, because this introduces another layer of workarounds to get this working, and the preferred behaviour is to NOT run workflows at all when they’re not necessary. Other coverage CI tools I’ve used do not have this problem.

We’d like to be able to use one single service for both static analysis and coverage, but until this works properly we can’t have the “Codacy Coverage Report” check block pull requests, and we’re still forced to use another service to manage coverage if we want to keep using Codacy.

PR example: feat: add shipments by EdieLemoine · Pull Request #24 · myparcelnl/pdk · GitHub

At the time of writing, we have a commit in main that has been analysed, so the Codacy Coverage Report has passed.

As soon as a new release commit is added to main and the PR is rebased, this check will be stuck in “pending” forever, and the Codacy dashboard will say no coverage has been uploaded, while the coverage rate in the “Quality evolution” block does still say 100%.

I really hope this can be resolved.

Thanks in advance!

Edie

Developer at MyParcel

2 Likes