Is codecov integration beneficial?

[chillenzer]

Is codecov integration a worthwhile aspect to teach (particularly in comparison to other 15 issues that will probably blow out the lesson content already)? What are the significant benefits over just having pytest-cov run in CI and maybe triggering a failure for coverage below X%? I would suggest to either remove it in favor of some of the other large-ish topics in the issue list or at least move it to the end in the potentially-undelivered part of the lesson material.

[edbennett]

I think having a CI failure from low test coverage would go against the spirit of this lesson (which largely tries to say "don't worry for now if you don't get close to 100% coverage; anything is better than nothing").

The reason the coverage section was added to the lesson was specifically because the repo we were going to use in the synoptic exercise used codecov. Since learners are likely to interact with repositories that use such tools, I think there is value in them seeing how to do so, although ideally we should revisit the instructions so we don't stall for 30 minutes trying to get it working when teaching it.

[chillenzer]

That specific approach to code coverage is probably an ideological discussion (see #28). Don't want to focus on that.

But that sounds like two weak reasons. We have lots of things in the material that "learners are likely to interact with" but only get a note box instead of a whole section. The only reason for going into that much detail and creating and account live and all of that should be to expect them to use it afterwards. For any existing project, it is enough to know that this nice badge comes from a convenient little website.

So, I'd argue that either there is a compelling reason (beyond "yeah, you see that a lot") or we can spend the time on expanding something else.

[edbennett]

I suspect I'm making weak arguments as I don't particularly use codecov myself, and am instead reasoning motivated by a sunk-cost fallacy.

@colinsauze Do you have a more robust defence of codecov?

[colinsauze]

I don't have any specific reason to use codecov other than because it provided us with a nice looking coverage report. I think there might have been some additional advantages to it when we first started teaching this material in the pre-github actions days.

But there should definitely be a use of some kind of coverage checking tool and I'd prefer something which could produce a nice annotated report highlighting where in the code coverage is lacking.