Detailed xfail output sometimes distracts from unexpected failures
envgap__gitpython-developers__GitPython-1891
01 / FAILURE SIGNATURE
As reported upstream
No identifying execution failure has been captured.
Not a benchmark task.
- The project already builds and runs before the fix, so there is nothing to repair.
02 / ENVIRONMENT RECIPE
- Base commit
6e61b1d3bdcd2d9b81bac4eb4d17c18b270e5787- Manifest
pyproject.toml- Reproduce
Awaiting issue-specific recipe- Run under trace
Awaiting a meaningful runtime command
03 / ORIGINAL ISSUE TEXT
gitpython-developers/GitPython #1891 · read the original issue
This issue is to track the problem noticed in #1853 and #1862 (see comments in both) where the specific details of new test failures are made less obvious in reading `pytest` output, in CI checks, due to the otherwise useful pytest 8 feature to show more such details. From https://github.com/gitpython-developers/GitPython/pull/1853#issuecomment-1974005684: > `pytest` output [now contains details](https://github.com/pytest-dev/pytest/issues/11233) of tests with xfail (expected failure) status, which are unrelated, and that may have distracted from the related test's failure. This [changed in major version 8 of `pytest`](https://docs.pytest.org/en/8.0.x/changelog.html#pytest-8-0-0rc2-2024-01-17) (which GitPython is using automatically except on Python 3.7 which doesn't support it). Overall, for most projects, it seems to me that this is an improvement, since it's useful to be able to see the specific details of xfailing tests without extra effort (and check if they really match up with the messages written in the xfail decorators). However, it may be that for GitPython's tests it makes common tasks more cumbersome. If so, then the `pytest` configuration can be changed (this does not require downgrading `pytest`). I'm unsure what the best solution is, but if necessary the `-ra` in https://github.com/gitpython-developers/GitPython/blob/53640535cf8314366a01da081947dd8504a299cd/pyproject.toml#L6 can be changed to something that does not imply `x` (xfail). The "full list of available characters" in the [Producing a detailed summary report](https://docs.pytest.org/en/8.0.x/how-to/output.html#producing-a-detailed-summary-report) section of the documentation documents how to specify which test statuses to emit detailed output for. I'm unsure of what should be done and I don't know of an easy way to eliminate the distracting effects while still having the benefit of seeing specific output of xfailing tests.
04 / LABELS
Labels from the report text only; not yet run
No supported category has been assigned.
Label rules and the text that matched
[]