Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

If possible, I would like @astrofrog and/or @Cadair to have a look because they set this up and they would know best if this refactoring still reflects their original design, as this isn't a one-to-one replacement.

I think this is fine as long as we accept that bumping oldestdeps will probably mean a full figure regeneration. Also if we have different oldestdeps on different branches things might break (I'm not 100% clear on how we handle figure tests on different branches at the moment, I get confused with the slight differences in setup between sunpy and astropy).

I think on a more philosophical level, this will stop you being able to do visual figure comparisons of historic commits (i.e. regenerating the figures for oldestdeps will mean there isn't old versions of the reference images like there is now). I'm not personally that worried about this, but thought it was worth mentioning. (We could solve this other ways if we wanted, like making the figure names use the installed version of mpl rather than having it hard-coded).

this will stop you being able to do visual figure comparisons of historic commits

On the contrary, this should be making env resolution for old commits reliably reproducible, so I'm not sure I get your point ?

I think we are maybe not talking about the same thing. Each figure test env uploads figures (in a folder named for the tox env) here: https://github.com/astropy/astropy-figure-tests/tree/astropy-main/figures at the moment the old envs stay around even after we stop updating them, but if they are all named oldestdeps then it would overwrite the figures when they are regenerated with a newer mpl version.

oh, I get it now. Isn't this repo used as a submodule here ? if so, it should trivially be pinned to a git commit hash, which would make this problem moot. Alternatively, we could also add git tags there ?

I don't think it is a Git submodule and I do not wish to reintroduce that after we managed to get rid of astropy-helpers

I think this is fine as long as we accept that bumping oldestdeps will probably mean a full figure regeneration.

This does seem okay to me too. Overall I don't see anything blocking here, so can we move forward with it ?

full figure regeneration

Let me try to understand, so we will need a lot more PRs like this?

• TST: Update mpldev hashes #19195

(rebased to resolve a merge conflict)

friendly reminder @Cadair

I have a concern related to what @Cadair raised above. Let's say we release astropy 8.0 which has a certain set of versions for oldestdeps. Now let's suppose that in main we bump the minimum version of matplotlib. At this point the version of matplotlib used for oldestdeps will have diverged between the release branch and main, which will cause issues when running tests on the release branch.

The only way I can think of to get around this if we want to avoid having to rename CI jobs is that when we construct the reference image URL, we inject the matplotlib version into it (like we used to).

I couldn't find how it used to be done in the repo history. Any pointers ?

Something like this:

https://github.com/astropy/astropy/blob/v3.0.x/astropy/tests/image_tests.py

Just wondering, is this really worth it though? We only bump the minimum MPL version once or twice a year, and not sure if it's worth over-engineering this?

We only bump the minimum MPL version once or twice a year

right, but this PR isn't really about removing this (admittedly marginal) maintenance burden: the goal is rather to eliminate a failure mode, which the current state of tox.ini is a testimony to (we have a redundant constraint on pyparsing). Currently we have no control over the frequency of this failure, and this PR would set an upper bound to the frequency at which we actively bump our minimal requirement for matplotlib.

and not sure if it's worth over-engineering this?

One could argue the existing setup is over-engineered already. I'd like to give a shot at figuring out a simpler solution that eliminates the defect you and Stuart pointed out.

I think we could probably retain the existing env names and remove metadata duplication by using recent tox features like extend=true to define mpl380 (https://tox.wiki/en/stable/config.html#configuration-reference)... will dig in some more.

I think I'll need to migrate tox.ini to tox.toml to benefit from these newest features, but this is temporarily blocked by scientific-python/cookie#739

apparently tox configuration is in a weird state right now: tox.toml is the current recommended configuration file, but does not support all features from tox.ini, including conditional env factors, so I think we cannot actually migrate yet. I'm now hoping I was wrong earlier when I concluded we needed to use this format to leverage new features, otherwise it would mean that tox.toml's feature sets is not even a subset of tox.ini's.

Apparently this "design" was actually intentional:
https://github.com/tox-dev/tox/pull/3353/changes#r1784021175

I'm at a loss for words.

tox.toml now officially supports everything that tox.ini did (though migration might not be trivial), and will be the only target for new features. I'll open a separate issue to track this.