BLD: Drop Python 3.10 support

[wirable23]

Research

• I have searched the [pandas] tag on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

NA

Question about pandas

Per spec0, python 3.10 support should be dropped. Will pandas 3 still support it?

[rhshadrach]

I believe we follow NumPy who has adopted Spec0, which means we should drop support with pandas 3.0 (or even 2.3?).

cc @jorisvandenbossche @WillAyd @mroeschke

[WillAyd]

I think 2.3 will continue to support 3.10 but yea probably worth dropping by the time 3.0 gets out

[jorisvandenbossche]

I would also certainly keep support for 2.3 (there is not that much reason to drop it now and backport those changes IMO)

[simonjayhawkins]

The motivation for SPEC 0 - Minimum Supported Dependencies is that limiting the scope of supported dependencies helps reduce the maintenance burden, allowing developers to focus on adding features, fixing bugs, and optimizing performance.

SPEC 0 states that support for Python versions should be dropped 3 years after their initial release. Python 3.10 was released in October 2021 and so, according to the spec we now follow, should be dropped for the forthcoming 2.3 release.

We don't drop support for Python in patch releases and so if not dropped for 2.3, we would need to ensure support for all subsequent 2.3.x releases. If we follow previous release schedules this could be around 6 months.

I would also certainly keep support for 2.3 (there is not that much reason to drop it now and backport those changes IMO)

there is also not that much reason not to drop it?

[jorisvandenbossche]

there is also not that much reason not to drop it?

For a maintenance branch like 2.3, it's more work to drop it than to not drop it ..

Also, SPEC0 does not require you to drop a version immediately when allowed according to their schedule, only that it is fine to drop it from that point

[WillAyd]

I would add that maintaining 2.3 has already been rather challenging. The bar to put any changes into that which are not related to the string data type should be really high, and I don't think dropping 3.10 would be worth it

[simonjayhawkins]

I must be underestimating the amount of work to drop support. I wasn't expecting the codebase on 2.3.x to be updated to take advantage of python 3.11 features just changing requires-python in pyproject.toml, removing some ci jobs and updating the release notes?

[WillAyd]

It's more risk of allowing 3.11+ features into main and then having to come up with an alternative during backport

[jorisvandenbossche]

@WillAyd that risk is more related to dropping 3.10 already now in main, not so much for dropping 3.10 in 2.3.x branch (and not only for having to come up with an alternative, but also just to have more conflicts. So for me that is a reason to wait a bit longer with dropping Python 3.10 for main, although we can certainly do that for 3.0).

Anyway, it's indeed not much work to drop it on 2.3.x (we probably already spent more time discussing this than it takes to actually do it ..), but it is still work with basically zero benefit to us (it's not that we are planning to use features on the 2.3.x branch, so then why drop it?), just for the reason to blindly follow a strict interpretation of SPEC0.
I would propose to spent our time on more useful things for the 2.3.x branch, and if at some point there is actually an issue with Python 3.10 on that branch, we can still drop it then.

[simonjayhawkins]

and if at some point there is actually an issue with Python 3.10 on that branch, we can still drop it then.

In our version policy we updated the Python support section to say pandas mirrors the NumPy guidelines for Python support. which itself states
Minimum Python and NumPy version support should be adjusted upward on every major and minor release, but never on a patch release. This is our current published policy as IIUC the adoption of SPEC 0 - Minimum Supported Dependencies has not yet been communicated to users.

So I don't think we could drop support for Python 3.10 in a patch release.

AFAICT there is not this restriction in SPEC 0.

It's more risk of allowing 3.11+ features into main and then having to come up with an alternative during backport

this will happen soon anyway when we drop 3.10 on main. I don't get that we would need a alternative. We would need an alternative if we don't drop 3.10 on 2.3.x?

[WillAyd]

What is the urgency to dropping 3.10 on main? I was under the impression we would just do it closer to the 3.x release, towards the end of the 2.3 lifecycle

[simonjayhawkins]

It was mentioned that we shouldn't blindly follow the spec but that spec has been created with recommendations that all projects across the Scientific Python ecosystem adopt it.

IIUC pandas 3.0.0rc could be released very shortly after 2.3.0. If this happens then I assume we will not be following our normal release lifecycle and will be actively maintaining 2 released versions simultaneously.

[simonjayhawkins]

of course, as users assume we are still following NEP29 until we communicate the changes (currently in the 3.0.0 release notes), we could drop Python 3.10 in 3.1 instead of 3.0 and this avoids the potential overlap with the 2.3.x branch.

[jack-mcivor]

FWIW numpy==2.3.0rc1 is out now and drops support for 3.10 numpy/numpy#27861 https://github.com/numpy/numpy/releases/tag/v2.3.0rc1

[simonjayhawkins]

in #60059 (comment) I wrote

We don't drop support for Python in patch releases and so if not dropped for 2.3, we would need to ensure support for all subsequent 2.3.x releases. If we follow previous release schedules this could be around 6 months.

6 months on from this comment and with 2.3 not yet released we should ensure that dropping Python 3.10 support is now done before the 2.3 release which is scheduled for the end of this month? #59664 (comment)

[mroeschke]

I think it's OK for 2.3 to keep support for Python 3.10. For pandas 3.0, I would support dropping Python 3.10