SPEC 0 — Minimum Supported Dependencies

Description#

This SPEC recommends that all projects across the Scientific Python ecosystem adopt a common time-based policy for dropping dependencies. From the perspective of this SPEC, the dependencies in question are core packages as well as older Python versions.

All versions refer to feature releases (i.e., Python 3.8.0, NumPy 1.19.0; not Python 3.8.1, NumPy 1.19.2).

Specifically, we recommend that:

1. Support for Python versions be dropped 3 years after their initial release.
2. Support for core package dependencies be dropped 2 years after their initial release.

Core Project Endorsement#

Core project endorsing this SPEC means that those projects encourage all projects across the Scientific Python ecosystem to limit how long they support older Python versions and older dependency versions. A core project endorsing this SPEC does not imply that that project will provide bug-fix releases for two full years after a release.

Ecosystem Adoption#

Implementation#

Motivation#

Limiting the scope of supported dependencies is an effective way for packages to limit maintenance burden. Combinations of packages need to be tested, which impacts also on continuous integration times and infrastructure upkeep. Code itself also becomes more complicated when it has to be aware of various combinations of configurations.

Adoption of this SPEC will ensure a consistent support policy across packages, and reduce the need for individual projects to divise similar policies.

Ultimately, reduced maintenance burden frees up developer time, which translates into more features, bugfixes, and optimizations for users.

Background#

In the past, longer support cycles were common. There were several reasons for this, including the Python 2 / 3 transition, difficulties installing packages, and users needing to use old, operating-system provided versions of Python. The situation has since improved due to improved installations via binary wheels, virtual environments becoming commonplace, and support for Python 2 being dropped.

Support Window#

gantt
dateFormat YYYY-MM-DD
axisFormat %m / %Y
title Support Window

section python
3.9 : 2020-10-05,2023-10-05
3.10 : 2021-10-04,2024-10-03
3.11 : 2022-10-24,2025-10-23
3.12 : 2023-10-02,2026-10-01

section numpy
1.22.0 : 2021-12-31,2023-12-31
1.23.0 : 2022-06-22,2024-06-21
1.24.0 : 2022-12-18,2024-12-17
1.25.0 : 2023-06-17,2025-06-16
1.26.0 : 2023-09-16,2025-09-15

section scipy
1.8.0 : 2022-02-05,2024-02-05
1.9.0 : 2022-07-29,2024-07-28
1.10.0 : 2023-01-03,2025-01-02
1.11.0 : 2023-06-25,2025-06-24
1.12.0 : 2024-01-20,2026-01-19

section matplotlib
3.5.0 : 2021-11-16,2023-11-16
3.6.0 : 2022-09-16,2024-09-15
3.7.0 : 2023-02-13,2025-02-12
3.8.0 : 2023-09-15,2025-09-14

section pandas
1.4.0 : 2022-01-22,2024-01-22
1.5.0 : 2022-09-19,2024-09-18
2.0.0 : 2023-04-03,2025-04-02
2.1.0 : 2023-08-30,2025-08-29
2.2.0 : 2024-01-20,2026-01-19

section scikit-image
0.19.0 : 2021-12-03,2023-12-03
0.20.0 : 2023-02-28,2025-02-27
0.21.0 : 2023-06-02,2025-06-01
0.22.0 : 2023-10-03,2025-10-02

section networkx
2.7 : 2022-02-28,2024-02-28
2.8 : 2022-04-09,2024-04-08
3.0 : 2023-01-08,2025-01-07
3.1 : 2023-04-04,2025-04-03
3.2 : 2023-10-19,2025-10-18

section scikit-learn
1.0 : 2021-09-24,2023-09-24
1.1.0 : 2022-05-12,2024-05-11
1.2.0 : 2022-12-08,2024-12-07
1.3.0 : 2023-06-30,2025-06-29
1.4.0 : 2024-01-18,2026-01-17

section xarray
0.20.0 : 2021-11-02,2023-11-02
0.21.0 : 2022-01-28,2024-01-28
2022.3.0 : 2022-03-02,2024-03-01
2022.6.0 : 2022-07-22,2024-07-21
2022.9.0 : 2022-09-29,2024-09-28
2022.10.0 : 2022-10-13,2024-10-12
2022.11.0 : 2022-11-04,2024-11-03
2022.12.0 : 2022-12-02,2024-12-01
2023.1.0 : 2023-01-18,2025-01-17
2023.2.0 : 2023-02-07,2025-02-06
2023.3.0 : 2023-03-22,2025-03-21
2023.4.0 : 2023-04-14,2025-04-13
2023.5.0 : 2023-05-19,2025-05-18
2023.6.0 : 2023-06-23,2025-06-22
2023.7.0 : 2023-07-17,2025-07-16
2023.8.0 : 2023-08-20,2025-08-19
2023.9.0 : 2023-09-26,2025-09-25
2023.10.0 : 2023-10-19,2025-10-18
2023.11.0 : 2023-11-17,2025-11-16
2023.12.0 : 2023-12-08,2025-12-07
2024.1.0 : 2024-01-17,2026-01-16

section ipython
7.27.0 : 2021-08-27,2023-08-27
7.28.0 : 2021-09-25,2023-09-25
7.29.0 : 2021-10-30,2023-10-30
7.30.0 : 2021-11-26,2023-11-26
7.31.0 : 2022-01-05,2024-01-05
7.32.0 : 2022-02-25,2024-02-25
7.33.0 : 2022-04-29,2024-04-28
7.34.0 : 2022-05-28,2024-05-27
8.0.0 : 2022-01-12,2024-01-12
8.1.0 : 2022-02-25,2024-02-25
8.2.0 : 2022-03-27,2024-03-26
8.3.0 : 2022-04-29,2024-04-28
8.4.0 : 2022-05-28,2024-05-27
8.5.0 : 2022-09-06,2024-09-05
8.6.0 : 2022-10-30,2024-10-29
8.7.0 : 2022-11-28,2024-11-27
8.8.0 : 2023-01-03,2025-01-02
8.9.0 : 2023-01-27,2025-01-26
8.10.0 : 2023-02-10,2025-02-09
8.11.0 : 2023-02-28,2025-02-27
8.12.0 : 2023-03-30,2025-03-29
8.13.0 : 2023-04-28,2025-04-27
8.14.0 : 2023-06-02,2025-06-01
8.15.0 : 2023-09-01,2025-08-31
8.16.0 : 2023-09-29,2025-09-28
8.17.0 : 2023-10-30,2025-10-29
8.18.0 : 2023-11-24,2025-11-23
8.19.0 : 2023-12-22,2025-12-21
8.20.0 : 2024-01-08,2026-01-07
8.21.0 : 2024-01-31,2026-01-30

section zarr
2.9.0 : 2021-08-23,2023-08-23
2.10.0 : 2021-09-19,2023-09-19
2.11.0 : 2022-02-07,2024-02-07
2.12.0 : 2022-06-23,2024-06-22
2.13.0 : 2022-09-22,2024-09-21
2.14.0 : 2023-02-10,2025-02-09
2.15.0 : 2023-06-14,2025-06-13
2.16.0 : 2023-07-20,2025-07-19
2.17.0 : 2024-02-14,2026-02-13

Drop Schedule#

Below is an auto generated schedule with recommended dates for dropping support. We suggest that the next release in a given quarter is considered as the one removing support for a given item.

You may want to delay the removal of support of an older Python version until your package fully works on the newly released Python, thus keeping the number of supported minor versions of Python the same for your package.

2023 - Quarter 4:#

Recommend drop support for:#

2024 - Quarter 1:#

Recommend drop support for:#

2024 - Quarter 2:#

Recommend drop support for:#

2024 - Quarter 3:#

Recommend drop support for:#

2024 - Quarter 4:#

Recommend drop support for:#

2025 - Quarter 1:#

Recommend drop support for:#

2025 - Quarter 2:#

Recommend drop support for:#

2025 - Quarter 3:#

Recommend drop support for:#

2025 - Quarter 4:#

Recommend drop support for:#

2026 - Quarter 1:#

Recommend drop support for:#

2026 - Quarter 4:#

Recommend drop support for:#

Notes#

• This document builds on NEP 29, which describes several alternatives including ad hoc version support, all CPython supported versions, default version on Linux distribution, N minor versions of Python, and time window from the X.Y.1 Python release.

• Code to generate support and drop schedule tables:

import requests
import itertools
import collections
from datetime import datetime, timedelta

import pandas as pd
from packaging.version import Version


py_releases = {
    "3.8": "Oct 14, 2019",
    "3.9": "Oct 5, 2020",
    "3.10": "Oct 4, 2021",
    "3.11": "Oct 24, 2022",
    "3.12": "Oct 2, 2023",
}
core_packages = [
    # Path(x).stem for x in glob("../core-projects/*.md") if "_index" not in x
    "numpy",
    "scipy",
    "matplotlib",
    "pandas",
    "scikit-image",
    "networkx",
    "scikit-learn",
    "xarray",
    "ipython",
    "zarr",
]
plus36 = timedelta(days=int(365 * 3))
plus24 = timedelta(days=int(365 * 2))
delta6month = timedelta(days=int(365 // 2))

# Release data

now = datetime.now()
cutoff = now - delta6month


def get_release_dates(package, support_time=plus24):
    releases = {}

    print(f"Querying pypi.org for {package} versions...", end="", flush=True)
    response = requests.get(
        f"https://pypi.org/simple/{package}",
        headers={"Accept": "application/vnd.pypi.simple.v1+json"},
    ).json()
    print("OK")

    file_date = collections.defaultdict(list)
    for f in response["files"]:
        ver = f["filename"].split("-")[1]
        try:
            version = Version(ver)
        except:
            continue

        if version.is_prerelease or version.micro != 0:
            continue

        release_date = None
        for format in ["%Y-%m-%dT%H:%M:%S.%fZ", "%Y-%m-%dT%H:%M:%SZ"]:
            try:
                release_date = datetime.strptime(f["upload-time"], format)
            except:
                pass

        if not release_date:
            continue

        file_date[version].append(release_date)

    release_date = {v: min(file_date[v]) for v in file_date}

    for ver, release_date in sorted(release_date.items()):
        drop_date = release_date + support_time
        if drop_date >= cutoff:
            releases[ver] = {
                "release_date": release_date,
                "drop_date": drop_date,
            }

    return releases


package_releases = {
    "python": {
        version: {
            "release_date": datetime.strptime(release_date, "%b %d, %Y"),
            "drop_date": datetime.strptime(release_date, "%b %d, %Y") + plus36,
        }
        for version, release_date in py_releases.items()
    }
}

package_releases |= {package: get_release_dates(package) for package in core_packages}

# filter all items whose drop_date are in the past
package_releases = {
    package: {
        version: dates
        for version, dates in releases.items()
        if dates["drop_date"] > cutoff
    }
    for package, releases in package_releases.items()
}


# Save Gantt chart

print("Saving Mermaid chart to chart.md")
with open("chart.md", "w") as fh:
    fh.write(
        """gantt
dateFormat YYYY-MM-DD
axisFormat %m / %Y
title Support Window"""
    )

    for name, releases in package_releases.items():
        fh.write(f"\n\nsection {name}")
        for version, dates in releases.items():
            fh.write(
                f"\n{version} : {dates['release_date'].strftime('%Y-%m-%d')},{dates['drop_date'].strftime('%Y-%m-%d')}"
            )
    fh.write("\n")

# Print drop schedule

data = []
for k, versions in package_releases.items():
    for v, dates in versions.items():
        data.append(
            (
                k,
                v,
                pd.to_datetime(dates["release_date"]),
                pd.to_datetime(dates["drop_date"]),
            )
        )

df = pd.DataFrame(data, columns=["package", "version", "release", "drop"])

df["quarter"] = df["drop"].dt.to_period("Q")

dq = df.set_index(["quarter", "package"]).sort_index()


print("Saving drop schedule to schedule.md")


def pad_table(table):
    rows = [[el.strip() for el in row.split("|")] for row in table]
    col_widths = [max(map(len, column)) for column in zip(*rows)]
    rows[1] = [
        el if el != "----" else "-" * col_widths[i] for i, el in enumerate(rows[1])
    ]
    padded_table = []
    for row in rows:
        line = ""
        for entry, width in zip(row, col_widths):
            if not width:
                continue
            line += f"| {str.ljust(entry, width)} "
        line += f"|"
        padded_table.append(line)

    return padded_table


def make_table(sub):
    table = []
    table.append("|    |    |    |")
    table.append("|----|----|----|")
    for package in sorted(set(sub.index.get_level_values(0))):
        vers = sub.loc[[package]]["version"]
        minv, maxv = min(vers), max(vers)
        rels = sub.loc[[package]]["release"]
        rel_min, rel_max = min(rels), max(rels)
        version_range = str(minv) if minv == maxv else f"{minv} to {maxv}"
        rel_range = (
            str(rel_min.strftime("%b %Y"))
            if rel_min == rel_max
            else f"{rel_min.strftime('%b %Y')} and {rel_max.strftime('%b %Y')}"
        )
        table.append(f"|{package:<15}|{version_range:<19}|released {rel_range}|")

    return pad_table(table)


def make_quarter(quarter, dq):
    table = ["#### " + str(quarter).replace("Q", " - Quarter ") + ":\n"]
    table.append("###### Recommend drop support for:\n")
    sub = dq.loc[quarter]
    table.extend(make_table(sub))
    return "\n".join(table)


with open("schedule.md", "w") as fh:
    # we collect package 6 month in the past, and drop the first quarter
    # as we might have filtered some of the packages out depending on
    # when we ran the script.
    tb = []
    for quarter in list(sorted(set(dq.index.get_level_values(0))))[1:]:
        tb.append(make_quarter(quarter, dq))

    fh.write("\n\n".join(tb))
    fh.write("\n")