Scientific Python

News

Wed, 08 Oct 2025 00:00:00 +0000

2025–10–08: Fourth Developer Summit in Copenhagen – 5-7 November, 2025#

See summit website.

2025–03–28: Third Developer Summit was held in Seattle – 11-14 May 2025#

See summit website.

2024–04–11: Second Developer Summit was held in Seattle – June 2-5, 2024#

See summit website.

2023–05–30: First Developer Summit was held in Seattle#

See summit website.

2021–04–21: SPECs announced at SciPy 2021#

Stéfan van der Walt announces the Scientific Python Ecosystem Coordination project and talks about the SPEC documents during SciPy 2021.

SPEC 8 — Securing the Release Process

Tue, 04 Jun 2024 00:00:00 +0000

Description#

Open source libraries constitute a significant portion of the world’s digital infrastructure. Securing the Open Source supply chain (OSSC) is therefore an increasing concern, with examples of sophisticated attacks against the ecosystem (e.g., the 2024 xz utils backdoor) and malware attacks on PyPI highlighting the need for supply chain security to be taken seriously. The Python Software Foundation (PSF) is also taking the importance of the OSSC seriously, as demonstrated by the creation of the PSF Security Developer in Residence position in 2023.

SPEC 9 — Governance

Tue, 04 Jun 2024 00:00:00 +0000

Description#

This SPEC describes what governance is, and offers recommendations for choosing a governance model for open-source projects.

Open-source project communities are amalgamations of software and the humans who use and maintain it. Governance models define methods for decision-making within a project, roles for the community members imbued with decision-making authority, and processes for filling (and, if needed, vacating) those leadership roles. In the context of open-source software projects, governance models often also define the granting of necessary permissions or access to organization secrets necessary for the implementation of project-related decisions. Governance models are complementary to Codes of Conduct and Community Guidelines, which define the expectations and responsibilities of all participants in a project (including users of the software), regardless of their status as a project leader (cf. SPEC XXX), ensuring the long-term sustainability of a project.

SPEC 7 — Seeding Pseudo-Random Number Generation

Wed, 19 Apr 2023 00:00:00 +0000

Description#

Currently, libraries across the ecosystem provide various APIs for seeding pseudo-random number generation. This SPEC suggests a unified, pragmatic API, taking into account technical and historical factors. Adopting such a uniform API will simplify the user experience, especially for those who rely on multiple projects.

We recommend:

standardizing the usage and interpretation of an rng keyword for seeding, and
avoiding the use of global state and legacy bitstream generators.

We suggest implementing these principles by:

SPEC 6 — Keys to the Castle

Mon, 19 Dec 2022 00:00:00 +0000

Description#

Projects engage with restricted resources all the time. Examples include access to add & remove team members, grant commit rights, or make uploads to certain hosts.

Documenting resources is critical to ensuring uninterrupted operations. It is important that team members know who have access to resources, and how to gain access to resources. For example, if updated project documentation needs to be uploaded to a remote server via SSH, who are the team members that can do that, and by which process can a release manager request access?

SPEC 5 — CI Best Practices

Thu, 22 Sep 2022 00:00:00 +0000

Description#

Core Project Endorsement#

Ecosystem Adoption#

Badges#

Projects can highlight their adoption of this SPEC by including a SPEC badge.

[![SPEC 5 — CI Best Practices](https://img.shields.io/badge/SPEC-5-green?labelColor=%23004811&color=%235CA038)](https://scientific-python.org/specs/spec-0005/)

|SPEC 5 — CI Best Practices| 

.. |SPEC 5 — CI Best Practices| image:: https://img.shields.io/badge/SPEC-5-green?labelColor=%23004811&color=%235CA038
   :target: https://scientific-python.org/specs/spec-0005/

To indicate adoption of multiple SPECS with one badge, see this.

SPEC 4 — Using and Creating Nightly Wheels

Mon, 05 Sep 2022 00:00:00 +0000

Description#

This SPEC describes how to test against nightly wheels of several widely used projects and how to create nightly wheels for your project. The document uses the word nightly to refer to some semi regular interval, like daily, weekly, or every three days.

Regularly running your project’s tests while using the nightly version of your dependencies allows you to spot problems caused by upstream changes before a new release is made. This way potential issues can be resolved before they find their way into a release, at which point it becomes much harder to change or revert something.

SPEC 3 — Accessibility

Mon, 14 Feb 2022 00:00:00 +0000

Description#

Technology is a great tool to make the world accessible to the full range of human experience, which includes those with disabilities. The reach of accessibility guidelines extends beyond the scientific Python ecosystem including the Web Content Accessibility Guidelines (W3C), a comprehensive set of international standards designed to make web content more accessible.

The primary objective of this SPEC (Scientific Python Accessibility) is to provide fundamental recommendations for the Scientific Python communities and their projects. These recommendations aim to ensure accessibility and inclusivity for individuals with disabilities, particularly regarding web-based content and tools.

SPEC 2 — API Dispatch

Thu, 16 Dec 2021 00:00:00 +0000

Description#

This SPEC (Scientific Python Ecosystem Coordination) recommendation for API dispatching is designed to enhance interoperability and performance within the scientific Python ecosystem, leveraging the capabilities of Python entry_points.

This recommendation presents a systematic approach that enables users to redirect function calls to alternative computation backends seamlessly. This flexibility allows users to take advantage of optimized implementations simply by configuring an environment variable or adding an additional keyword argument (more on this later), rather than having to learn a new API’s interface, which might not be very “python-user”-friendly. Such adaptability facilitates the integration of hardware-specific optimizations, reimplementations in other programming languages (such as C or Rust), and the utilization of entirely new data structures.

SPEC 0 — Minimum Supported Dependencies

Thu, 17 Dec 2020 00:00:00 +0000

Description#

This SPEC recommends that all projects across the Scientific Python ecosystem adopt a common time-based policy for dropping dependencies. From the perspective of this SPEC, the dependencies in question are core packages as well as older Python versions.

All versions refer to feature releases (i.e., Python 3.8.0, NumPy 1.19.0; not Python 3.8.1, NumPy 1.19.2).

Specifically, we recommend that:

Support for Python versions be dropped 3 years after their initial release.
Support for core package dependencies be dropped 2 years after their initial release.

Note

Core packages may or may not decide to provide bug fix releases during the full 2 year period after release. Therefore, projects may occasionally want to drop support for core package dependencies earlier than recommended by this SPEC. For instance, if a newer minimum version of a core package is needed by a project due to a critical bug fix, which is not backported to older versions.

SPEC 1 — Lazy Loading of Submodules and Functions

Thu, 17 Dec 2020 00:00:00 +0000

Description#

This SPEC recommends a lazy loading mechanism—targeted at libraries—that avoids import slowdowns and provides explicit submodule exports.

For example, it allows the following behavior:

import skimage as ski  # cheap operation; does not load submodules

ski.filters  # cheap operation; loads the filters submodule, but not
             # any of its submodules or functions

ski.filters.gaussian(...)  # loads the file in which gaussian is implemented
                           # and calls that function

This has several advantages:

Mon, 01 Jan 0001 00:00:00 +0000

SPEC#

The Scientific Python Ecosystem Coordination (SPEC) mechanism is used to recommend project policies, coding conventions, and standard tooling.

See https://scientific-python.org/specs/ for a list of SPECs and their statuses.

Astropy

Mon, 01 Jan 0001 00:00:00 +0000

Code of Conduct

Mon, 01 Jan 0001 00:00:00 +0000

This code of conduct applies to all spaces managed by the scientific Python ecosystem project, including all public and private mailing lists, issue trackers, wikis, and any other communication channel used by our community.

This code of conduct should be honored by everyone who participates in the scientific Python ecosystem community formally or informally, or claims any affiliation with the project, in any project-related activities and especially when representing the project, in any role.

General Planning Meeting

Mon, 01 Jan 0001 00:00:00 +0000

Information#

Date: Monday, February 27th 9AM - 10AM Pacific time (click for your timezone)
Zoom Link: https://caltech.zoom.us/j/87686129450

Participants#

Jarrod Millman
Stéfan van der Walt
Brigitta Sipőcz
Juanita Gomez

Agenda#

The summit is scheduled for 1 hour and will consist of a series of high-level birds-of-a-feather (BoF)-style talks, followed by more focused discussion.

(5 min) Welcome & Introductions (Jarrod Millman)
(10 min) Logistics
(30) Meeting Topics

Governance and Decision Making

Mon, 01 Jan 0001 00:00:00 +0000

Abstract#

Scientific Python is a consensus-based community project. Anyone with an interest in the project can join the community, contribute to the project design, and participate in the decision making process. This document describes how that participation takes place, how to find consensus, and how deadlocks are resolved.

Note that this document only applies to the Scientific Python project, which aims to better coordinate the ecosystem and prepare scientific Python for the next decade of data science. However, every project in the ecosystem has its own governance structure.

IPython

Mon, 01 Jan 0001 00:00:00 +0000

Matplotlib

Mon, 01 Jan 0001 00:00:00 +0000

Meeting 1

Mon, 01 Jan 0001 00:00:00 +0000

Information#

When: September 6th 8am PDT
Where:
- https://berkeley.zoom.us/j/93150321184?pwd=aEp4RGFhc1pLQ2l3b1VKNHJRUGpOUT09
- Passcode: 029244
Main page: https://scientific-python.org/summits/domain-stacks/
Discussion: https://discuss.scientific-python.org/t/domain-stacks-summit/329

Participants#

Stéfan van der Walt (Scientific Python)
Juanita Gomez (Scientific Python)
Pamphile Roy (Scientific Python)
Paige Martin (Pangeo)
Ryan Abernathey (Pangeo)
Jim Pivarski (Scikit-HEP)
Henry Schreiner (Scikit-HEP)
Levi Wolf (Geography)
Martin Fleischmann (Geography)
Pey Lian Lim (Astropy)
Isaac Virshup (scverse)

Agenda#

(10 min) Stéfan discuss purpose of meeting and goals (10 min)

Meeting 1

Mon, 01 Jan 0001 00:00:00 +0000

Information#

Date: Monday, September 26th 11AM - 12PM Pacific time (click for your timezone)
Zoom Link (sign in required): https://berkeley.zoom.us/j/96628687450?pwd=Q29oeDc5NnJEYlFHVU1yVUFITzNQUT09

Participants#

Hameer Abbasi
Nezar Abdennur
Ross Barnowski
CJ Carey
Jérémie du Boisberranger
Julien Jerphanion
Jim Kitchen
Jarrod Millman
Dan Schult
Stéfan van der Walt
Isaac Virshup
Erik Welch
Meekail Zain

Agenda#

The summit is scheduled for 1 hour and will consist of a series of high-level birds-of-a-feather (BoF)-style talks, followed by more focused discussion.

Meeting 2

Mon, 01 Jan 0001 00:00:00 +0000

Information#

Date: Thursday, April 27th 11AM - 12PM Pacific time (click for your timezone)
Zoom Link: https://ucsc.zoom.us/j/93937164560?pwd=UmZKTmlHeGRMRkN3a2h2SDNJY3lvZz09

Participants#

Agenda#

The summit is scheduled for 1 hour and will focus on the proposed array semantics in scipy.sparse (download) and what work can be accomplished during the 2023 Scientific Python developer summit.

(5 min) Welcome & Introductions (Jarrod Millman)
(50 min) Discussion
- Who will be working on scipy sparse arrays during the summit?

MNE-Python

Mon, 01 Jan 0001 00:00:00 +0000

Narwhals

Mon, 01 Jan 0001 00:00:00 +0000

NetworkX

Mon, 01 Jan 0001 00:00:00 +0000

NumPy

Mon, 01 Jan 0001 00:00:00 +0000

pandas

Mon, 01 Jan 0001 00:00:00 +0000

Press kit

Mon, 01 Jan 0001 00:00:00 +0000

We would like to make it easy for you to include the Scientific Python project identity in your next academic paper, course materials, or presentation.

You will find SVG versions of the Scientific Python logo here. Note that by using the scientific-python.org resources, you accept the Scientific Python Code of Conduct.

PySAL

Mon, 01 Jan 0001 00:00:00 +0000

Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

With an extensive and high-quality ecosystem of libraries, scientific Python has emerged as the leading platform for data analysis. This ecosystem is sustained by independent volunteers with separate mailing lists, websites, roadmaps, documentation, engineering and packaging solutions, and governance structures.

The Scientific Python project aims to better coordinate the ecosystem and prepare scientific Python for the next decade of data science. We provide resources, planning, and coordination for the community.

Scikit-HEP

Mon, 01 Jan 0001 00:00:00 +0000

scikit-image

Mon, 01 Jan 0001 00:00:00 +0000

scikit-learn

Mon, 01 Jan 0001 00:00:00 +0000

SciPy

Mon, 01 Jan 0001 00:00:00 +0000

Summit Follow Up June 17

Mon, 01 Jan 0001 00:00:00 +0000

Friday June 17 2023, 14:00 to 15:00 UTC#

Attendees#

Dan
Erik
Isaac
CJ
Julien

Agenda#

Sort out and attribute what remains to be done
Roadmap/ plan for array-like sparse arrays
- Deprecation plan for matrices
- Creation routines
- Interop with other sparse matrix packages (isaac)
Blog post

Notes#

Introduction, Erik works on libraries using sparse datastructures
- [Julien] Erik: here is the worklog of the summit

What remains to be done#

How do we work
- [Dan] Where does in-progress work go
  - CJ’s fork, make PRs onto branch there
  - Using the view on the diff on GitHub might prevent the creation of spurious Draft PR
- Dense pretending to be sparse can be ephermeral, never needs to go to scipy
- Dan will make a branch for exploration that people can fork and make PRs to. Dan will try to keep it up-to-date with the main branch too.
1d sparse array support Timeline? - Dan + CJ working on it - Targeting 1.12
nd sparse array support
- Julien, CJ: seems like low priority and this would be a huge maintenance cost SciPy for relatively rare use-cases
- CJ: Array api makes interop easier, we can lean on external packages for now
creation functions scipy#18592
- Want to get them out in a single release
- Is Ross going to implement these, or need to assign elsewhere?
- Current plan: _array methods, so we can apply this to the fast_matrix_market io at the same type
- Bundling in api changes (like tuples for shapes)
Deprecate isspmatrix_ methods?
- Sounds like combination of classes + format attribute cover this
- (Julien) Deprecation plan: have suggested code to replace these things with
- Add a section in the documentation explaining the change of semantic and a migration plan for downstream libraries during the deprecation cycle.
  - Make a roadmap for array api
scikit-learn
- When will scikit-learn be willing to support a compatible version of scipy?
  - Julien:
    - reasoning is based on some linux package manager not supporting latest numpy scipy
    - Personally would like scikit-learn to follow SPEC 0
    - See discussion for 1.3
    - Dedicated RFC
    - Some part of scikit-learn’s behavior depends on SciPy’s version (e.g. see this one); we might potentially rely on similar mechanism for the support of sparse arrays (e.g. reshaping outputs)
    - scikit-learn uses nightly builds to test the developer version of SciPy and NumPy and failing tests are updated in this issue. We can watch those for breakage in scikit-learn when we make changes in SciPy.
General speed ups
- “Fast path for canonical”
  - TODO (Isaac)
    - Figure out canonical for all the formats, probably normalize APIs
    - Add keyword public argument for constructors

Broad plan for deprecation#

Add support for sparse_arrays everywhere we can
- Creation functions: target 1.12
- 1d COO: target 1.12
Deprecate spmatrix-specific functions (where there’s an easy replacement)
- Target: 1.12
Once everything works with sparse arrays, then deprecate spmatrix
- And the sparse array API is stable! (specifically, 1d support)
- Target: 1.13

Actionable items#

[Julien] Add a section in the documentation explaining the change of semantic and a migration plan for downstream libraries during the deprecation cycle.
[Julien] Drop isspmatrix_- checks in scikit-learn and use issparse and format

Next meeting date and time#

Every two weeks with the option to drop every other meeting (at least once a month)

[name=Julien] +1. I proposed to meet every two week for 1h on the same timeslot

xarray

Mon, 01 Jan 0001 00:00:00 +0000

Zarr-Python

Mon, 01 Jan 0001 00:00:00 +0000