Upgrade dependencies to latest stable and modernize for Python 3.11 - #825

⚠️ Breaking change: minimum Python is now 3.11

Python 3.9 and 3.10 are end-of-life, and several of the latest releases require 3.11 (ipython 9, ipykernel 7, pymongo 4.19, hypothesis). requires-python is now >=3.11,<4.0.0; CI workflows, black/mypy targets and the README/docs are updated to match.

Commit 1: dependency upgrade

• All runtime and dev dependencies bumped to their latest stable releases, e.g. pydantic 2.14.1, mypy 2.4.0, black 26.10.1, pylint 4.1.3, ipython 9.17, ipykernel 7.4, pymongo 4.19, dependency-injector 4.49.1, gmpy2 2.3.2, and uv_build 0.13. uv.lock regenerated.
• pytest-mock and types-python-dateutil moved from runtime dependencies to the dev group (only tests and mypy use them).
• pylint import-self and cyclic-import disabled: pylint 4.1 reports false positives on the mkinit-generated __init__.py files when a submodule name starts with the package name (e.g. encrypt_ballots/encrypt_ballots_*). Noted in pyproject.toml; the trade-off is that real import cycles are no longer reported until this is fixed upstream in pylint.

Commit 2: refactor

• PEP 585/604 annotations (list[...], X | None), collections.abc imports, datetime.UTC. __init__.py re-exports are untouched.
• Bug fix in Singleton: all subclasses shared one instance slot, so get_instance() returned the base Singleton object rather than the subclass. It now keeps one instance per class and is typed as returning Self. Added a regression test that fails on the old code.
• Simplified redundant comprehensions, nested ifs, boolean returns and dict.keys() checks.
• Moved broad-exception-caught pylint pragmas onto the except line (pylint 4.1 no longer applies the preceding-line form to the except clause) and use the current message name.
• test_utc_to_str_with_valid_utc_date no longer depends on the local timezone (it failed in UTC+3, where the hour has two digits).

Commit 3: GitHub Actions

• actions/checkout, actions/setup-python → v7, actions/upload-artifact → v7, github/codeql-action → v4, nick-fields/retry → v4, astral-sh/setup-uv → v10.3.0 (still pinned by SHA). The previous codeql-action@v2 and upload-artifact@v2 have been retired by GitHub, so CI could not run with them.
• The archived actions/create-release + actions/upload-release-asset pair is replaced by softprops/action-gh-release@v3. The release is now created after the PyPI upload, together with its zip asset.

Commits 4–5: design pattern fixes

An audit of the library's patterns (Singleton, Context Manager, Repository, Mediator, Builder, CLI step pipeline, GUI dependency injection) found these defects. Each fix comes with a test that fails on the old code:

• DiscreteLog (Singleton): call sites created a throwaway instance with DiscreteLog(), so set_max_exponent / set_lazy_evaluation never took effect. The library now uses DiscreteLog.get_instance(). precompute_cache_async filled a module-level dict instead of the instance cache; it now fills the instance cache.
• Scheduler (Context Manager): with Scheduler() opened the pools twice and leaked the first set, and tally.py / decryption.py created a scheduler per call (per contest during decryption) without ever closing it. Added use_scheduler(), which reuses a caller's scheduler or closes the one it creates. close() is now idempotent. The docstring no longer claims the class guarantees a single set of pools.
• ReadOnlyDataStore (Repository view): __eq__ called itself (RecursionError), __iter__ yielded (key, value) pairs, and __getitem__ returned None instead of raising KeyError. As a result in, keys(), items() and dict(view) were all wrong. It now follows the Mapping contract.
• The Mediators, Builder, CLI step pipeline and GUI DI container were reviewed; no defects found.
• The remaining Union[...] type aliases now use | syntax.

Verification

• pytest tests: 252 passed, 0 skipped on both Python 3.11.17 and 3.13 (Windows). Before these changes: 221 passed, 1 failed (the timezone test).
• black, mypy (170 files) and pylint (10.00/10) clean; uv build, twine check and mkdocs build --strict pass; workflow YAML validated.

Commits 6–20: fixes found by end-to-end testing, CI and review, plus dependency refreshes

Installing the branch into a clean environment and running real elections through the eg CLI found four more defects, and the CI run on this PR found a fifth. All five also exist on main. Every fix except the hypothesis dependency (a packaging change, checked by a clean install) comes with a test that fails on the old code:

• eg did not start after pip install electionguard: the CLI imports electionguard_tools, whose factories import hypothesis, which was only a dev dependency (ModuleNotFoundError). hypothesis is now a runtime dependency.
• The CLI could not decrypt when a guardian was missing, even with a quorum: the decrypt step only announced the present guardians and never the missing ones, so import-ballots with fewer guardian keys than the guardian count crashed with AssertionError: Unwrap called on None. It now announces missing guardians and has the present ones compensate, using the guardian keys exchanged during the key ceremony. Below the quorum it fails with a clear message.
• Missing guardians' records were not published: the election record only held records for the guardians present at decryption, so verifiers could not check compensated shares' recovery keys against the missing guardian's commitments. Every guardian's record is now published.
• Encryption flooded the console: encrypt.py and elgamal.py logged every ballot, selection and ciphertext at INFO, which the default console handler prints. Encrypting the six sample ballots printed 288 of the 380 output lines (about 280 KB of hex and raw bytes). The traces only contain public values, so this was noise rather than a leak; they now log at DEBUG.
• negate_q(0) overflowed: it returned q instead of 0, so encryption raised OverflowError whenever the random nonce in the disjunctive Chaum-Pedersen proof was 0. This is practically unreachable with the production q, but the test constants use q = 65521, so property tests failed at random (the FlakyFailure in Linux Check on CI run #1682). The result is now reduced mod q.

Two commits refresh dependencies released since the PR was opened, including python-dateutil 4.0.0 (a major version; the code only uses dateutil.parser.parse) and pydantic 2.14.3.

One commit replaces test_encrypt_contest_overvote_fails, skipped since 2022 as "runs forever". It no longer hangs, but it expected encrypt_contest to return None for an overvote, which is not what the code does now: the contest is encrypted with no votes counted and the overvote is recorded in the encrypted contest data. The new test checks that behaviour, including the proofs, and fails if the overvote handling is removed. The suite now has no skipped tests.

The last seven commits come from looking for hangs, reviewing that work and repeating the test suite. All of the defects below also exist on main:

• rand_range_q could hang or return a constant: a start at or above Q looped forever, a start just below the 256-bit Q needed about 2^256 draws, and a start of 0 always returned 0. It no longer loops. The only caller in the library passes 2, so none of this is reachable from the library today.
• discrete_log on a value with no small exponent: a wrong key or corrupted data made it search up to the 100 million limit, storing every step (estimated from a 300,000-step run at about 42 minutes and 214 GB). A baby-step giant-step check now rejects these values in under a second (measured 0.37 s, 22 MB) and leaves the cache untouched. decrypt_selection_with_decryption_shares returns None for them, as its docstring says. A value more than 100,000 steps past the cache now stores only its own entry, so g^100,000,000 takes about 0.14 s instead of storing every step. One known edge case: after such a value is cached, turning off lazy evaluation and precomputing can leave smaller values uncached, which raises DiscreteLogNotFoundError rather than giving a wrong result.
• Zero nonces: Nonces could derive 0 while elgamal_encrypt requires [1,Q), so encryption returned None. With the 16-bit test Q this is about 1 in 65,521 draws and made tests fail at random in setUp. A zero is now replaced by a deterministic re-hash. With the production Q a zero is practically never hashed, so existing nonces and records do not change.
• Fixed test deadlines: property tests failed with DeadlineExceeded on a slow machine even though their results were correct. The deadlines are removed; CI keeps its overall timeout.
• Random test failures from the test data: get_contest_description_well_formed could repeat a candidate id, producing an invalid description. Ids are now unique.
• eel warnings: these are hidden in pytest, not fixed. eel 0.18.2 uses names pyparsing has deprecated and only eel can change that, but its repository is archived, so no fix can land there. The filter only covers DeprecationWarning from module eel. Replacing eel would be a separate change.

End-to-end verification on Windows

• Clean installs from this branch (Python 3.11 and 3.13, latest PyPI packages including pydantic 2.14.2) and from main ran the same eg e2e election with identical tallies.
• Cross-version check: each version read and fully verified the others' election records (9/9), and re-serialized JSON is byte-for-byte identical, so the record format is unchanged.
• setup → mark-ballots → encrypt-ballots → submit-ballots → import-ballots with 5 guardians / quorum 3 on all five manifests in data/ (including Hamilton County, 68 selections), decrypted with all 5 guardians and with 2 missing: 10/10 tallies match a hand count of the plaintext ballots, and every record passes an independent check of ballot proofs, aggregation, decryption-share proofs, compensated-share recovery keys and Lagrange reconstruction. A deliberately tampered record is rejected.
• On the final code: 252 passed, 0 skipped in 8 of 8 runs on Python 3.13 and 2 of 2 on Python 3.11 (91% coverage). black, mypy and pylint are clean.
• These tests failed at random before, so clean runs are good evidence rather than proof. An earlier test_reconstruct_decryption_share failure was not captured; it uses the same setUp as the zero-nonce failure, so that is the likely cause but it is not confirmed.
• With pylint's cyclic-import check disabled, a one-off scan found no load-time cycles across 149 modules. It does not cover __init__.py re-exports or imports inside functions.
• egui graphical interface, one full election on Windows with a local MongoDB 7.0 and no Docker, in a browser, with one admin and two guardian instances: key ceremony (2 guardians, quorum 2), create election, upload 8 ballots made with eg (6 cast, 2 spoiled), tally and decrypt with both guardians. The tally page and a spoiled ballot page show the correct values, the tally matches a hand count, and the exported record passes the independent check on both this branch and main. No errors in the logs.
• Not covered in that egui run: the docker compose setup, the USB ballot upload wizard (the legacy upload form was used), a guardian missing at decryption, and the native file picker (files were set on the inputs by script). The encryption package and record exports were called through the app's eel functions, not their pages' buttons.

CI

This workflow passed on my fork at 7e41533 (Linux Check, MacOS Check, Code Analysis), without waiting for workflow approval here: https://github.com/ebubekirrcihangirr6060/electionguard-python/actions/runs/42328992493