fix: Correct Flink PyArrow dependency constraints#6604
Conversation
Signed-off-by: Zane Wang <[email protected]>
|
the CI pipeline fails, @zelinewang can you try to update the "pixi.lock" file by running the following command? |
|
Also run |
|
done, ran pixi lock plus the requirements recompile for all three python versions. heads up the pybuild-deps step fails because faiss-cpu==1.14.3 has no sdist on pypi and that pin is already in master's minimal-sdist files, so I left the three -build.txt files untouched. also the fresh resolve moves pyarrow to 21.0.0 on py3.10 (24 on 3.11/3.12), both within the corrected floor |
|
here is the error info: |
Signed-off-by: Zane Wang <[email protected]>
|
oops, my local pixi was 0.73 which writes the v7 format. redid it with an older one, back to v6 now and the diff is way smaller. also fixed the missing DCO sign-off |
d3fc474 to
3309da1
Compare
|
@zelinewang needs rebase on master, there are conflicts |
The Flink extra currently combines a base
pyarrow>=21.0.0requirement with apyarrow<21.0.0extra requirement. The attemptedextra != 'flink'exclusiondoes not prevent pip or uv from evaluating both requirements, so
feast[flink]is unsatisfiable.This change puts the shared
pyarrow>=16.1.0lower bound in the basedependencies and adds only
pyarrow<21.0.0to the Flink extra. A normal Feastinstall can still select the latest PyArrow release, while a Flink install gets
the compatible
>=16.1.0,<21.0.0intersection.The dependency regression check now lives in a focused test module that does
not import the Flink runtime.
Which issue(s) this PR fixes
Fixes #6583.
Checks
git commit -s).Testing Strategy
Validated with:
.[flink]and the default project;Requires-Distmetadata;Release note