Skip to content

Commit 9b430ca

Browse files
achalsachals
authored
Remove duplicates in setup.py and run rudimentary verifications (#2016)
* Remove duplicates in setup.py Signed-off-by: Achal Shah <[email protected]> * move pip-tools Signed-off-by: Achal Shah <[email protected]> * Add step Signed-off-by: Achal Shah <[email protected]> * use extend Signed-off-by: Achal Shah <[email protected]> * remove ~ Signed-off-by: Achal Shah <[email protected]> * dev_required Signed-off-by: Achal Shah <[email protected]> * break deps Signed-off-by: Achal Shah <[email protected]> * unbreak Signed-off-by: Achal Shah <[email protected]> * Add proto-plus Signed-off-by: Achal Shah <[email protected]>
1 parent cbe45a0 commit 9b430ca

4 files changed

Lines changed: 24 additions & 21 deletions

File tree

‎.github/workflows/unit_tests.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ jobs:
4141
run: make install-python-ci-dependencies
4242
- name: Test Python
4343
run: FEAST_USAGE=False pytest -n 8 --cov=./ --cov-report=xml --verbose --color=yes sdk/python/tests
44+
- name: Ensure conflict-free dependencies
45+
run: FEAST_USAGE=False pip-compile --dry-run sdk/python/setup.py --extra ci
4446
- name: Upload coverage to Codecov
4547
uses: codecov/codecov-action@v1
4648
with:

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ pip install --upgrade pip
5959

6060
4. Install development dependencies for Feast Python SDK / CLI
6161
```sh
62-
pip install -e "sdk/python[ci]"
62+
pip install -e "sdk/python[dev]"
6363
```
6464

6565
### Code Style & Linting

‎README.md‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -137,31 +137,36 @@ The list below contains the functionality that contributors are planning to deve
137137
* [x] [Redshift source](https://docs.feast.dev/reference/data-sources/redshift)
138138
* [x] [BigQuery source](https://docs.feast.dev/reference/data-sources/bigquery)
139139
* [x] [Parquet file source](https://docs.feast.dev/reference/data-sources/file)
140-
* [ ] Kafka source (Planned for Q4 2021)
141-
* [ ] Synapse source (Planned for Q4 2021)
140+
* [x] [Synapse source (community plugin)](https://github.com/Azure/feast-azure)
141+
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
142+
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
143+
* [x] Kafka source (with [push support into the online store](reference/alpha-stream-ingestion.md))
142144
* [ ] Snowflake source (Planned for Q4 2021)
143145
* [ ] HTTP source
144146
* **Offline Stores**
145147
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
146148
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
149+
* [x] [Synapse (community plugin)](https://github.com/Azure/feast-azure)
150+
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
151+
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
147152
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
148153
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/adding-a-new-offline-store)
149-
* [x] [Hive (community maintained)](https://github.com/baineng/feast-hive)
150-
* [x] [Postgres (community maintained)](https://github.com/nossrannug/feast-postgres)
151154
* [ ] Snowflake (Planned for Q4 2021)
152-
* [ ] Synapse (Planned for Q4 2021)
155+
* [ ] Trino (Planned for Q4 2021)
153156
* **Online Stores**
154157
* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
155158
* [x] [Redis](https://docs.feast.dev/reference/online-stores/redis)
156159
* [x] [Datastore](https://docs.feast.dev/reference/online-stores/datastore)
157160
* [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
161+
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
162+
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
158163
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
159-
* [x] [Postgres (community maintained)](https://github.com/nossrannug/feast-postgres)
160164
* [ ] Bigtable
161-
* [ ] Cassandra\\
165+
* [ ] Cassandra
162166
* **Streaming**
163167
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
164-
* [ ] Streaming ingestion on AWS (Planned for Q4 2021)
168+
* [x] [Push based streaming data ingestion](reference/alpha-stream-ingestion.md)
169+
* [ ] Streaming ingestion on AWS
165170
* [ ] Streaming ingestion on GCP
166171
* **Feature Engineering**
167172
* [x] On-demand Transformations (Alpha release. See [RFC](https://docs.google.com/document/d/1lgfIw0Drc65LpaxbUu49RCeJgMew547meSJttnUqz7c/edit#))
@@ -171,14 +176,14 @@ The list below contains the functionality that contributors are planning to deve
171176
* [x] AWS Lambda (Alpha release. See [RFC](https://docs.google.com/document/d/1eZWKWzfBif66LDN32IajpaG-j82LSHCCOzY6R7Ax7MI/edit))
172177
* [ ] Cloud Run
173178
* [ ] Kubernetes
174-
* [ ] KNative\\
179+
* [ ] KNative
175180
* **Feature Serving**
176181
* [x] Python Client
177182
* [x] REST Feature Server (Python) (Alpha release. See [RFC](https://docs.google.com/document/d/1iXvFhAsJ5jgAhPOpTdB3j-Wj1S9x3Ev\_Wr6ZpnLzER4/edit))
178183
* [x] gRPC Feature Server (Java) (See [#1497](https://github.com/feast-dev/feast/issues/1497))
184+
* [x] Push API
179185
* [ ] Java Client
180186
* [ ] Go Client
181-
* [ ] Push API
182187
* [ ] Delete API
183188
* [ ] Feature Logging (for training)
184189
* **Data Quality Management**

‎sdk/python/setup.py‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@
5454
"pandas>=1.0.0",
5555
"pandavro==1.5.*",
5656
"protobuf>=3.10",
57+
"proto-plus",
5758
"pyarrow>=4.0.0",
5859
"pydantic>=1.0.0",
5960
"PyYAML>=5.4.*",
@@ -113,15 +114,10 @@
113114
"firebase-admin==4.5.2",
114115
"pre-commit",
115116
"assertpy==1.1",
116-
"proto-plus<1.19.7",
117-
"google-cloud-bigquery>=2.28.1",
118-
"google-cloud-bigquery-storage >= 2.0.0",
119-
"google-cloud-datastore>=2.1.*",
120-
"google-cloud-storage>=1.20.*,<1.41",
121-
"google-cloud-core==1.4.*",
122-
"redis-py-cluster==2.1.2",
123-
"boto3==1.17.*",
124-
]
117+
"pip-tools"
118+
] + GCP_REQUIRED + REDIS_REQUIRED + AWS_REQUIRED
119+
120+
DEV_REQUIRED = ["mypy-protobuf==1.*", "grpcio-testing==1.*"] + CI_REQUIRED
125121

126122
# Get git repo root directory
127123
repo_root = str(pathlib.Path(__file__).resolve().parent.parent.parent)
@@ -215,7 +211,7 @@ def run(self):
215211
# https://stackoverflow.com/questions/28509965/setuptools-development-requirements
216212
# Install dev requirements with: pip install -e .[dev]
217213
extras_require={
218-
"dev": ["mypy-protobuf==1.*", "grpcio-testing==1.*"],
214+
"dev": DEV_REQUIRED,
219215
"ci": CI_REQUIRED,
220216
"gcp": GCP_REQUIRED,
221217
"aws": AWS_REQUIRED,

0 commit comments

Comments
 (0)