Skip to content

fix: Complete data source form support for all SDK-configurable parameters#6613

Open
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps
Open

fix: Complete data source form support for all SDK-configurable parameters#6613
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps

Conversation

@ntkathole

Copy link
Copy Markdown
Member

What this PR does / why we need it:

The Feast UI data source creation/edit forms were missing many configuration parameters that the Python SDK supports. This PR performs a full audit of all 16 data source types and closes the gaps so users can fully configure any data source from the UI without needing to edit Python code.

When creating data sources through the Feast UI, users could only configure a subset of available options. For example:

  • SparkSource only exposed table and path, but not query, file_format, table_format (Iceberg/Delta/Hudi), or date_partition_column
  • Snowflake was missing query and warehouse
  • Kafka/Kinesis were missing message_format/record_format (required for deserialization)
  • File source was missing file_format and s3_endpoint_override
  • Several sources were missing date_partition_column for query optimization

This made it impossible to configure sources like SparkSource(path="...", table_format=IcebergFormat(...)) or RaySource(reader_type="parquet", path="abfss://...") entirely from the UI.

@ntkathole ntkathole self-assigned this Jul 18, 2026
@ntkathole
ntkathole requested a review from a team as a code owner July 18, 2026 07:32
@codecov-commenter

codecov-commenter commented Jul 18, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 55.76923% with 69 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.67%. Comparing base (e0a8573) to head (3031f02).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/api/registry/rest/data_sources.py 54.41% 28 Missing and 3 partials ⚠️
...k/python/feast/api/registry/rest/saved_datasets.py 70.42% 15 Missing and 6 partials ⚠️
sdk/python/feast/dataset_utils.py 0.00% 17 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6613      +/-   ##
==========================================
+ Coverage   45.64%   45.67%   +0.03%     
==========================================
  Files         412      412              
  Lines       48761    48915     +154     
  Branches     6913     6948      +35     
==========================================
+ Hits        22257    22343      +86     
- Misses      24957    25016      +59     
- Partials     1547     1556       +9     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 46.95% <55.76%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/dataset_utils.py 0.00% <0.00%> (ø)
...k/python/feast/api/registry/rest/saved_datasets.py 48.47% <70.42%> (+7.84%) ⬆️
sdk/python/feast/api/registry/rest/data_sources.py 63.95% <54.41%> (-5.04%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0a8573...3031f02. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch from 5ef6de9 to 3031f02 Compare July 18, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants