Skip to content

feat: Add Oracle DB as Offline store in python sdk & operator#6017

Merged
ntkathole merged 40 commits intofeast-dev:masterfrom
aniketpalu:oracle-db-offline-store
Mar 14, 2026
Merged

feat: Add Oracle DB as Offline store in python sdk & operator#6017
ntkathole merged 40 commits intofeast-dev:masterfrom
aniketpalu:oracle-db-offline-store

Conversation

@aniketpalu
Copy link
Contributor

@aniketpalu aniketpalu commented Feb 24, 2026

What this PR does / why we need it:

Oracle DB Offline Store for Feast (ibis-based)

This PR adds a native Oracle Database offline store for Feast, built on the ibis-framework's Oracle backend (ibis.oracle). It follows the same proven pattern used by the existing MSSQL ibis offline store — delegating to the shared ibis.py functions for point-in-time joins, deduplication, and feature retrieval.
The implementation is ~300 lines total (vs 1096 lines in the SQLAlchemy-based approach), with no Jinja2 templates, no dialect-specific SQL workarounds, and no manual query generation. ibis generates Oracle SQL automatically and provides native Arrow transfer via to_pyarrow().

Oracle Column Name Handling

Oracle stores unquoted identifiers in UPPERCASE. For example, CREATE TABLE t (user_id INT) stores the column as USER_ID. This store uses a passthrough approach — column names are returned exactly as Oracle stores them, with no automatic case transformation.
This means:
Unquoted columns (the standard Oracle convention): The user references them in UPPERCASE, matching what they see in Oracle tools like SQL Developer, DBeaver, or DESCRIBE table.
Quoted columns (e.g., "CamelCase"): The user references them with the exact casing used when creating the table.
This design ensures predictable behavior — what you see in Oracle is what you use in Feast. No hidden transformations, no conventions to learn.

How to Use

feature_store.yaml

project: my_project
provider: local
registry: data/registry.db

offline_store:
  type: oracle
  user: feast
  password: feast
  host: localhost
  port: 1521
  service_name: FREEPDB1

online_store:
  type: sqlite
  path: data/online_store.db

What's Tested

Tested against a live Oracle 23ai database (Oracle Free 23.26.1.0.0):

Core Feast Operations:

  • Oracle connectivity via ibis.oracle.connect() with schema introspection
  • feast apply — registering entities, feature views, and feature services
  • get_historical_features — point-in-time correct feature retrieval with entity DataFrame
  • materialize — pulling latest features from Oracle offline store into SQLite online store
  • get_online_features — retrieving materialized features from the online store
  • pull_all_from_table_or_query — direct offline store batch retrieval with date range filtering
  • to_pyarrow() — native Arrow transfer via ibis (no intermediate pandas conversion)

Non-Entity Retrieval (entity_df=None):

  • With both start_date and end_date provided
  • With only end_date (start calculated from feature view TTL)
  • With only start_date (end defaults to current time)
  • With neither (end defaults to current time, start calculated from TTL)

Oracle Column Name Handling:

  • Unquoted identifiers (UPPERCASE) — e.g., USER_ID, FEATURE_TIME, TXN_LAST_24H
  • Quoted mixed-case identifiers — e.g., "User_ID" and "user_iD" in the same test run
  • Full Feast flow (apply → historical → materialize → online) verified for both casing styles

Which issue(s) this PR fixes:

#6018

Misc


Open with Devin

@aniketpalu aniketpalu requested a review from a team as a code owner February 24, 2026 09:40
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@Srihari1192
Copy link
Contributor

@aniketpalu Tested oracle DB configuration with feast operator and feast SDK , LGTM
Thanks

@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from de7b133 to 79b59da Compare February 26, 2026 18:07

def _read_data_source(data_source: DataSource, repo_path: str = "") -> Table:
table = _read_oracle_table(con, data_source)
return ibis.memtable(table.execute())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reading entire table in memory? Isn't there a better way to read filtered table based on timestamps may be?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty sure _build_data_source_reader_for_retrieval function is completely redundant, it just repackages _build_data_source_reader for no good reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ntkathole @tokoko

Added pre-filter to avoid reading the whole table. Thanks for the catch

Copy link
Collaborator

@tokoko tokoko Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be clear, I don't understand why we need _build_data_source_reader_for_retrieval at all. can't you use _build_data_source_reader in get_historical_features_ibis call? that way there will be no materialization

Copy link
Contributor Author

@aniketpalu aniketpalu Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tokoko

Thanks for clarification, I misunderstood your earlier comment. You are right, materialization can be avoided by using _build_data_source_reader.

After this fix, needed to do a little change in building entity_row_id in ibis as r = ibis.literal("") in _generate_row_id() creates broken entity_row_id due to Oracle DB casting it as NULL.

@ntkathole
Copy link
Member

Can we add documentation as well for offline store ?

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from d8c2abb to 4fffb65 Compare March 6, 2026 07:38
@ntkathole
Copy link
Member

@aniketpalu Need rebase

@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from ecc47ac to 9b363eb Compare March 9, 2026 11:50
devin-ai-integration[bot]

This comment was marked as resolved.

@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from 9b363eb to ecc47ac Compare March 9, 2026 12:45
Signed-off-by: Aniket Paluskar <[email protected]>
…s & validation for mutually exclusive fields

Signed-off-by: Aniket Paluskar <[email protected]>
Signed-off-by: Aniket Paluskar <[email protected]>
…0 EnumTypeWrapper incompatibility

Signed-off-by: Aniket Paluskar <[email protected]>
@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from ecc47ac to 33c0b38 Compare March 9, 2026 13:02
devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Contributor

@Srihari1192 Srihari1192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the changes LGTM
Thanks @aniketpalu

@aniketpalu aniketpalu requested a review from ntkathole March 10, 2026 09:03
@aniketpalu aniketpalu requested a review from tokoko March 10, 2026 18:19
Signed-off-by: Aniket Paluskar <[email protected]>
@aniketpalu aniketpalu requested a review from ntkathole March 11, 2026 05:39
@aniketpalu aniketpalu force-pushed the oracle-db-offline-store branch from ff2104a to 466e2d2 Compare March 11, 2026 07:14
Signed-off-by: Aniket Paluskar <[email protected]>
Copy link
Member

@ntkathole ntkathole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! Thanks @aniketpalu

@ntkathole ntkathole merged commit 9d35368 into feast-dev:master Mar 14, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants