Skip to content

fix: raise processing error where filter is materialising null values - #162

Open
stevenhsd wants to merge 1 commit into
release_v010from
bug/sr_ndsp_656_notify_null_filter
Open

stevenhsd wants to merge 1 commit into
release_v010from
bug/sr_ndsp_656_notify_null_filter

Conversation

@stevenhsd

@stevenhsd stevenhsd commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

TLDR of changes

What kind of changes does this PR introduce?

Tick all that apply

  • fix: A bug fix. Correlates with PATCH in SemVer
  • feat: A new feature. Correlates with MINOR in SemVer
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
  • refactor: A code change that neither fixes a bug nor adds a feature
  • perf: A code change that improves performance
  • test: Adding missing or correcting existing tests
  • build: Changes that affect the build system or external dependencies (example scopes: pip, docker, npm)
  • ci: Changes to CI configuration files and scripts (example scopes: GitLabCI)

Please check if the PR fulfills these requirements

  • I have read and followed the Contributing guidance
  • Docs have been added / updated
  • Tests and Linting in the CI are passing
  • Changes have been reviewed and approved by a Project Maintainer

@sonarqubecloud

Copy link
Copy Markdown

entity = entities[config.entity_name]

if config.error_if_expression_null:
if entity.filter(f"({config.expression}) IS NULL").shape[0] > 0:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth using duckdb_get_entity_count for maintainability?

entity = entities[config.entity_name]

if config.error_if_expression_null:
if entity.filter(f"({config.expression}) IS NULL").count() > 0:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as previous comment but for spark?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants