Skip to content

profiling.query_engine config flag is broken for Rust query engine #341

@milindsrivastava1997

Description

@milindsrivastava1997

Problem

The profiling.query_engine: true option in config.yaml was originally implemented for the Python query engine using py-spy. Since the query engine was rewritten in Rust, this flag is completely non-functional:

  1. remote_monitor.py raises NotImplementedError — when --profile_query_engine is set and the Rust query engine process keyword is detected (either bare-metal or container), the code explicitly raises NotImplementedError("Profiling for Rust query engine is not implemented yet"). The experiment aborts.

  2. do_profiling in the Rust engine is a dead config fieldengine_config.rs declares and parses do_profiling: bool from the YAML config, but no Rust code ever reads or acts on it.

  3. py-spy is Python-only — even if the guard were removed, main_prometheus_client.py would try to run py-spy record --pid <pid> against a Rust binary, producing garbage output. py-spy understands CPython stack frame internals and cannot profile native binaries.

Context

  • The Arroyo engine (also Rust) already has working flamegraph profiling in remote_monitor.py via ~/.cargo/bin/flamegraph --pid <pid> — the exact same tool that would work for the Rust query engine.
  • The Rust workspace Cargo.toml has no [profile.release] overrides, so release binaries are built without debug symbols, making flamegraph output unreadable without a Cargo.toml change.
  • Container mode adds extra complexity (multi-stage Docker build strips symbols); bare-metal mode (use_container.query_engine: false) sidesteps this entirely since the binary is run directly from target/release/.

Impact

Setting profiling.query_engine: true crashes the experiment immediately. The flag is effectively dead config with no indication to the user that it no longer works.

Metadata

Metadata

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions