Skip to content

Memory budgets for derived state (V15) #321

Closed

Description

@jsam

Part of epic #305.

Problem. Per-request memory is metered, but no budget covers standing-query result sets, plan caches, HNSW graphs or arrangements; max_graph_memory_bytes bounds stored facts only (src/storage_engine/program_commit.rs:84-87; src/execution/memory.rs; /metrics memory = tuples x 64, rest/handlers/admin.rs:124-125).

Target. Per-KG accounting of arrangement bytes (trace batch sizes) and view rows, charged against max_graph_memory_bytes extended to derived state (or a new max_view_memory_bytes); +rule refused at registration when the initial build would exceed the budget ("view too large", with the measured size); a KG whose views grow past the budget enters views_over_budget (reads keep serving at the last frontier, writes refuse with resource_exhausted, same contract as #275); metrics per KG/view.

Acceptance. Tests: registration refusal, over-budget state and recovery after deletes; metric values match map_batches sums; the lab pack at 1M edges reports about the micro-benchmark's 344 MB.

Size M. Dependencies V2. Order slot 6.

Activity

  1. jsam commented on Oct 6, 2026

    @jsam
    ContributorAuthor

    The budget, the views_over_budget state and the per-graph and per-relation metrics landed in https://github.com/inputlayer/inputlayer-symbolic/pull/27. Still open: refusing a rule at registration when its initial build would exceed the budget (with the measured size), per-view metrics, and the 1M-edge memory figure; these wait for rules compiled into the maintainer (issue 310).

  2. jsam commented on Oct 10, 2026

    @jsam
    ContributorAuthor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions