Skip to content

BUG: Deleting a repo, while being added needs a dedicated algo #13

Closed

Description

@Shashankss1205

The code should first check if the code to be deleted is currently being added. If yes, then it should first kill the job and then delete the package/code.

Activity

  1. Shashankss1205 commented on Aug 20, 2025

    @Shashankss1205
    CollaboratorAuthor

    Even without the explicit cancel_job mechanism proposed, the indexing process for any package stops as soon as we initiate the deletion.

    Here's why that happens:

    1. Database Transactions and Locking: Neo4j, like most databases, operates on transactions and uses locking
      mechanisms. When the add_package_to_graph (which calls build_graph_from_path_async) is running, it's
      continuously performing write operations (MERGE statements) to create nodes and relationships for each file and
      its contents.
    2. DETACH DELETE's Power: When you call delete_repository_from_graph, it executes a powerful Cypher query (MATCH
      (r:Repository {path: $path}) OPTIONAL MATCH (r)-[:CONTAINS*]->(e) DETACH DELETE r, e). This query attempts to:
      • Find the repository node.
      • Find all nodes and relationships contained within that repository (files, functions, classes, variables,
        imports, calls, etc.) to any depth.
      • Acquire locks on all these nodes and relationships.
      • DETACH DELETE: This is the key. It first removes all relationships connected to the nodes being deleted,
        and then deletes the nodes themselves. This is a very aggressive and efficient way to remove an entire
        subgraph.
    3. Conflict and Failure: As soon as the DETACH DELETE query successfully acquires its locks and starts deleting,
      any ongoing MERGE operations from the indexing job that try to write to those same nodes or relationships will
      immediately fail. The target nodes either no longer exist or are locked for deletion.
    4. Error Handling in build_graph_from_path_async: The build_graph_from_path_async function has try...except
      blocks. When a session.run() call (which is part of add_file_to_graph) fails due to the deletion, an exception
      is raised. This exception is caught, and the indexing job's status is updated to FAILED.

    In essence, the deletion didn't "cancel" the job gracefully; it effectively "pulled the rug out" from under the
    indexing process. The indexing job stopped because its attempts to write to the database were met with errors due
    to the concurrent deletion.

    Why the cancel_job is still beneficial (even if it seems to work now):

    While the current behavior stops the indexing, it's an ungraceful termination.

    • Potential for Inconsistency: In more complex scenarios, if the indexing job was in the middle of a multi-step
      transaction or a series of related writes, an abrupt failure could leave the database in a partially updated or
      inconsistent state.
    • Cleaner Shutdown: A cancel_job mechanism would allow the indexing process to acknowledge the cancellation
      request, finish any current atomic operation (like processing a single file), and then gracefully exit, marking
      its job as CANCELLED rather than FAILED. This leads to a cleaner state and more predictable behavior.

    So, this observation is correct that it stops, but the proposed cancel_job aims for a more controlled and robust
    shutdown, minimizing potential data integrity issues.

  2. sreeja-code-dev commented on Sep 29, 2025

    @sreeja-code-dev
    Contributor

    Hi @Shashanks1205 👋,

    I'd love to work on this issue as part of my Hacktoberfest contributions! The bug and its implications are super interesting, and I’ve already started exploring the 'build_graph_from_path_async' function and how the cancel_job mechanism could be improved for graceful termination.

    Could you please assign this issue to me? Also, if possible, kindly add the 'Hacktoberfest' label so it counts toward my contribution.

    Looking forward to collaborating—thanks for opening such a thoughtful issue!

    Best,
    Sreeja

  3. Shashankss1205 commented on Sep 29, 2025

    @Shashankss1205
    CollaboratorAuthor

    Yes assigned. Good luck 🤞 any sort of confusions can be asked for either here or in the discord server (link in readme)

  4. sreeja-code-dev commented on Sep 29, 2025

    @sreeja-code-dev
    Contributor

    For any confusions ,I will reach out. Thanks for assigning me this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workinggood first issueGood for newcomers

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions