Skip to content

run cannot be rerun; its workflow file may be broken #9989

Closed

Description

@wknapik

Describe the bug

We have a workflow that's meant to trigger reruns of another workflow.

In the most recent run, we hit the following failure:

Rerunning 11620789306 for https://github.com/brave/brave-core/pull/26329
run 11620789306 cannot be rerun; its workflow file may be broken
Error: Process completed with exit code 1.

The workflow file appears to be fine. The workflow ran successfully 2 months ago, but can no longer be rerun automatically, or manually (there's no button to rerun in the web UI).

Steps to reproduce the behavior

  1. Try to rerun workflow run 11620789306

Expected vs actual behavior

Successful execution vs failed execution

Logs

See: above

Activity

  1. williammartin commented on Dec 2, 2024

    @williammartin
    Member

    Can you share the output with GH_DEBUG=api turned on?

    This error message was introduced here: https://github.com/cli/cli/pull/3333/files#diff-963fe8161ea8e4c6652311e8471eb63614f0eb2e11c31e82df5c62d29fa48ed7R106-R108

    For some reason 403 is treated as it possibly being broken, but I haven't found any evidence for that being the case.

  2. wknapik commented on Dec 2, 2024

    @wknapik
    Author

    The debug message says

    Unable to retry this workflow run because it was created over a month ago
    

    Why does it matter how long ago a workflow ran?

  3. williammartin commented on Dec 2, 2024

    @williammartin
    Member

    Assuming that message was in the API, we should probably expose it. I don't see a good reason to suppress it and guess.

    Why does it matter how long ago a workflow ran?

    Honestly, I have no idea why, but it's documented here.

  4. wknapik commented on Dec 2, 2024

    @wknapik
    Author

    Assuming that message was in the API, we should probably expose it.

    That would be nice.

    Why does it matter how long ago a workflow ran?

    Honestly, I have no idea why, but it's documented here.

    The full message with GH_API=debug was

    {
      "message": "Unable to retry this workflow run because it was created over a month ago",
      "documentation_url": "https://docs.github.com/rest/actions/workflow-runs#re-run-a-workflow",
      "status": "403"
    }

    but the linked page doesn't mention the 1 month limit. Would be good to update the url in the API response.

    Also, at the url you linked, it says "30 days" instead of "1 month", which isn't very friendly towards ^F page search (the API message uses the word "month", so that's what I immediately searched for).

  5. williammartin commented on Dec 2, 2024

    @williammartin
    Member

    Yeh it's a bit weird to be vague about the exact limit in the message. I'll bring it up with the team that own the API.

    Acceptance Criteria

    Given I have a workflow run that is older than 30 days
    When I run gh run rerun ...
    Then I see an error that the workflow can't be rerun because it is older than 30 days

  6. added
    gh-runrelating to the gh run command
    and removed
    more-info-neededMore info needed from user/contributor
    on Dec 2, 2024
  7. iamrajhans commented on Jan 13, 2025

    @iamrajhans
    Contributor

    @williammartin can i take this issue
    i have figured the code reside here to resolve am thinking to check the createdAt date is passed 30 days and return error accordingly, please let me know am going in the right direction, thanks !

    totalHours := time.Now().Sub(run.CreatedAt).Hours()
    if totalHours > 30*24 {
    	return fmt.Errorf("run %d cannot be rerun; workflow was created 30 days ago", run.ID)
    }
    
  8. williammartin commented on Jan 13, 2025

    @williammartin
    Member

    Hey @iamrajhans,

    Sure you can open a PR for this, thanks. I don't think your approach is quite right though. As mentioned in #9989 (comment) the API actually returns"

    {
      "message": "Unable to retry this workflow run because it was created over a month ago",
      "documentation_url": "https://docs.github.com/rest/actions/workflow-runs#re-run-a-workflow",
      "status": "403"
    }
    

    I think we want to have a look at the .Message field on the error here: https://github.com/cli/cli/blob/trunk/pkg/cmd/run/rerun/rerun.go#L203C16-L203C17 and see if it matches. If it does then maybe surface it directly rather than creating a new error.

  9. iamrajhans commented on Jan 13, 2025

    @iamrajhans
    Contributor

    @williammartin initially I had a similar thought but decided to check the date instead since error messages can vary. Thanks so much for your guidance!, so it will be something like this

    if errors.As(err, &httpError) && httpError.StatusCode == 403 {
        if httpError.Message == "Unable to retry this workflow run because it was created over a month ago" {
              return fmt.Errorf("run %d  %s", run.ID, httpError.Message)
    	}
        return fmt.Errorf("run %d cannot be rerun; its workflow file may be broken", run.ID)
    }
    
  10. williammartin commented on Jan 13, 2025

    @williammartin
    Member

    I'll have to play around with it to see how it feels but yes that seems about right, thanks.

  11. jtmcg commented on Jan 16, 2025

    @jtmcg
    Contributor

    @iamrajhans and @williammartin, why match on the error message and not just return the error block?

    if errors.As(err, &httpError) && httpError.StatusCode == 403 {
        return fmt.Errorf("run %d cannot be rerun; %s", run.ID, httpError.Message)
    }

    What's the reason to mask the other 403 error messages (besides historical precedent)?

  12. iamrajhans commented on Jan 21, 2025

    @iamrajhans
    Contributor

    @jtmcg
    Matching on the specific error message ensures that users receive a clear and actionable explanation for the “older than 30 days” scenario, which is a known special case. If we returned all 403 errors verbatim, unrelated messages could be surfaced to users in a way that might be confusing or unhelpful.

    Rerunning 11620789306 for brave/brave-core#26329
    run 11620789306 cannot be rerun; its workflow file may be broken
    Error: Process completed with exit code 1.

    By checking for the “over a month ago” text, we provide a precise message for that particular error and maintain a fallback for other potential 403 causes. \cc @williammartin

  13. williammartin commented on Jan 21, 2025

    @williammartin
    Member

    I honestly don't know @jtmcg. Sadly it's forever old and there's no obvious trail to follow.

    To be honest I'd probably be fine removing it in another PR.

  14. jtmcg commented on Jan 21, 2025

    @jtmcg
    Contributor

    By checking for the “over a month ago” text, we provide a precise message for that particular error and maintain a fallback for other potential 403 causes. \cc @williammartin

    Ah, I think I wasn't clear, what I'm suggesting is that we return ALL 403 error messages so that we can let the API tell us what went wrong. When that error is "Unable to retry this workflow run because it was created over a month ago", you'd get that message. If that wasn't the error message, you'd presumably get a more useful message back from the API besides run <runId> cannot be rerun; its workflow file may be broken

    To @williammartin's point, though, we don't know why it was implemented this way and there may be a reason we don't understand to mask the other 403 responses (though it's not obvious to me because they would all be accessible via GH_DEBUG=api anyway).

    All said, I'm fine to accept this as-is

  15. jtmcg commented on Feb 21, 2025

    @jtmcg
    Contributor

    This was fixed in #10227, so closing this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggh-runrelating to the gh run commandhelp wantedContributions welcome

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions