Skip to content

Handle pagination of results with more than 10,000 results gracefully #1506

Description

@Siyfion

Description of the problem, including code/CLI snippet

Currently if you try to get a list with a lot of results, eg. the main GitLab project's issues list

gl = gitlab.Gitlab('https://gitlab.com', private_token='XXXXXXX')
gl.auth()
gitlab_prj = gl.projects.get(278964, lazy=True)
issues = gitlab_prj.issues.list(order_by='updated_at', sort='asc', per_page=100, as_list=False)

You'll get an error when you try and use the list:

Exception has occurred: TypeError
int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Due to the fact that, for performance reasons GitLab don't return certain headers when the responses exceed 10,000: https://docs.gitlab.com/ee/user/gitlab_com/index.html#pagination-response-headers

Expected Behavior

Gracefully handle this lack of x-total and x-total-pages header, either by limiting the results to 10,000 responses, or allowing a user to manually set the _list._total and _list._total_pages values.

Actual Behavior

Crashes with the above exception.

Specifications

  • python-gitlab version: 2.7.1
  • API version you are using (v3/v4): v4
  • Gitlab server version (or gitlab.com): gitlab.com

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions