Skip to content

Issue with cloning subdirectories which contain large number of files #9

Description

@Sobiru

Github truncates subdirectories which have many files such as https://github.com/iamcal/emoji-data/tree/master/img-apple-64 . As you can see, it says Sorry, we had to truncate this directory to 1,000 files. 2,524 entries were omitted from the list.

When you want to ghclone such subdirectory, it clones only the 1000 files and not all of the files. Is there a workaround?

Thank you.

Activity

  1. HR commented on Mar 4, 2022

    @HR
    Owner

    Thanks for reporting this @Sobiru

    So according to the GitHub API docs, the repo directory contents API has an upper limit of 1,000 files for a directory. However, it recommends using the Git Trees API to retrieve more.

    Here's the request for that https://api.github.com/repos/iamcal/emoji-data/git/trees/d779b828b2ee049e35f69e2b6edf71173504f364 and it returns all of the files but requires at least another request to find the tree_sha for the given path/directory (i.e. img-apple-64)

    I'll look into implementing when I have time but you're welcome to add it yourself and open a PR! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions