Conversation
Welcome to GitGitGadgetHi @dcastro, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests. Please make sure that either:
You can CC potential reviewers by adding a footer to the PR description with the following syntax: NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description, Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:
It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code. Contributing the patchesBefore you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a comment to your PR of the form Both the person who commented An alternative is the channel Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment If you want to see what email(s) would be sent for a After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail). If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the curl -g --user "<EMailAddress>:<Password>" \
--url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txtTo iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description): To send a new iteration, just add another PR comment with the contents: Need help?New contributors who want advice are encouraged to join [email protected], where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join. You may also be able to find help in real time in the developer IRC channel, |
|
There is an issue in commit f68f660:
|
`git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which calculate the length of the common prefix of all *positive* pathspecs, `max_prefix_len`. `max_prefix_len` is then passed to `match_pathspec()` -> `match_pathspec_with_flags()` -> `do_match_pathspec()`, which strips `max_prefix_len` bytes off of *all* paths and `match_pathspec_item()` strips *all* pathspecs (positive or negative). This causes the bug previously reported in [1]. As a result, when we run `git ls-files -- sub/sub/sub/file ':(exclude)nonexistent'`: * The common prefix of the positive pathspecs is `sub/sub/sub`, 11 bytes * 11 bytes get stripped off both pathspecs: * "sub/sub/sub/file" becomes "/file" * "nonexistent" becomes "" * Since the negative pathspec degenerated into "", it matches every file, and thus no results are returned. When the common prefix is longer than the negative pathspec, we read out of bounds. `git add` suffers from the same issue. It uses `fill_directory()`, which returns the common prefix length, but doesn't strip the trailing slash. Using the same pathspecs as in the example above, the common prefix would be `sub/sub/sub/`, 12 bytes. Only `git ls-files` and `git add` are impacted. Other callers pass in `0` as the prefix. Bug introduced in: ef79b1f (Support pathspec magic :(exclude) and its short form :!, 2013-12-06). Solution: in `do_match_pathspec()`, only strip the prefix when handling positive pathspecs, not when handling negative pathspecs. [1]: https://lore.kernel.org/git/[email protected] Reported-by: Thomas Haller <[email protected]> Signed-off-by: Diogo Castro <[email protected]>
f68f660 to
02eb907
Compare
|
/allow |
|
User dcastro is now allowed to use GitGitGadget. |
|
/preview |
|
Preview email sent as [email protected] |
|
/submit |
|
Submitted as [email protected] To fetch this version into To fetch this version to local tag |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): "Diogo Castro via GitGitGadget" <[email protected]> writes:
> From: Diogo Castro <[email protected]>
>
> `git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which
> calculate the length of the common prefix of all *positive* pathspecs,
> `max_prefix_len`.
> ...
> Solution: in `do_match_pathspec()`, only strip the prefix when handling
> positive pathspecs, not when handling negative pathspecs.
Hmph, if the command line were
git ls-files -- a/b/c a/b/d !a/b/
shouldn't we strip a/b/ from all three? Would it make sense to
leave the negative one relative to the full tree? I am wondering
if the solution is to compute common prefix across both positive and
negative ones instead. |
|
Diogo Castro wrote on the Git mailing list (how to reply to this email): I don't think so.
As far as I can tell, the "strip the common prefix" feature is a
performance optimization aimed at avoiding walking the working
directory needlessly.
So for `git add -- a/b/c a/b/d`, there's no need to look anywhere
other than in `a/b/`.
But extending the "strip the common prefix" to negative pathspecs
could end up negating the benefits we get from this perf optimization.
E.g. in `git add -- a/b/c a/b/d ':!*.md'`, there is no prefix common
to *all* pathspecs, so we'd revert to walking the entire working
directory, even though `a/b/` would still suffice.
On Sun, 30 Aug 2026 at 15:25, Junio C Hamano <[email protected]> wrote:
>
> "Diogo Castro via GitGitGadget" <[email protected]> writes:
>
> > From: Diogo Castro <[email protected]>
> >
> > `git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which
> > calculate the length of the common prefix of all *positive* pathspecs,
> > `max_prefix_len`.
> > ...
> > Solution: in `do_match_pathspec()`, only strip the prefix when handling
> > positive pathspecs, not when handling negative pathspecs.
>
> Hmph, if the command line were
>
> git ls-files -- a/b/c a/b/d !a/b/
>
> shouldn't we strip a/b/ from all three? Would it make sense to
> leave the negative one relative to the full tree? I am wondering
> if the solution is to compute common prefix across both positive and
> negative ones instead.
> |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): Diogo Castro <[email protected]> writes:
> I don't think so.
>
> As far as I can tell, the "strip the common prefix" feature is a
> performance optimization aimed at avoiding walking the working
> directory needlessly.
> So for `git add -- a/b/c a/b/d`, there's no need to look anywhere
> other than in `a/b/`.
>
> But extending the "strip the common prefix" to negative pathspecs
> could end up negating the benefits we get from this perf optimization.
> E.g. in `git add -- a/b/c a/b/d ':!*.md'`, there is no prefix common
> to *all* pathspecs, so we'd revert to walking the entire working
> directory, even though `a/b/` would still suffice.
I was wondering more about case like this:
$ git add -- a/b/c a/b/d ':!a/b/x
I agree that it is nonsense to compute the common prefix over only
positive ones, and then to strip the common prefix from both
positive and negative ones, and it needs to be corrected. |
|
Diogo Castro wrote on the Git mailing list (how to reply to this email): I think there's some misunderstanding, please allow me to take a step
back and attempt to clarify.
My previous message was a reply to this:
> I am wondering if the solution is to compute common prefix across both positive and negative ones instead.
As far as I can tell, this "common prefix" feature does not affect the
semantics of "ls-files" or "add", it doesn't affect which files are
reported.
It only affects the performance.
Your first example of "git ls-files -- a/b/c a/b/d :!a/b/" already
works correctly, the pattern ":!a/b/" excludes everything from the
first 2 pathspecs.
So the discussion to be had is purely about performance.
My point was that computing the common prefix across both positive
*and* negative pathspecs would not improve performance, and might
actually make it worse.
The "common prefix" is mainly used to avoid walking the entire working
directory.
A couple of examples to illustrate:
* "git add -- a/b/c a/b/d ':!a/b/x'"
* Under the current implementation, the common prefix is "a/b/",
so as a performance optimization, we can look only into the "a/b/"
directory and ignore the others.
* Under your proposal of computing the "common prefix across both
positive and negative ones", the common prefix would still be "a/b/",
so performance wouldn't be affected.
* "git add -- a/b/c a/b/d ':!a/**/x'"
* Under the current implementation, the common prefix is "a/b/",
like in the example above.
* Under your proposal, the common prefix would be "a/", so we'd
have to walk _more_ directories, which would hurt performance.
Does that answer your question? Or perhaps I misunderstood your point? |
|
User |
|
Junio C Hamano wrote on the Git mailing list (how to reply to this email): Diogo Castro <[email protected]> writes:
> My point was that computing the common prefix across both positive
> *and* negative pathspecs would not improve performance, and might
> actually make it worse.
OK. Then that points at the right solution. Ignore negative ones
when finding what the common prefix is, strip it only from positive
ones to reduce the width of the traversal to come up with the list
of possible match candidates, and match them as full paths against
the negative ones to cull "within the positive set but is excluded"
paths, and the posted patch looks good.
I still wonder if we need different implementation when we have many
more negative patterns than the positive ones. In such a case, the
stage to filter paths that matched one positive pattern by finding
matches with a negative pattern among many of them, which may
benefit from having a similar common prefix (among negative
patterns) optimization, but that is a separate topic.
Thanks.
|
…ith ls-files negative pathspecs See: * https://lore.kernel.org/git/[email protected]/T/#u * git/git#2391
…ith ls-files negative pathspecs See: * https://lore.kernel.org/git/[email protected]/T/#u * git/git#2391
CC: Thomas Haller [email protected], Jeff King [email protected]
cc: Diogo Castro [email protected]