Skip to content

dir: fix negative pathspecs in git ls-files and git add - #2391

Open
dcastro wants to merge 1 commit into
git:maintfrom
dcastro:diogo.castro/fix-pathspecs-common-prefix
Open

dcastro wants to merge 1 commit into
git:maintfrom
dcastro:diogo.castro/fix-pathspecs-common-prefix

Conversation

@dcastro

@dcastro dcastro commented Aug 28, 2026 •

Copy link
Copy Markdown

CC: Thomas Haller [email protected], Jeff King [email protected]
cc: Diogo Castro [email protected]

@gitgitgadget-git

Copy link
Copy Markdown

Welcome to GitGitGadget

Hi @dcastro, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests.

Please make sure that either:

  • Your Pull Request has a good description, if it consists of multiple commits, as it will be used as cover letter.
  • Your Pull Request description is empty, if it consists of a single commit, as the commit message should be descriptive enough by itself.

You can CC potential reviewers by adding a footer to the PR description with the following syntax:

CC: Revi Ewer <[email protected]>, Ill Takalook <[email protected]>

NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description,
because it will result in a malformed CC list on the mailing list. See
example.

Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:

  • the lines should not exceed 76 columns,
  • the first line should be like a header and typically start with a prefix like "tests:" or "revisions:" to state which subsystem the change is about, and
  • the commit messages' body should be describing the "why?" of the change.
  • Finally, the commit messages should end in a Signed-off-by: line matching the commits' author.

It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code.

Contributing the patches

Before you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a comment to your PR of the form /allow. A good way to find other contributors is to locate recent pull requests where someone has been /allowed:

Both the person who commented /allow and the PR author are able to /allow you.

An alternative is the channel #git-devel on the Libera Chat IRC network:

<newcontributor> I've just created my first PR, could someone please /allow me? https://github.com/gitgitgadget/git/pull/12345
<veteran> newcontributor: it is done
<newcontributor> thanks!

Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment /submit.

If you want to see what email(s) would be sent for a /submit request, add a PR comment /preview to have the email(s) sent to you. You must have a public GitHub email address for this. Note that any reviewers CC'd via the list in the PR description will not actually be sent emails.

After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail).

If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the (raw) link), then import it into your mail program. If you use GMail, you can do this via:

curl -g --user "<EMailAddress>:<Password>" \
    --url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txt

To iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description):

Changes since v1:
- Fixed a typo in the commit message (found by ...)
- Added a code comment to ... as suggested by ...
...

To send a new iteration, just add another PR comment with the contents: /submit.

Need help?

New contributors who want advice are encouraged to join [email protected], where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join.

You may also be able to find help in real time in the developer IRC channel, #git-devel on Libera Chat. Remember that IRC does not support offline messaging, so if you send someone a private message and log out, they cannot respond to you. The scrollback of #git-devel is archived, though.

@gitgitgadget-git

Copy link
Copy Markdown

There is an issue in commit f68f660:
dir: fix negative pathspecs in git ls-filesandgit add``

  • Lines in the body of the commit messages should be wrapped between 60 and 76 characters.
    Indented lines, and lines without whitespace, are exempt

`git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which
calculate the length of the common prefix of all *positive* pathspecs,
`max_prefix_len`.

`max_prefix_len` is then passed to `match_pathspec()` ->
`match_pathspec_with_flags()` -> `do_match_pathspec()`, which strips
`max_prefix_len` bytes off of *all* paths and `match_pathspec_item()`
strips *all* pathspecs (positive or negative).

This causes the bug previously reported in [1].

As a result, when we run `git ls-files -- sub/sub/sub/file
':(exclude)nonexistent'`:
* The common prefix of the positive pathspecs is `sub/sub/sub`, 11 bytes
* 11 bytes get stripped off both pathspecs:
  * "sub/sub/sub/file" becomes "/file"
  * "nonexistent" becomes ""
* Since the negative pathspec degenerated into "", it matches every
  file, and thus no results are returned.

When the common prefix is longer than the negative pathspec, we read out
of bounds.

`git add` suffers from the same issue. It uses `fill_directory()`, which
returns the common prefix length, but doesn't strip the trailing slash.
Using the same pathspecs as in the example above, the common prefix
would be `sub/sub/sub/`, 12 bytes.

Only `git ls-files` and `git add` are impacted. Other callers pass in
`0` as the prefix.

Bug introduced in: ef79b1f (Support pathspec magic :(exclude) and its
short form :!, 2013-12-06).

Solution: in `do_match_pathspec()`, only strip the prefix when handling
positive pathspecs, not when handling negative pathspecs.

[1]: https://lore.kernel.org/git/[email protected]

Reported-by: Thomas Haller <[email protected]>
Signed-off-by: Diogo Castro <[email protected]>
@dcastro
dcastro force-pushed the diogo.castro/fix-pathspecs-common-prefix branch from f68f660 to 02eb907 Compare August 28, 2026 14:10
@dscho

dscho commented Aug 28, 2026

Copy link
Copy Markdown
Member

/allow

@gitgitgadget-git

Copy link
Copy Markdown

User dcastro is now allowed to use GitGitGadget.

@dcastro

dcastro commented Aug 28, 2026

Copy link
Copy Markdown
Author

/preview

@gitgitgadget-git

Copy link
Copy Markdown

Preview email sent as [email protected]

@dcastro

dcastro commented Aug 28, 2026

Copy link
Copy Markdown
Author

/submit

@gitgitgadget-git

Copy link
Copy Markdown

Submitted as [email protected]

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2391/dcastro/diogo.castro/fix-pathspecs-common-prefix-v1

To fetch this version to local tag pr-git-2391/dcastro/diogo.castro/fix-pathspecs-common-prefix-v1:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2391/dcastro/diogo.castro/fix-pathspecs-common-prefix-v1

@gitgitgadget-git

Copy link
Copy Markdown

Junio C Hamano wrote on the Git mailing list (how to reply to this email):

"Diogo Castro via GitGitGadget" <[email protected]> writes:

> From: Diogo Castro <[email protected]>
>
> `git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which
> calculate the length of the common prefix of all *positive* pathspecs,
> `max_prefix_len`.
> ...
> Solution: in `do_match_pathspec()`, only strip the prefix when handling
> positive pathspecs, not when handling negative pathspecs.

Hmph, if the command line were

	git ls-files -- a/b/c a/b/d !a/b/

shouldn't we strip a/b/ from all three?  Would it make sense to
leave the negative one relative to the full tree?  I am wondering
if the solution is to compute common prefix across both positive and
negative ones instead.

@gitgitgadget-git

Copy link
Copy Markdown

Diogo Castro wrote on the Git mailing list (how to reply to this email):

I don't think so.

As far as I can tell, the "strip the common prefix" feature is a
performance optimization aimed at avoiding walking the working
directory needlessly.
So for `git add -- a/b/c a/b/d`, there's no need to look anywhere
other than in `a/b/`.

But extending the "strip the common prefix" to negative pathspecs
could end up negating the benefits we get from this perf optimization.
E.g. in `git add -- a/b/c a/b/d ':!*.md'`, there is no prefix common
to *all* pathspecs, so we'd revert to walking the entire working
directory, even though `a/b/` would still suffice.


On Sun, 30 Aug 2026 at 15:25, Junio C Hamano <[email protected]> wrote:
>
> "Diogo Castro via GitGitGadget" <[email protected]> writes:
>
> > From: Diogo Castro <[email protected]>
> >
> > `git ls-files` calls `common_prefix()` / `get_common_prefix_len()` which
> > calculate the length of the common prefix of all *positive* pathspecs,
> > `max_prefix_len`.
> > ...
> > Solution: in `do_match_pathspec()`, only strip the prefix when handling
> > positive pathspecs, not when handling negative pathspecs.
>
> Hmph, if the command line were
>
>         git ls-files -- a/b/c a/b/d !a/b/
>
> shouldn't we strip a/b/ from all three?  Would it make sense to
> leave the negative one relative to the full tree?  I am wondering
> if the solution is to compute common prefix across both positive and
> negative ones instead.
>

@gitgitgadget-git

Copy link
Copy Markdown

Junio C Hamano wrote on the Git mailing list (how to reply to this email):

Diogo Castro <[email protected]> writes:

> I don't think so.
>
> As far as I can tell, the "strip the common prefix" feature is a
> performance optimization aimed at avoiding walking the working
> directory needlessly.
> So for `git add -- a/b/c a/b/d`, there's no need to look anywhere
> other than in `a/b/`.
>
> But extending the "strip the common prefix" to negative pathspecs
> could end up negating the benefits we get from this perf optimization.
> E.g. in `git add -- a/b/c a/b/d ':!*.md'`, there is no prefix common
> to *all* pathspecs, so we'd revert to walking the entire working
> directory, even though `a/b/` would still suffice.

I was wondering more about case like this:

    $ git add -- a/b/c a/b/d ':!a/b/x

I agree that it is nonsense to compute the common prefix over only
positive ones, and then to strip the common prefix from both
positive and negative ones, and it needs to be corrected.

@gitgitgadget-git

Copy link
Copy Markdown

Diogo Castro wrote on the Git mailing list (how to reply to this email):

I think there's some misunderstanding, please allow me to take a step
back and attempt to clarify.
My previous message was a reply to this:

> I am wondering if the solution is to compute common prefix across both positive and negative ones instead.

As far as I can tell, this "common prefix" feature does not affect the
semantics of "ls-files" or "add", it doesn't affect which files are
reported.
It only affects the performance.

Your first example of "git ls-files -- a/b/c a/b/d :!a/b/" already
works correctly, the pattern ":!a/b/" excludes everything from the
first 2 pathspecs.


So the discussion to be had is purely about performance.
My point was that computing the common prefix across both positive
*and* negative pathspecs would not improve performance, and might
actually make it worse.

The "common prefix" is mainly used to avoid walking the entire working
directory.
A couple of examples to illustrate:

* "git add -- a/b/c a/b/d ':!a/b/x'"
    * Under the current implementation, the common prefix is "a/b/",
so as a performance optimization, we can look only into the "a/b/"
directory and ignore the others.
    * Under your proposal of computing the "common prefix across both
positive and negative ones", the common prefix would still be "a/b/",
so performance wouldn't be affected.
* "git add -- a/b/c a/b/d ':!a/**/x'"
    * Under the current implementation, the common prefix is "a/b/",
like in the example above.
    * Under your proposal, the common prefix would be "a/", so we'd
have to walk _more_ directories, which would hurt performance.

Does that answer your question? Or perhaps I misunderstood your point?

@gitgitgadget-git

Copy link
Copy Markdown

User Diogo Castro <[email protected]> has been added to the cc: list.

@gitgitgadget-git

Copy link
Copy Markdown

Junio C Hamano wrote on the Git mailing list (how to reply to this email):

Diogo Castro <[email protected]> writes:

> My point was that computing the common prefix across both positive
> *and* negative pathspecs would not improve performance, and might
> actually make it worse.

OK.  Then that points at the right solution.  Ignore negative ones
when finding what the common prefix is, strip it only from positive
ones to reduce the width of the traversal to come up with the list
of possible match candidates, and match them as full paths against
the negative ones to cull "within the positive set but is excluded"
paths, and the posted patch looks good.

I still wonder if we need different implementation when we have many
more negative patterns than the positive ones.  In such a case, the
stage to filter paths that matched one positive pattern by finding
matches with a negative pattern among many of them, which may
benefit from having a similar common prefix (among negative
patterns) optimization, but that is a separate topic.

Thanks.

dcastro added a commit to dcastro/lsp-xreferee that referenced this pull request Sep 3, 2026
@ytausch

ytausch commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

dcastro added a commit to dcastro/lsp-xreferee that referenced this pull request Sep 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants