Skip to content

add tag.gpgSign config option to force all tags be GPG-signed - #419

Closed
kofemann wants to merge 1 commit into
git:masterfrom
kofemann:tag-sign
Closed

add tag.gpgSign config option to force all tags be GPG-signed#419
kofemann wants to merge 1 commit into
git:masterfrom
kofemann:tag-sign

Conversation

@kofemann

Copy link
Copy Markdown
Contributor

Motivation:
in some workflows we have no control on how git command is executed,
however a signed tags are required.

The new config-file option tag.gpgSign enforces signed tags. Additional
command line option --no-gpg-sign is added to disable such behavior if
needed.

Signed-off-by: Tigran Mkrtchyan [email protected]

Thanks for taking the time to contribute to Git! Please be advised that the
Git community does not use github.com for their contributions. Instead, we use
a mailing list ([email protected]) for code submissions, code reviews, and
bug reports. Nevertheless, you can use submitGit to conveniently send your Pull
Requests commits to our mailing list.

Please read the "guidelines for contributing" linked above!

@kofemann

Copy link
Copy Markdown
Contributor Author

The alternative is to update the behaviour of forcesignannotated to do the same.

Motivation:
in some workflows we have no control on how git command is executed,
however a signed tags are required.

The new config-file option tag.gpgSign enforces signed tags. Additional
command line option --no-gpg-sign is added to disable such behavior if
needed.

Signed-off-by: Tigran Mkrtchyan <[email protected]>
@fawkesley

fawkesley commented Jan 16, 2019

Copy link
Copy Markdown

This seems like a great plan, is there some conversation about this elsewhere?

Edit:
This was discussed (and effectively rejected) on the mailing list:
https://public-inbox.org/git/[email protected]/

@dscho

dscho commented Jun 3, 2019

Copy link
Copy Markdown
Member

This was discussed (and effectively rejected) on the mailing list:

This is not my understanding after reading the replies.

Instead, I think what the reviewers asked for was a better commit message, i.e. a description that shows why this is a useful change, and there was even a hint how to phrase it: imitate the rationale for the commit.gpgSign option.

And even the latest answer by the author does a good job, and it would only have to be copy-edited into a new paragraph in the commit message, then to be submitted as v2 of the patch.

@kofemann

kofemann commented Jun 3, 2019 via email

Copy link
Copy Markdown
Contributor Author

@kofemann

kofemann commented Jun 4, 2019

Copy link
Copy Markdown
Contributor Author

@dscho

dscho commented Jun 4, 2019

Copy link
Copy Markdown
Member

Excellent!

@dscho

dscho commented Jul 1, 2019

Copy link
Copy Markdown
Member

This has been integrated into next: 1c6b565. It seems well on its way into master, so I'll close the PR.

@dscho dscho closed this Jul 1, 2019
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Sep 13, 2021
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Sep 21, 2021
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Nov 2, 2021
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Nov 2, 2021
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Nov 10, 2021
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Nov 10, 2021
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Nov 16, 2021
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Nov 16, 2021
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee added a commit to derrickstolee/git that referenced this pull request Jan 18, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jan 28, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Jan 28, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Mar 16, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Mar 16, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request May 22, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request May 22, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jun 23, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Jun 23, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jun 27, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Jun 27, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jul 14, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Jul 14, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Sep 30, 2022
diff: enable and test the sparse index
derrickstolee added a commit to derrickstolee/git that referenced this pull request Sep 30, 2022
One thing I forgot when talking about the sparse index is that we have a performance test: `t/perf/p2000-sparse-operations.sh`. This test wasn't helpful for commands like `git merge` that need a particular set of input, but work for more read-only operations.

Here is a quick demonstration of how this performance test works so we could have a definitive measure of how your previous updates improved performance. 

To get these results, I ran the following command in `t/perf`:

```
 ./run 4bcd533 f9255a5 f28fc01 b713582 -- p2000-sparse-operations.sh
```

The short-shas correspond to the merge commits for these PRs:

* git#410 
* git#421 
* git#417 
* git#419

The test takes a copy of the Git repository and creates several copies within a nested directory heirarchy.


```
Test                                                   4bcd533       f9255a5              f28fc01              b713582           
-------------------------------------------------------------------------------------------------------------------------------------------------
2000.2: git status (full-v3)                           0.19(0.15+0.05)   0.19(0.16+0.05) +0.0%    0.20(0.18+0.03) +5.3%    0.19(0.17+0.04) +0.0% 
2000.3: git status (full-v4)                           0.20(0.18+0.04)   0.19(0.15+0.06) -5.0%    0.21(0.18+0.05) +5.0%    0.18(0.18+0.02) -10.0%
2000.4: git status (sparse-v3)                         0.04(0.04+0.04)   0.05(0.07+0.04) +25.0%   0.04(0.04+0.05) +0.0%    0.04(0.06+0.04) +0.0% 
2000.5: git status (sparse-v4)                         0.04(0.03+0.06)   0.04(0.05+0.05) +0.0%    0.05(0.05+0.04) +25.0%   0.05(0.06+0.04) +25.0%
2000.6: git add -A (full-v3)                           0.36(0.29+0.05)   0.38(0.28+0.07) +5.6%    0.36(0.31+0.05) +0.0%    0.37(0.31+0.05) +2.8% 
2000.7: git add -A (full-v4)                           0.34(0.27+0.06)   0.34(0.29+0.05) +0.0%    0.34(0.29+0.04) +0.0%    0.35(0.28+0.06) +2.9% 
2000.8: git add -A (sparse-v3)                         0.06(0.07+0.04)   0.06(0.05+0.06) +0.0%    0.06(0.09+0.01) +0.0%    0.06(0.08+0.03) +0.0% 
2000.9: git add -A (sparse-v4)                         0.05(0.05+0.04)   0.05(0.05+0.07) +0.0%    0.05(0.04+0.06) +0.0%    0.06(0.06+0.05) +20.0%
2000.10: git add . (full-v3)                           0.38(0.31+0.05)   0.37(0.29+0.06) -2.6%    0.37(0.30+0.07) -2.6%    0.37(0.29+0.06) -2.6% 
2000.11: git add . (full-v4)                           0.35(0.31+0.04)   0.35(0.29+0.07) +0.0%    0.35(0.29+0.05) +0.0%    0.34(0.29+0.06) -2.9% 
2000.12: git add . (sparse-v3)                         0.06(0.06+0.05)   0.06(0.05+0.06) +0.0%    0.06(0.07+0.05) +0.0%    0.06(0.09+0.03) +0.0% 
2000.13: git add . (sparse-v4)                         0.06(0.06+0.06)   0.06(0.07+0.04) +0.0%    0.05(0.06+0.05) -16.7%   0.05(0.05+0.07) -16.7%
2000.14: git commit -a -m A (full-v3)                  0.48(0.37+0.08)   0.45(0.36+0.08) -6.2%    0.45(0.35+0.09) -6.2%    0.44(0.36+0.07) -8.3% 
2000.15: git commit -a -m A (full-v4)                  0.45(0.40+0.06)   0.43(0.34+0.07) -4.4%    0.45(0.37+0.06) +0.0%    0.42(0.36+0.05) -6.7% 
2000.16: git commit -a -m A (sparse-v3)                0.05(0.05+0.06)   0.05(0.05+0.03) +0.0%    0.05(0.06+0.06) +0.0%    0.05(0.04+0.06) +0.0% 
2000.17: git commit -a -m A (sparse-v4)                0.05(0.06+0.03)   0.05(0.06+0.04) +0.0%    0.06(0.07+0.05) +20.0%   0.05(0.04+0.06) +0.0% 
2000.18: git checkout -f - (full-v3)                   0.55(0.43+0.08)   0.54(0.46+0.05) -1.8%    0.55(0.46+0.07) +0.0%    0.54(0.40+0.10) -1.8% 
2000.19: git checkout -f - (full-v4)                   0.55(0.41+0.09)   0.50(0.40+0.09) -9.1%    0.51(0.46+0.05) -7.3%    0.51(0.44+0.06) -7.3% 
2000.20: git checkout -f - (sparse-v3)                 0.06(0.09+0.03)   0.06(0.08+0.03) +0.0%    0.06(0.06+0.05) +0.0%    0.07(0.09+0.03) +16.7%
2000.21: git checkout -f - (sparse-v4)                 0.06(0.08+0.04)   0.05(0.07+0.05) -16.7%   0.05(0.07+0.04) -16.7%   0.06(0.09+0.03) +0.0% 
```

All of the above were already integrated.

```
2000.22: git reset (full-v3)                           0.41(0.32+0.06)   0.40(0.31+0.06) -2.4%    0.41(0.33+0.05) +0.0%    0.42(0.34+0.04) +2.4% 
2000.23: git reset (full-v4)                           0.37(0.32+0.05)   0.35(0.30+0.05) -5.4%    0.37(0.30+0.05) +0.0%    0.35(0.31+0.03) -5.4% 
2000.24: git reset (sparse-v3)                         0.68(0.65+0.05)   0.55(0.52+0.04) -19.1%   0.04(0.05+0.04) -94.1%   0.04(0.05+0.04) -94.1%
2000.25: git reset (sparse-v4)                         0.70(0.65+0.05)   0.54(0.50+0.06) -22.9%   0.04(0.07+0.01) -94.3%   0.03(0.05+0.05) -95.7%
2000.26: git reset --hard (full-v3)                    0.54(0.43+0.07)   0.53(0.43+0.06) -1.9%    0.55(0.46+0.05) +1.9%    0.55(0.44+0.06) +1.9% 
2000.27: git reset --hard (full-v4)                    0.50(0.45+0.03)   0.50(0.43+0.05) +0.0%    0.49(0.41+0.06) -2.0%    0.50(0.42+0.05) +0.0% 
2000.28: git reset --hard (sparse-v3)                  0.83(0.76+0.06)   0.68(0.62+0.05) -18.1%   0.07(0.05+0.02) -91.6%   0.07(0.05+0.02) -91.6%
2000.29: git reset --hard (sparse-v4)                  0.80(0.75+0.05)   0.69(0.62+0.06) -13.8%   0.07(0.04+0.02) -91.2%   0.07(0.04+0.03) -91.2%
```

As expected, `git reset [--hard]` improves with the sparse index integration, but remains constant across the full index case.

```
2000.30: git update-index --add --remove (full-v3)     0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.01+0.01) +0.0% 
2000.31: git update-index --add --remove (full-v4)     0.03(0.02+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.03+0.00) +0.0%    0.03(0.02+0.01) +0.0% 
2000.32: git update-index --add --remove (sparse-v3)   0.57(0.54+0.02)   0.43(0.42+0.00) -24.6%   0.44(0.41+0.03) -22.8%   0.44(0.42+0.01) -22.8%
2000.33: git update-index --add --remove (sparse-v4)   0.56(0.52+0.04)   0.43(0.42+0.01) -23.2%   0.44(0.42+0.02) -21.4%   0.42(0.41+0.01) -25.0%
```

These do not change significantly because git#423 is not merged.

```
2000.34: git diff (full-v3)                            0.07(0.05+0.03)   0.06(0.05+0.03) -14.3%   0.07(0.05+0.03) +0.0%    0.06(0.05+0.03) -14.3%
2000.35: git diff (full-v4)                            0.06(0.05+0.03)   0.06(0.05+0.02) +0.0%    0.06(0.05+0.02) +0.0%    0.06(0.06+0.02) +0.0% 
2000.36: git diff (sparse-v3)                          0.25(0.23+0.03)   0.17(0.17+0.02) -32.0%   0.18(0.18+0.02) -28.0%   0.01(0.03+0.03) -96.0%
2000.37: git diff (sparse-v4)                          0.25(0.22+0.05)   0.16(0.16+0.01) -36.0%   0.18(0.15+0.04) -28.0%   0.01(0.04+0.02) -96.0%
2000.38: git diff --staged (full-v3)                   0.03(0.01+0.01)   0.03(0.02+0.01) +0.0%    0.03(0.02+0.01) +0.0%    0.03(0.02+0.00) +0.0% 
2000.39: git diff --staged (full-v4)                   0.04(0.03+0.01)   0.03(0.02+0.01) -25.0%   0.03(0.03+0.00) -25.0%   0.03(0.03+0.00) -25.0%
2000.40: git diff --staged (sparse-v3)                 0.21(0.19+0.01)   0.15(0.13+0.01) -28.6%   0.15(0.14+0.01) -28.6%   0.01(0.01+0.00) -95.2%
2000.41: git diff --staged (sparse-v4)                 0.22(0.21+0.01)   0.14(0.11+0.03) -36.4%   0.15(0.13+0.02) -31.8%   0.01(0.01+0.00) -95.5%
```

The `git diff` improvements are measurable.

```
2000.42: git sparse-checkout reapply (full-v3)         0.63(0.54+0.05)   0.56(0.48+0.04) -11.1%   0.57(0.48+0.03) -9.5%    0.59(0.48+0.05) -6.3% 
2000.43: git sparse-checkout reapply (full-v4)         0.60(0.54+0.02)   0.51(0.46+0.03) -15.0%   0.54(0.48+0.02) -10.0%   0.50(0.44+0.04) -16.7%
2000.44: git sparse-checkout reapply (sparse-v3)       0.91(0.86+0.05)   0.05(0.05+0.00) -94.5%   0.06(0.05+0.01) -93.4%   0.06(0.06+0.00) -93.4%
2000.45: git sparse-checkout reapply (sparse-v4)       0.92(0.88+0.04)   0.05(0.05+0.00) -94.6%   0.05(0.05+0.01) -94.6%   0.05(0.04+0.01) -94.6%
```

Finally, the `git sparse-checkout` measurements are also present.

This test script is particularly valuable when contributing changes upstream. It can be good to start by adding the lines to the performance test in an early commit, then demonstrating the performance change by copying the necessary lines from the output table into your commit message.
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Oct 4, 2022
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Nov 8, 2022
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Mar 15, 2023
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Mar 15, 2023
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request May 11, 2023
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Aug 23, 2023
diff: enable and test the sparse index
vdye pushed a commit to vdye/git that referenced this pull request Jan 9, 2024
diff: enable and test the sparse index
vdye pushed a commit to vdye/git that referenced this pull request Jan 9, 2024
diff: enable and test the sparse index
vdye pushed a commit to vdye/git that referenced this pull request Jan 9, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Apr 30, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request May 31, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jun 19, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Jul 19, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Aug 23, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Sep 30, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Oct 9, 2024
diff: enable and test the sparse index
dscho pushed a commit to derrickstolee/git that referenced this pull request Dec 10, 2024
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Aug 5, 2025
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Sep 2, 2025
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Nov 7, 2025
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Feb 3, 2026
diff: enable and test the sparse index
derrickstolee pushed a commit to derrickstolee/git that referenced this pull request Apr 22, 2026
diff: enable and test the sparse index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants