TL;DR
kcov is unconditionally scanning all files under --execdir and then filtering out results from the report. This causes a fatal error in shellspec if kcov doesn't have permissions to access everything under the --execdir, even if it's excluded. It also hugely hurts execution time if there are lots of files in the project that are excluded from the coverage report.
No matter what options are passed to kcov with the --kcov-options argument, it always scans everything recursively under the --execdir currently. The kcov options --include-path, --exclude-path, --include-pattern, and --exclude-pattern seem to only affect what does or doesn't get included in the generated report, and does not seem to affect what it tries to scan.
The --execdir is usually expected to be the project root, and there are very good reasons for this (including being able to generate a single report for everything in project).
It's very common to have things within the project folder that kcov shouldn't scan, both because of performance, and because kcov may not be able to scan everything. Common reasons include:
- Some excluded things are very large or slow to scan
- There are a ton of excluded things in certain locations
- An excluded folder is inaccessible to
kcov due to permissions limits
Case 1 and 2 are performance related, and significantly hurt execution time when kcov takes a lot of time to scan things it then throws away the results for.
Case 3 is much worse, if kcov can't access files it exits with an error, which causes shellspec to return a fatal error. If you have (generated) files in your project that kcov does not have permissions to access, it always causes shellspec to exit with a fatal error. There is no workaround for this.
The necessary solution seems to be that instead of always unconditionally telling kcov to scan everything under --execdir and relying on the --kcov-options to filter things out of the results, some shellspec options equivalent to the kcov --include-paths, --exclude-paths, --include-patterns, and --exclude-patterns are needed that will pre-filter what files are passed to kcov in the first place.
TL;DR
kcovis unconditionally scanning all files under--execdirand then filtering out results from the report. This causes a fatal error inshellspecifkcovdoesn't have permissions to access everything under the--execdir, even if it's excluded. It also hugely hurts execution time if there are lots of files in the project that are excluded from the coverage report.No matter what options are passed to
kcovwith the--kcov-optionsargument, it always scans everything recursively under the--execdircurrently. Thekcovoptions--include-path,--exclude-path,--include-pattern, and--exclude-patternseem to only affect what does or doesn't get included in the generated report, and does not seem to affect what it tries to scan.The
--execdiris usually expected to be the project root, and there are very good reasons for this (including being able to generate a single report for everything in project).It's very common to have things within the project folder that
kcovshouldn't scan, both because of performance, and becausekcovmay not be able to scan everything. Common reasons include:kcovdue to permissions limitsCase 1 and 2 are performance related, and significantly hurt execution time when
kcovtakes a lot of time to scan things it then throws away the results for.Case 3 is much worse, if
kcovcan't access files it exits with an error, which causesshellspecto return a fatal error. If you have (generated) files in your project thatkcovdoes not have permissions to access, it always causesshellspecto exit with a fatal error. There is no workaround for this.The necessary solution seems to be that instead of always unconditionally telling
kcovto scan everything under--execdirand relying on the--kcov-optionsto filter things out of the results, someshellspecoptions equivalent to thekcov--include-paths,--exclude-paths,--include-patterns, and--exclude-patternsare needed that will pre-filter what files are passed to kcov in the first place.