A high-performance tool for extracting JavaScript files from websites using concurrent crawling techniques.
go install github.com/rix4uni/jscrawler@latest
wget https://github.com/rix4uni/jscrawler/releases/download/v0.0.4/jscrawler-linux-amd64-0.0.4.tgz
tar -xvzf jscrawler-linux-amd64-0.0.4.tgz
rm -rf jscrawler-linux-amd64-0.0.4.tgz
mv jscrawler ~/go/bin/jscrawler
Or download binary release for your platform.
git clone --depth 1 https://github.com/rix4uni/jscrawler.git
cd jscrawler; go install
Usage of jscrawler:
--complete Get Complete URL (default false)
-o, --output string Output file to save results
--silent Silent mode.
-t, --threads int Number of threads to use (default 50)
--timeout int Timeout (in seconds) for http client (default 15)
--verbose Enable verbose output for debugging purposes.
--version Print the version of the tool and exit.cat targets.txt | jscrawler [options]or
echo "https://example.com" | jscrawler [options]| Flag | Short | Description | Default |
|---|---|---|---|
--complete |
Output complete URLs (absolute paths) | false |
|
--output |
-o |
Save results to specified file | |
--silent |
Suppress banner and non-essential output | false |
|
--threads |
-t |
Number of concurrent threads | 50 |
--timeout |
HTTP client timeout in seconds | 15 |
|
--verbose |
Enable detailed debug output | false |
|
--version |
Display version information and exit |
# Single target
echo "https://example.com" | jscrawler
# Multiple targets from file
cat subdomains.txt | jscrawlerecho "https://example.com" | jscrawler --completecat targets.txt | jscrawler --complete -o javascript_files.txt# Verbose mode for debugging
echo "https://example.com" | jscrawler --verbose --complete
# Silent operation in pipelines
cat targets.txt | jscrawler --silent --complete
# Custom performance tuning
cat targets.txt | jscrawler --threads 100 --timeout 30jscrawler demonstrates superior coverage compared to similar tools:
# Test against example.com
echo "https://www.example.com" | getJS --complete | wc -l
# Output: 4
echo "https://www.example.com" | subjs | wc -l
# Output: 8
echo "https://www.example.com" | jscrawler --silent --complete | wc -l
# Output: 13# Filter active hosts before processing
cat domains.txt | httpx -silent | jscrawler --complete# Remove duplicates and sort results
cat targets.txt | jscrawler --complete | unew | unique_js.txt# Balance between speed and resource usage
cat large_target_list.txt | jscrawler --threads 50 --timeout 20