-
Notifications
You must be signed in to change notification settings - Fork 75
Add prek pre-commit hooks and CI #587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
43e96f1
chore: add prek pre-commit hook configuration and CI
kvinwang 1e0d995
fix: add SPDX header to prek.toml and permissions to CI workflow
kvinwang c8d2cac
chore: rename pre-commit to prek in CI workflow
kvinwang d71ef88
fix: explicitly pass ruff config path for correct resolution
kvinwang ec0df8a
fix: use root-level ruff.toml instead of sdk/python config
kvinwang b6f9d85
fix: sort imports in vmm-cli.py to pass ruff isort check
kvinwang e77c647
fix: resolve all E501 line-too-long issues in vmm-cli.py
kvinwang 194b6d9
fix: sort imports and remove f-string prefix in Python files
kvinwang 6a3c130
fix: add docstrings to all Python files and remove ruff.toml
kvinwang a5cc52e
fix: redo vmm-cli.py lint fixes on latest master version
kvinwang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # SPDX-FileCopyrightText: © 2025 Phala Network <[email protected]> | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| name: Prek checks | ||
|
|
||
| on: | ||
| push: | ||
| branches: [ master, next, dev-* ] | ||
| pull_request: | ||
| branches: [ master, next, dev-* ] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| prek: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Install Rust | ||
| uses: dtolnay/rust-toolchain@master | ||
| with: | ||
| toolchain: 1.92.0 | ||
| components: rustfmt | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.22' | ||
|
|
||
| - name: Install prek | ||
| run: pip install prek | ||
|
|
||
| - name: Run prek checks (PR) | ||
| if: github.event_name == 'pull_request' | ||
| run: prek run --from-ref ${{ github.event.pull_request.base.sha }} --to-ref ${{ github.event.pull_request.head.sha }} --show-diff-on-failure | ||
|
|
||
| - name: Run prek checks (push) | ||
| if: github.event_name == 'push' | ||
| run: prek run --from-ref ${{ github.event.before }} --to-ref ${{ github.event.after }} --show-diff-on-failure | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # SPDX-FileCopyrightText: © 2025 Phala Network <[email protected]> | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # ============================================================================= | ||
| # prek configuration for dstack | ||
| # ============================================================================= | ||
|
|
||
| # --- General hooks (trailing whitespace, file checks, etc.) --- | ||
| [[repos]] | ||
| repo = "https://github.com/pre-commit/pre-commit-hooks" | ||
| rev = "v5.0.0" | ||
| hooks = [ | ||
| { id = "trailing-whitespace", args = ["--markdown-linebreak-ext=md"] }, | ||
| { id = "end-of-file-fixer" }, | ||
| { id = "check-yaml", args = ["--allow-multiple-documents"], exclude = "gateway/templates/" }, | ||
| { id = "check-toml" }, | ||
| { id = "check-json" }, | ||
| { id = "check-merge-conflict" }, | ||
| { id = "check-added-large-files", args = ["--maxkb=500"] }, | ||
| { id = "check-symlinks" }, | ||
| { id = "mixed-line-ending", args = ["--fix=lf"] }, | ||
| ] | ||
|
|
||
| # --- Rust: rustfmt --- | ||
| [[repos]] | ||
| repo = "local" | ||
|
|
||
| [[repos.hooks]] | ||
| id = "cargo-fmt" | ||
| name = "cargo fmt" | ||
| entry = "cargo fmt --all" | ||
| language = "system" | ||
| types = ["rust"] | ||
| pass_filenames = false | ||
|
|
||
| # --- Python: ruff (lint + format) --- | ||
| [[repos]] | ||
| repo = "https://github.com/astral-sh/ruff-pre-commit" | ||
| rev = "v0.11.4" | ||
| hooks = [ | ||
| { id = "ruff", args = ["--fix", "--select", "E,F,I,D", "--ignore", "D203,D213,E501"] }, | ||
| { id = "ruff-format" }, | ||
| ] | ||
|
|
||
| # --- Go: go vet --- | ||
| [[repos]] | ||
| repo = "local" | ||
|
|
||
| [[repos.hooks]] | ||
| id = "go-vet" | ||
| name = "go vet" | ||
| entry = "bash -c 'cd sdk/go && go vet ./...'" | ||
| language = "system" | ||
| files = "sdk/go/.*\\.go$" | ||
| pass_filenames = false | ||
|
|
||
| # --- Shell: shellcheck --- | ||
| [[repos]] | ||
| repo = "https://github.com/shellcheck-py/shellcheck-py" | ||
| rev = "v0.10.0.1" | ||
| hooks = [ | ||
| { id = "shellcheck" }, | ||
| ] | ||
|
|
||
| # --- Conventional commits (used by cliff.toml for changelog) --- | ||
| [[repos]] | ||
| repo = "https://github.com/compilerla/conventional-pre-commit" | ||
| rev = "v4.1.0" | ||
| hooks = [ | ||
| { id = "conventional-pre-commit", stages = ["commit-msg"] }, | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,13 @@ | ||
| # SPDX-FileCopyrightText: © 2024 Phala Network <[email protected]> | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| """Monitor certificate transparency logs for a given domain.""" | ||
|
|
||
| import argparse | ||
| import sys | ||
| import time | ||
|
|
||
| import requests | ||
| import argparse | ||
| from cryptography import x509 | ||
| from cryptography.hazmat.backends import default_backend | ||
| from cryptography.hazmat.primitives import serialization | ||
|
|
@@ -14,38 +16,50 @@ | |
|
|
||
|
|
||
| class PoisonedLog(Exception): | ||
| """Indicate a poisoned certificate transparency log entry.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| class Monitor: | ||
| """Monitor certificate transparency logs for a domain.""" | ||
|
|
||
| def __init__(self, domain: str): | ||
| """Initialize the monitor with a validated domain.""" | ||
| if not self.validate_domain(domain): | ||
| raise ValueError("Invalid domain name") | ||
| self.domain = domain | ||
| self.last_checked = None | ||
|
|
||
| def get_logs(self, count: int = 100): | ||
| """Fetch recent certificate transparency log entries.""" | ||
| url = f"{BASE_URL}/?q={self.domain}&output=json&limit={count}" | ||
| response = requests.get(url) | ||
| return response.json() | ||
|
|
||
| def check_one_log(self, log: object): | ||
| """Fetch and inspect a single certificate log entry.""" | ||
| log_id = log["id"] | ||
| cert_url = f"{BASE_URL}/?d={log_id}" | ||
| cert_data = requests.get(cert_url).text | ||
| # Extract PEM-encoded certificate | ||
| import re | ||
| pem_match = re.search(r'-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----', cert_data, re.DOTALL) | ||
|
|
||
| pem_match = re.search( | ||
| r"-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----", | ||
| cert_data, | ||
| re.DOTALL, | ||
| ) | ||
| if pem_match: | ||
| pem_cert = pem_match.group(0) | ||
|
|
||
| # Parse PEM certificate | ||
| cert = x509.load_pem_x509_certificate(pem_cert.encode(), default_backend()) | ||
| # Extract the public key | ||
| public_key = cert.public_key() | ||
| pem_public_key = public_key.public_bytes( | ||
| encoding=serialization.Encoding.PEM, | ||
| format=serialization.PublicFormat.SubjectPublicKeyInfo | ||
| format=serialization.PublicFormat.SubjectPublicKeyInfo, | ||
| ) | ||
| print("Public Key:") | ||
| print(pem_public_key.hex()) | ||
|
|
@@ -62,18 +76,20 @@ def check_one_log(self, log: object): | |
| print("No valid certificate found in the response.") | ||
|
|
||
| def check_new_logs(self): | ||
| """Check for new log entries since the last check.""" | ||
| logs = self.get_logs(count=10000) | ||
| print("num logs", len(logs)) | ||
| for log in logs: | ||
| print(f"log id={log["id"]}") | ||
| print(f"log id={log['id']}") | ||
| if log["id"] <= (self.last_checked or 0): | ||
| break | ||
| self.check_one_log(log) | ||
| print('next') | ||
| print("next") | ||
| if len(logs) > 0: | ||
| self.last_checked = logs[0]["id"] | ||
|
|
||
| def run(self): | ||
| """Run the monitor loop indefinitely.""" | ||
| print(f"Monitoring {self.domain}...") | ||
| while True: | ||
| try: | ||
|
|
@@ -87,12 +103,12 @@ def run(self): | |
|
|
||
| @staticmethod | ||
| def validate_domain(domain: str): | ||
| # ensure domain is a valid DNS domain | ||
| """Validate that the given string is a well-formed DNS domain name.""" | ||
| import re | ||
|
|
||
| # Regular expression for validating domain names | ||
| domain_regex = re.compile( | ||
| r'^(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,}$' | ||
| r"^(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,}$" | ||
| ) | ||
|
|
||
| if not domain_regex.match(domain): | ||
|
|
@@ -102,7 +118,10 @@ def validate_domain(domain: str): | |
|
|
||
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser(description="Monitor certificate transparency logs") | ||
| """Parse arguments and start the certificate transparency monitor.""" | ||
| parser = argparse.ArgumentParser( | ||
| description="Monitor certificate transparency logs" | ||
| ) | ||
| parser.add_argument("-d", "--domain", help="The domain to monitor") | ||
| args = parser.parse_args() | ||
| monitor = Monitor(args.domain) | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.