ci(nightly): fetch API keys from Key Vault via OIDC#120
Open
miguelgfierro wants to merge 9 commits intomainfrom
Open
ci(nightly): fetch API keys from Key Vault via OIDC#120miguelgfierro wants to merge 9 commits intomainfrom
miguelgfierro wants to merge 9 commits intomainfrom
Conversation
Replace GitHub repo secrets (ANTHROPIC_API_KEY, EMBEDDING_BINDING_HOST, EMBEDDING_BINDING_API_KEY) with Azure Key Vault lookups. Add OIDC login and kv-firefly-signature fetch to both the test and benchmark jobs. Move from a single top-level permissions block to per-job permissions so the benchmark job no longer inherits issues:write from the test job.
Separate the single shared service principal into two: - AZURE_CI_CLIENT_ID (nightly/benchmark): Key Vault Secrets User only - AZURE_DEPLOY_CLIENT_ID (deploy-mcp): AcrPush + Container App write only Also hardcode tenant and subscription IDs as plain values — they are not credentials and do not belong in the secrets store.
Contributor
|
Nightly does not work with this branch https://github.com/fireflyframework/fireflyframework-agentic/actions/runs/25543811013 Service principal error |
The 25 870-row CSV converted to 24 399 chunks via MarkdownChunker (one flat table, no heading boundaries). Embedding and storing that many vectors flooded the SQLite WAL and corrupted the database, causing "database disk image is malformed" on the xlsx ingest that follows. None of the 32 benchmark queries ask about billing data; the CSV is purely a distractor. 200 rows (190 chunks) is enough to test that billing-ledger content is not surfaced for company-fact queries, without overwhelming the in-memory SQLite corpus.
paraphrase_ceo_idiomatic now returns the correct doc at rank 1 but a different chunk (distractor docs shifted relative vector scores). The answer is still correct; only substring_match drops to 27/28 = 0.9643.
shutil.copyfile on a WAL-mode SQLite database copies only the main file, leaving committed WAL frames behind. When two DatabaseStore instances share the same backend path (e.g. SqliteCorpus + SqliteVecVectorStore both pointing to corpus.sqlite), each upload produces an etag change that triggers the other store to re-download. The download overwrites the shared cache and stales the WAL, wiping all data written in that round. Fix: detect SQLite files by their 16-byte magic header and use sqlite3.Connection.backup() instead of shutil.copyfile. The backup API walks both the main file and any unflushed WAL frames, so the uploaded file is always a complete, consistent snapshot.
sqlite3.Connection.backup() opens the source without loading the sqlite_vec extension, which makes schema parsing fail when the database has a vec0 virtual table — producing a malformed copy. The correct fix is to issue PRAGMA wal_checkpoint(FULL) on the existing connection (which already has sqlite_vec loaded) immediately after each COMMIT, so the main database file is fully up-to-date before shutil.copyfile captures it for upload. Reverts the _copy_for_upload / backup() helper added in the previous attempt and restores plain shutil.copyfile in LocalBackend._upload_sync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ANTHROPIC_API_KEY,EMBEDDING_BINDING_HOST, andEMBEDDING_BINDING_API_KEYas GitHub repo secrets innightly.ymlkv-firefly-signature) after OIDC login, with::add-mask::applied before writing to$GITHUB_ENVpermissionsblocks —benchmarkno longer inheritsissues:writefrom the test jobInfrastructure required (see PR description)
Before this workflow runs successfully, the following must be in place:
Key Vault secrets (in
kv-firefly-signature):ANTHROPIC-API-KEYEMBEDDING-BINDING-HOSTEMBEDDING-BINDING-API-KEYOIDC app registration (
firefly-github-actions):repo:fireflyframework/fireflyframework-agentic:ref:refs/heads/mainKey Vault Secrets Useron the KV,AcrPushonfireflysignature,Contributoronfirefly-mcpGitHub repository secrets (Actions → Secrets):
AZURE_CLIENT_ID→ client ID offirefly-github-actionsAZURE_TENANT_ID→cda419af-f2d8-444c-9248-4ada4f168c8aAZURE_SUBSCRIPTION_ID→e8b8063e-f842-4a59-9754-427ddb7bfb63Delete old GitHub secrets once the workflow is confirmed working:
ANTHROPIC_API_KEY,EMBEDDING_BINDING_HOST,EMBEDDING_BINDING_API_KEY