Add Slack community pulse workflow - #23
Conversation
pvams
left a comment
There was a problem hiding this comment.
Reviewed at 47bb27b against its actual base agent/add-slack-post-to-x (f52ee32, #22) — one commit, 16 new files, all under starter/slack-community-pulse/. All four checks in the description pass: bun install --frozen-lockfile, bun run typecheck, bun build, bun run start --help.
The design is clean — two analysts fanning out to their own read-only tool, a reporter merging them, tool-call enforcement in the step invoker, untrusted-text framing in the prompts. The problems are in the X collection layer.
1. Truncation silently inverts the week-over-week comparison — blocking
src/x-client.ts:129 issues one 14-day request spanning both periods, capped at 2x100 posts. X timelines return newest-first, so when the cap hits, what gets dropped is the oldest data — the previous period. The comparison the whole workflow exists to produce is the thing that breaks first.
Ran it against a stub returning 200 posts inside the current week:
current: 200 previous: 0 truncated: true
The reporter is prompted for "key week-over-week metrics" and hands Slack a report showing growth from nothing. Any account averaging >~14 posts/day (or >~14 mentions/day, which is a low bar for the mentions endpoint) hits this.
truncated is computed and shipped in the JSON, but grep confirms it is never referenced by either analyst's system prompt, nor forwarded to the reporter — nothing downstream can catch the condition. Either fetch each period with its own start_time/end_time request, or surface truncated in the prompts with an instruction to refuse the comparison when set.
2. A mistyped handle produces an unintelligible Slack error — blocking
X v2 returns HTTP 200 with an errors payload and no data when a username doesn't exist. src/x-client.ts:83 destructures { data } and calls toUser(data) unconditionally. Confirmed against a real not-found body, this is what reaches the user:
:warning: Weekly pulse failed
undefined is not an object (evaluating 'user.id')
Typing the wrong handle is the single most likely user error in this workflow. The response.ok check at src/x-client.ts:65 isn't a boundary — X puts failures in a 200 body. Parse the response before trusting its shape, and turn a missing user into X account @foo not found.
3. Vercel gateway path names the wrong missing variable
providerEnv (src/config.ts:82) is new to this starter — no sibling has it. It spreads env and then sets OPENAI_API_KEY: env.AI_GATEWAY_API_KEY, so an unset gateway key overwrites a real OPENAI_API_KEY with undefined. Confirmed:
INTX_PROVIDER=vercel, AI_GATEWAY_API_KEY unset, OPENAI_API_KEY=sk-real
-> "OPENAI_API_KEY is not set."
The variable it names is set; the one that's actually missing is never mentioned.
4. Vercel gateway defaults to an unroutable model
Same path, src/config.ts:90: with no AI_GATEWAY_MODEL, the model falls through to gpt-4o-mini and is sent to https://ai-gateway.vercel.sh/v1, which needs namespaced IDs (openai/gpt-4o-mini). Confirmed — config validation passes and it fails at the first inference call instead. Also, AI_GATEWAY_BASE_URL is read here but isn't in .env.example.
Minor
- Account cache never expires —
src/x-client.ts:57. The client is constructed once inresolveConfigand entries are only evicted on failure, so a worker running for weeks reports a follower count from its first run. The comment says the goal is deduping between the two parallel analysts; that only needs per-run scope. - X credentials validated with
.trim(), passed raw —src/config.ts:50vssrc/config.ts:69. Slack secrets are trimmed; X ones aren't. A key pasted with a trailing newline clears validation and yields an opaque 401. - Teaching comments dropped in the copied
source.ts. The siblingslack-post-to-x/src/source.tsopens with a 19-line explanation of provider resolution and documentsProviderSpec; this copy strips all of it and renames the type toProvider. Comment density across starters: 7 lines / 964 here, vs 31/1119 and 44/621 in siblings. In an examples repo the explanation is the deliverable. .env.example:30— "Use the provider configuration inherited from the stacked approval-flow" leaks PR-stack internals into a file the reader copies.
Checked and dismissed
- OAuth parameter sort uses
localeComparewhere the spec requires byte order. I enumerated the actual parameter set and both orderings are identical — latent, not live. bun.lockcommitted while gitignored, and the../slack-agent/vendor/corbits-tagworkspace pointer reaching into another example (which contradicts the root README's self-containment rule) — both matchslack-post-to-xandslack-approval-flowexactly. Pre-existing convention, not this PR's to fix.- No tests — there is no test file anywhere in the repo, so this matches convention.
Findings 1 and 2 are the ones I'd hold the merge on; both are in the collection layer and both are demonstrable. 3 and 4 are in genuinely new code and cheap to fix.
|
@pvams comments are resolved |
TheGreatAxios
left a comment
There was a problem hiding this comment.
Self-review (agent)
Verdict: Approve
What this PR does
Pure additive starter/slack-community-pulse: parallel Community Listener + Content Analyst collect read-only X evidence for adjacent seven-day windows, then a Reporter merges into one Slack-ready report. Reuses the pinned Corbits Tag workspace from the stack (no second gitlink).
What looks solid
- Workflow:
mentionsandcontentrun with noafter(parallel);reportafter both with object merge - X is structurally read-only: client only GETs username lookup, mentions, and user tweets; OAuth signature hardcodes
GET; no write endpoints - Shared Tag via
../slack-agent/vendor/corbits-tag/packages/*only - Diff is entirely under
starter/slack-community-pulse/
Findings
| Severity | Finding |
|---|---|
| minor | After review feedback, per-client username cache was removed — parallel analysts each re-resolve the same user (extra GETs; rate-limit pressure under load). |
| minor | XPostPage.meta is required; a response missing meta throws a raw TypeError instead of a clean X error. |
| minor | System prompts still say “filter … current week against previous week” while the client already partitions into current/previous — agents only need to analyze. |
| minor | Slack scope users:read.email is unused for this report-only flow. |
| nit | No unit tests for periods, username resolve, or OAuth param signing (acceptable for a starter). |
| nit | bun.lock committed while listed in .gitignore (same sibling pattern). |
Checks
- Incremental diff base…HEAD (16 files, +1576)
bun install --frozen-lockfile, typecheck, build,start --help(pass)- Grep confirmed no X write methods/endpoints in this starter
Residual risks
- OAuth app may still have write permissions; prefer a read-only X app in ops
- Reporter still sees model-generated analysis of untrusted X text (mitigated by framing; not eliminated)
- In-process session map lost on restart (documented)
Safe to merge once #22 lands. Squash-on-merge cleans stack merge commits.
Summary
Stack
Depends on corbitsdev/examples#22 and targets its head branch,
agent/add-slack-post-to-x.This branch is authored directly on top of PR #22's head commit,
f52ee32417ac8bf168c197a7f0c469a05f3ad67c.Verification
bun install --frozen-lockfilebun run typecheckbun build src/cli.ts --target=bun --outdir=tmp/buildbun run start --help