You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TypeSafe's official SDK (@typesafe-ai/sdk, latest 0.6.0) calls the Jev evaluation model with TypeSafeClient.systemOne({ state, questions, model? }). Users who call Jev directly (not through Vercel AI experimental_evaluate, see #24692) get no span today, only a plain http.client span from fetch.
Scope
Add a new AI integration in packages/server-utils/: logic in src/ai/typesafe/, integration in src/integrations/typesafe.ts, registered in getTracingIntegrations(). Re-export it from the runtime packages.
Instrument TypeSafeClient.prototype.systemOne. Make one span for each call, not one span for each retry.
models.list() is out of scope.
Target span
Use the same shape as the Vercel AI evaluate span in #24692, so that both paths look the same in the product.
Name: evaluate <model>, op gen_ai.evaluate, origin auto.ai.typesafe
TypeSafe's official SDK (
@typesafe-ai/sdk, latest0.6.0) calls the Jev evaluation model withTypeSafeClient.systemOne({ state, questions, model? }). Users who call Jev directly (not through Vercel AIexperimental_evaluate, see #24692) get no span today, only a plainhttp.clientspan fromfetch.Scope
packages/server-utils/: logic insrc/ai/typesafe/, integration insrc/integrations/typesafe.ts, registered ingetTracingIntegrations(). Re-export it from the runtime packages.TypeSafeClient.prototype.systemOne. Make one span for each call, not one span for each retry.models.list()is out of scope.Target span
Use the same shape as the Vercel AI evaluate span in #24692, so that both paths look the same in the product.
evaluate <model>, opgen_ai.evaluate, originauto.ai.typesafegen_ai.operation.name: evaluate,gen_ai.provider.name: typesafegen_ai.request.model(themodeloption, or the client default,jev-latest),gen_ai.response.model(result.model, for examplejev-1.13.0)gen_ai.usage.input_tokens,output_tokens,total_tokens(fromresult.usage.input_tokens/output_tokens)gen_ai.input.messages: [{ type: 'evaluation', state, questions }](only whenrecordInputsis on)gen_ai.output.messages: [{ type: 'evaluation', answers }](only whenrecordOutputsis on)errorwhen the call throws (TypeSafeErrorand API errors).Notes
fetchtoPOST /v1/systemone. It has no tracing channel, so we must patch the client.@sentry/conventionshas nogen_ai.evaluateop yet (same gap as Instrument Vercel AIexperimental_evaluate#24692).