Metrics and Session Reports

Two related but distinct signals come out of a session: live metrics and the end-of-session report. metrics_update events give you a periodic, in-session snapshot for a HUD or overlay. The end-of-session report arrives as an artifact event with kind: "report" — a single structured summary (scores, coaching tips, and the final metric snapshot), emitted once, right before the session closes. Both are driven by the same server-side metrics config; the report additionally depends on final_report.

This is the developer-facing counterpart to Session Reports, which explains what the end user sees. This article covers the wire payloads, the config that drives them, and the SDK APIs that consume them.

The `metrics` Config

Live metrics and the report's final snapshot are both populated from the same metrics config, schema_version: 1. It is not something the SDK sends per-event — it's baked into LiveKit room metadata at session-token time (either from the client's session request, or — for a coaching preset — loaded server-side from the trusted preset row) and read once by the agent at session start.

Shape (server-side, validated by adminLiveKitToken):

{
  "schema_version": 1,
  "emit_interval_seconds": 5,
  "requires_provider": ["gemini-live"],
  "definitions": [
    { "id": "audio.pace", "extractor": "audio.words_per_minute", "format": { "display": "{value} wpm" }, "trend": { "enabled": true } }
  ]
}

emit_interval_seconds — clamped server-side to 2–30 seconds; how often the agent computes and emits a metrics_update snapshot.
requires_provider — optional list of realtime providers the config depends on (e.g. verbatim-STT filler/word-timing metrics only exist on cascade).
definitions — up to 20 metric definitions, each with an id (the key the client sees in the snapshot) and an extractor that must appear on the server's allow-list.

Extractor Families

Each extractor value belongs to one of three families:

Family	Extractors	Source
`vision.*`	`face_present_ratio`, `gaze_to_camera_ratio`, `gaze_on_camera_ratio`, `posture_score`, `hip_shoulder_separation`, `arm_slot`, `stride_ratio`, `rep_count`, `range_of_motion`, `symmetry`, `rep_tempo`, `trunk_flexion`	Pose keypoints from the video CV processor (`yolo-pose`, etc.)
`audio.*`	`words_per_minute`, `filler_count`, `silence_ratio`, `vocal_authority_index`, `hedge_count`, `pause_balance`	Transcript window analysis (pace, fillers, silence, vocal authority)
`behavior.*`	`questions_asked`, `tips_delivered`, `steps_completed`	Orchestrator/session-level counters

vision.* extractors require device_mode: "video" with a CV processor active; requesting them on a voice-only session yields no data for that id (it simply never populates in the snapshot).

The Extractor Allow-List Applies Only to Client-Supplied Metrics

adminLiveKitToken validates every client-submitted metrics blob against a server-side ALLOWED_EXTRACTORS set (picoagentfunctions/functions/adminLiveKitToken/index.mjs) — any definition whose extractor isn't on that list is silently dropped. This validation is bypassed entirely for coaching presets: when a session is started with a presetId, the agent-config row's stored metrics value is used verbatim, with no allow-list filtering. This is intentional — presets are trusted, curated server-side rows, not arbitrary client input — but it means you cannot infer a session's live metric ids purely from the public extractor list if it's running a preset; inspect the preset's catalog entry instead.

Live Metrics — `metrics_update`

Once metrics are configured, the agent emits a metrics_update event on the emit_interval_seconds cadence for the life of the session.

Exact payload (from AgentEvent in packages/embed-core/src/types.ts):

{
  type: "metrics_update";
  timestamp: string;             // ISO8601 — when the agent computed the snapshot
  sessionElapsedSeconds: number; // seconds since the agent's session start
  snapshot: MetricsSnapshot;     // Record<string, MetricSnapshotEntry>
}

MetricsSnapshot is a plain object keyed by the metric id from the metrics config (not the extractor name). Each value is a MetricSnapshotEntry:

interface MetricSnapshotEntry {
  value: number;      // raw numeric value, post-rounding by the agent
  formatted: string;  // already-rendered per the metric's format.display template
  trend?: number[];   // historical values, oldest first — present only if trend.enabled
}

trend is entirely absent from the entry (not an empty array) when the definition didn't set trend: { enabled: true } — check for the key, don't assume it exists.

Live Emission Is Suppressed in `post_session` Mode

If the session's feedback_timing is "post_session" (via feedbackTiming on CreateSessionOptions, or baked into a preset), the agent still computes metrics internally but does not emit metrics_update events during the session — only the final snapshot at the end, inside the report artifact's finalSnapshot. Don't build a live HUD that assumes metrics_update will fire for every session; check feedback_timing (or simply handle the case where no live events ever arrive) before wiring up a persistent metrics panel.

Consuming Live Metrics

Raw event:

session.on("metrics_update", (e) => {
  // e.timestamp, e.sessionElapsedSeconds, e.snapshot
  updateMetricsHud(e.snapshot);
});

React (@atanzoai/embed-react):

import { useMetrics } from "@atanzoai/embed-react";

const metrics = useMetrics(session); // MetricsSnapshot, {} until first update
// metrics["audio.pace"]?.value
// metrics["audio.pace"]?.formatted   // e.g. "142 wpm"
// metrics["audio.pace"]?.trend       // number[] | undefined

useMetrics returns {} until the first snapshot arrives, and forever on sessions where live emission never fires (post_session timing, or an agent/protocol version that predates metrics_update). Guard on Object.keys(metrics).length before rendering a HUD.

End-of-Session Report — the `artifact` event

final_report (also schema_version: 1, baked into room metadata the same way as metrics) configures the LLM scorer that runs once the session ends. Its shape, per validateFinalReportConfig in adminLiveKitToken:

{
  "schema_version": 1,
  "summary": null,
  "criteria": [],
  "include_metric_snapshot": true,
  "include_tip_history": true,
  "include_question_log": false,
  "max_transcript_chars": 12000
}

criteria (up to 10 entries) defines the scored dimensions the LLM rates — these become the scores array in the report. When the scorer completes, the agent transitions status to "scoring", then emits an artifact event carrying the report, followed by session_ended.

The report is not a standalone event — it is a generic Artifact (the same channel every durable output flows through: documents, generated images, etc.) with kind: "report":

{
  type: "artifact";
  runId: string;
  artifact: {
    artifactId: string;
    version: number;
    kind: "report";
    state: "ready";
    title: string;
    format: "json";
    createdAt: string;             // ISO8601
    data: SessionReport;           // the structured report — see below
  };
}

interface SessionReport {
  runId: string;
  generatedAt: string;          // ISO8601
  summary: string | null;
  scores: SessionScore[];
  coachingTips: string[];       // ordered, most impactful first
  finalSnapshot: MetricsSnapshot;
  schemaVersion: 1;
}

interface SessionScore {
  id: string;
  label: string;
  value: number;
  scaleType: "percent" | "score_5" | "score_10";
  rationale: string;
}

Notes on fields:

summary is nullable — a report can have scores/tips with no narrative summary if final_report.summary wasn't configured to request one.
scores — one entry per configured criteria item; check scaleType before rendering value (a score_5 of 4 and a percent of 4 mean very different things).
coachingTips is always an array (possibly empty), pre-ordered by the agent — don't re-sort.
finalSnapshot is the same MetricsSnapshot shape as metrics_update.snapshot — it's the metrics state at session end, present whenever include_metric_snapshot was left enabled (the default).
schemaVersion is a TypeScript literal 1 — useful as a discriminant if you ever branch on report schema.

Consuming the Report

Raw event:

session.on("artifact", (e) => {
  if (e.artifact.kind === "report") {
    // e.artifact.data.summary, .scores, .coachingTips, .finalSnapshot
    renderReportScreen(e.artifact.data);
  }
});

React:

import { useSessionReport } from "@atanzoai/embed-react";

const report = useSessionReport(session); // SessionReport | null

{report && (
  <div className="session-report">
    <p>{report.summary}</p>
    <ul>
      {report.scores.map((s) => (
        <li key={s.id}>{s.label}: {s.value} ({s.scaleType}) — {s.rationale}</li>
      ))}
    </ul>
    <ol>
      {report.coachingTips.map((tip, i) => <li key={i}>{tip}</li>)}
    </ol>
  </div>
)}

useSessionReport returns null until the report artifact arrives (it fires once, just before session_ended) and holds only the live-session report — it does not persist across page reloads or re-fetch on remount. For retrieving a report after the WebRTC session has closed (e.g. a manager reviewing a past run in the console), call your backend's session-report endpoint rather than relying on this hook. For every durable output a run produced (not just the report), use the more general useArtifacts(session) hook.

What the End User Sees

The console and embed UIs render the report artifact as the "Performance Snapshot / Coach's Notes / Scores" report screen described in Session Reports — finalSnapshot maps to the performance snapshot, summary to the coach's notes, scores to the scores section. If you're building a custom report UI, that article is a useful reference for what fields users expect to see and how they're typically grouped.

Session Reports — the end-user-facing explanation of what a report contains and where to find it in the console
Coaching Presets — how presetId sessions get their metrics/final_report config from a trusted server-side row, bypassing the client extractor allow-list
Using the React SDK — full hook reference including useMetrics and useSessionReport
Plan State, Glanceable, Knowledge, Credits — the other session-state events and vanilla-JS subscription pattern these hooks follow