Analytics Annotations for AI Assistant Launches

Dashboards need context. Annotate major events so stakeholders stop guessing.

Events to annotate

• Prompt version release
• Crawl run completion or failure
• Adaptive threshold change
• New locale or embed launch
• Billing grace period start/end
• Incident response or kill switch
• SSO enablement for a tenant

Implementation

• Store annotations in a simple table with timestamp, event type, tenant, message, and links.
• Expose an API/UI for ops to add notes manually.
• Auto-generate annotations from audit logs (prompt update, crawl run) so no one forgets.

Dashboard integration

• Display annotations as markers on timeseries charts (e.g., containment, impressions).
• Filter by tenant or embed to keep noise low.
• Include annotation feed alongside metrics for quick scanning.

Ops workflow

• Require annotations for every prompt change or crawl run.
• Review them during weekly ops meetings; ensure corrective actions are linked.
• Export annotations with analytics when tenants request raw data.

CrawlBot example

CrawlBot’s analytics service attaches prompt_version, crawl_run_id, and fallback_reason to every event, making annotations automatic. Copy this approach to keep stakeholders aligned.***