Overview

Loading

User Feedback

User feedback

Star ratings and optional comments submitted from the chat assistant (most recent first).

Retrieval Quality

Retrieval Quality

How well hybrid search is finding relevant documents — retrieval calls are logged automatically and this data feeds future knowledge-gap detection.

Total queries
Daily trend
DateQueriesZero-resultCitation rate Avg top scoreAvg feedback
Loading…
Most common zero-result / low-score queries
QueryCategoryOccurrencesAvg results Avg top scoreLast seen
Loading…
Feedback rating distribution

Filterable by advice category and advisor cohort/team — never by named individual advisor (that breakdown doesn't exist on this dashboard).

Knowledge Gap Report

Knowledge Gap Report

Weekly-ish snapshot of retrieval quality gaps: low-relevance queries, recurring low-feedback topics, stale KB pages, and sessions escalated to a human advisor. Generated on demand here, or by an external scheduled job hitting the same endpoint.

Low-relevance queries
QueryCategoryOccurrencesAvg top scoreLast seen
No report generated yet.
Recurring low-feedback topics
CategoryOccurrencesAvg rating
No report generated yet.
Stale KB pages
URLOldest verifiedChunk count
No report generated yet.
Escalated to a human advisor
QueryReasonWhen
No report generated yet.

Multi-Issue Cases

Multi-Issue Cases

% of finalized cases tagged with more than one advice category, and the most common category pairings.

Loading
Most common category pairings
Category ACategory BCases
Loading…

Follow-up / Re-open Rate

Follow-up / Re-open Rate

% of cases marked "Resolved" that have a new case for the same client within the window below. Client matching is by exact client reference text (the same convention Client History already uses) — typo or casing variants of the same client won't merge.

Loading

Users & Roles

Users & Roles (Casebook access per advisor)

Not connected to a real Casebook API yet — the submission client uses a placeholder endpoint (see docs/4.4-casebook-integration-DESIGN-PROPOSAL.md). This toggle controls who can attempt submission once it's wired up. "Default" defers to the global CASEBOOK_INTEGRATION_ENABLED setting; Enabled/Disabled explicitly override it for that advisor (e.g. a pilot group). The manual "Copy for Casebook" fallback always works for every advisor regardless of this setting. Team is an optional cohort label (6.2) used only to filter the feedback-rating distribution below by team — it's never shown broken down by individual advisor anywhere on this dashboard. Case Note Access (6.8) controls View Saved Case Notes visibility: "Own cases only" (default) restricts an advisor to cases they created; "All cases" grants full visibility, same as admin/manager. Read-only — it never grants edit or delete rights on another advisor's case. Every change here is logged (who changed it, for whom, when), and so is every time an "All cases" advisor actually opens a case they didn't create. This has no effect on Client History, which stays open to every advisor regardless.

EmailRoleCasebook accessTeamCase Note Access
Loading…

Few-Shot Example Review

Few-Shot Example Review

When an advisor rates a response poorly and provides a correction, it lands here as a candidate. Approving one adds it to that domain's system prompt as a "learn from this" example — no code change or deploy needed. Nothing here is used until you approve it.

Local Services Referral KB

Local Services Referral KB

Manually-curated local organisations (food banks, legal aid, mental health, housing charities, etc.) that advisors can be shown as referral suggestions after generating case notes. Higher priority shows first within a domain. Deactivated services are kept (never deleted) so past referral records stay meaningful.

NameCategoryAlso applies toPriorityActiveContact
Loading…

System Health

System Health

AI response latency, LLM token cost, and knowledge base freshness. Latency/cost figures only reflect activity since this tracking shipped — they'll look sparse until enough data accumulates.

Latency (30d)
Loading
Session success rate (30d)
Loading

Proxy metric: a session left "active" for several hours is presumed abandoned/failed — an advisor who simply forgot to click "stop" would also count here.

LLM cost (30d)
Loading
Knowledge base freshness
Loading
Retired (dead-link) URL
Loading…

Sitemap Management

Sitemap Management

Configure the sitemaps that will be discovered and scraped when knowledge base sync runs.

Loading...

Knowledge Base

Knowledge Base Sync

Refresh the knowledge base by re-scraping all configured URLs and updating the vector database. This process runs in the background and may take several minutes to complete.

Starting... 0%
Discovered: 0 Success: 0 Failed: 0

Scrape Specific URL

Quickly scrape and index a single specific URL without running the full knowledge base sync.

Settings

About this dashboard's controls

Read-only reference — no new controls here, just what the other sections' controls do.

Knowledge Base pipeline

Refresh Knowledge Base re-scrapes every configured sitemap URL and updates the vector database incrementally. Reset & Resync Everything clears the existing index first — use only when the knowledge base needs a genuine full rebuild. Scrape Specific URL indexes a single page without running the full sync, useful right after a single source page changes.

Casebook integration

The global CASEBOOK_INTEGRATION_ENABLED setting is the default for every advisor; per-advisor overrides and team labels are managed under Users & Roles, not here. The manual "Copy for Casebook" fallback always works regardless of this setting.