Overview
User Feedback
Star ratings and optional comments submitted from the chat assistant (most recent first).
Retrieval Quality
How well hybrid search is finding relevant documents — retrieval calls are logged automatically and this data feeds future knowledge-gap detection.
| Date | Queries | Zero-result | Citation rate | Avg top score | Avg feedback |
|---|---|---|---|---|---|
| Loading… | |||||
| Query | Category | Occurrences | Avg results | Avg top score | Last seen |
|---|---|---|---|---|---|
| Loading… | |||||
Filterable by advice category and advisor cohort/team — never by named individual advisor (that breakdown doesn't exist on this dashboard).
Knowledge Gap Report
Weekly-ish snapshot of retrieval quality gaps: low-relevance queries, recurring low-feedback topics, stale KB pages, and sessions escalated to a human advisor. Generated on demand here, or by an external scheduled job hitting the same endpoint.
| Query | Category | Occurrences | Avg top score | Last seen |
|---|---|---|---|---|
| No report generated yet. | ||||
| Category | Occurrences | Avg rating |
|---|---|---|
| No report generated yet. | ||
| URL | Oldest verified | Chunk count |
|---|---|---|
| No report generated yet. | ||
| Query | Reason | When |
|---|---|---|
| No report generated yet. | ||
Multi-Issue Cases
% of finalized cases tagged with more than one advice category, and the most common category pairings.
| Category A | Category B | Cases |
|---|---|---|
| Loading… | ||
Follow-up / Re-open Rate
% of cases marked "Resolved" that have a new case for the same client within the window below. Client matching is by exact client reference text (the same convention Client History already uses) — typo or casing variants of the same client won't merge.
Users & Roles
Not connected to a real Casebook API yet — the submission client uses a
placeholder endpoint (see docs/4.4-casebook-integration-DESIGN-PROPOSAL.md). This toggle
controls who can attempt submission once it's wired up. "Default" defers to the global
CASEBOOK_INTEGRATION_ENABLED setting; Enabled/Disabled explicitly override it for
that advisor (e.g. a pilot group). The manual "Copy for Casebook" fallback always works for
every advisor regardless of this setting. Team is an optional cohort label
(6.2) used only to filter the feedback-rating distribution below by team — it's never
shown broken down by individual advisor anywhere on this dashboard.
Case Note Access (6.8) controls View Saved Case Notes visibility: "Own
cases only" (default) restricts an advisor to cases they created; "All cases" grants full
visibility, same as admin/manager. Read-only — it never grants edit or delete rights on
another advisor's case. Every change here is logged (who changed it, for whom, when), and so
is every time an "All cases" advisor actually opens a case they didn't create. This has no
effect on Client History, which stays open to every advisor regardless.
| Role | Casebook access | Team | Case Note Access | |
|---|---|---|---|---|
| Loading… | ||||
Few-Shot Example Review
When an advisor rates a response poorly and provides a correction, it lands here as a candidate. Approving one adds it to that domain's system prompt as a "learn from this" example — no code change or deploy needed. Nothing here is used until you approve it.
Local Services Referral KB
Manually-curated local organisations (food banks, legal aid, mental health, housing charities, etc.) that advisors can be shown as referral suggestions after generating case notes. Higher priority shows first within a domain. Deactivated services are kept (never deleted) so past referral records stay meaningful.
| Name | Category | Also applies to | Priority | Active | Contact | |
|---|---|---|---|---|---|---|
| Loading… | ||||||
System Health
AI response latency, LLM token cost, and knowledge base freshness. Latency/cost figures only reflect activity since this tracking shipped — they'll look sparse until enough data accumulates.
Proxy metric: a session left "active" for several hours is presumed abandoned/failed — an advisor who simply forgot to click "stop" would also count here.
| Retired (dead-link) URL |
|---|
| Loading… |
Sitemap Management
Configure the sitemaps that will be discovered and scraped when knowledge base sync runs.
Knowledge Base
Refresh the knowledge base by re-scraping all configured URLs and updating the vector database. This process runs in the background and may take several minutes to complete.
Quickly scrape and index a single specific URL without running the full knowledge base sync.
Settings
Read-only reference — no new controls here, just what the other sections' controls do.
Refresh Knowledge Base re-scrapes every configured sitemap URL and updates the vector database incrementally. Reset & Resync Everything clears the existing index first — use only when the knowledge base needs a genuine full rebuild. Scrape Specific URL indexes a single page without running the full sync, useful right after a single source page changes.
The global CASEBOOK_INTEGRATION_ENABLED setting is the default for every
advisor; per-advisor overrides and team labels are managed under
Users & Roles, not here. The manual "Copy for Casebook" fallback
always works regardless of this setting.