# Insights (IN)

Research date: 13 November 2026. Anything not confirmed in the code or on a linked source is marked [unverified].

## Decisions (owner delegated, my recommendation, 2026-11-14)

1. **Design approved.**
2. **"Resolved"** as defined under "Resolution rate", always shown split into confirmed and assumed.
3. **Attribution window: 7 days, last chat before the order**; reports say "orders after a chat", never "orders caused by the chat".
4. **Order of work:** IN-2 to IN-8 first; IN-10 (lead and meeting conversions) after the lead capture track (LD) is merged, since it records LD's leads and meetings.

## What exists today

- **Per turn, not per conversation.** `kimaai_chats` (`app/Models/ChatModel.php`) has one row per question and answer: `user_id`, `ip` (blank for visitors), `session_id`, `input` JSON (`prompt`, `postId`, `lang`, tool trace), `response` JSON (with `_kimaai_renderables`), `created_at`. No session table, no URL beyond `postId`, no referrer, no outcome, and no index besides the primary key, so date and session queries scan the table.
- **Session cookie** `kimaai_chatbot_session_id`, HttpOnly, SameSite Strict, 30 days (`app/Helper.php`): readable server-side at checkout or form submission.
- **Feedback analytics** (premium, `premium/app/ChatbotFeedback/FeedbackAnalyticsService.php`, `manage_options`): feedback volume, helpful and negative rate, CSAT average and distribution, helpful per day, CSAT per week, most downvoted answers, top pages by negative rate (from `postId`), message list with admin notes. UI in `src/admin/feedback/pages`; page shell in free `app/Admin/AdminInit.php`.
- **Knowledge-gap miner** (premium, `premium/app/Eval/KnowledgeGapMiner.php`): the last 300 visitor prompts in N days, every one embedded again on every run (`PinnedAnswers::embedText()`), greedy clustering at cosine 0.85 against the first member's vector (never updated), ranked by thumbs down then volume; returns count, negatives, sample, five variants. Only via `wp kimaai gaps` (`premium/app/Cli/GapsCommand.php`) and the MCP tool `KnowledgeGapsTool`; no labels, storage or admin page.
- **Outcome signals**: `kimaai_chatbot_ungrounded_answer` (premium `AnswerQuality.php`), `kimaai_chatbot_handoff` and option `kimaai_handoff_escalated` (`HumanHandoff.php`), CSAT per session, thumbs per answer, daily token usage (`app/Ai/UsageTracker.php`).
- **No conversion hooks**: nothing listens to WooCommerce orders, `wpcf7_mail_sent` or `gform_after_submission`.
- Action Scheduler is bundled and used for recurring jobs (`EvalScheduler`, `IndexHealth`, retention).

## Best practice in short

- **Intercom Fin**: a resolution is *confirmed* (the customer says it worked) or *assumed* (the customer leaves without asking for more); it is taken back if the customer returns to the conversation. Resolution rate is the share of conversations resolved without a person. No fixed idle window is published. [1] Its model-based CX Score (1 to 5) is shown per topic in Topics Explorer. [2, 3]
- **Kapa**: "Coverage Gaps" clusters the *uncertain* answers per week, month or quarter, counts only the first question of each thread, and adds a finding and a recommendation per cluster, with CSV export; "Top Questions" clusters all questions. [4, 5]
- **Chatbase**: topics computed once a day for the day before, first shown after a day with at least 30 conversations, up to five topics per conversation; resolution rate counts ended conversations only. [6]
- **Topic discovery**: Anthropic's Clio embeds a facet per conversation, clusters with k-means, has a model name each cluster, and enforces minimum cluster sizes for privacy; the OpenAI cookbook does the same with k-means and five samples per cluster. [7, 8]
- **WooCommerce order attribution** stores `_wc_order_attribution_*` meta (source type, UTM, session pages) on each order, sets its own cookies and respects the WP Consent API. The chat signal should sit beside it. [9, 10]

## Design

### Conversation facts

New table `kimaai_chat_sessions` (`ChatSessionModel`, free): `session_id` VARCHAR(64) unique, `first_at`, `last_at`, `turns`, `user_id`, `first_post_id`, `lang`, `outcome` VARCHAR(16) (`open`, `resolved_confirmed`, `resolved_assumed`, `unresolved`, `escalated`), `outcome_reasons` VARCHAR(255), `lead_id` NULL (track LD), `updated_at`. Filled on `kimaai_chat_response_saved` with one upsert per turn. The same migration adds `KEY (session_id)` and `KEY (created_at)` to `kimaai_chats` with `Db::addIndexIfMissing()`, and backfills sessions from existing chats in chunks within the migration time budget (pattern in `VersionYYYYMMDDNN.php`).

### Resolution rate (free)

An hourly Action Scheduler job closes sessions idle for 30 minutes (setting) and sets the outcome:

- `escalated`: handoff fired, or the lead step was shown as "before handoff".
- `unresolved`: the last answer has a thumbs down, CSAT 1 or 2, or the last answer was ungrounded.
- `resolved_confirmed`: CSAT 4 or 5, or a thumbs up on the last answer with no thumbs down after it.
- `resolved_assumed`: none of the above and at least one answer.

Resolution rate = confirmed plus assumed over closed sessions, always shown split. "Deflection" is labelled "answered without a person", because KimaAI has no ticket system to deflect from. Overview tiles: conversations, messages per conversation, resolution split, escalations, top pages by conversations.

### Topic clusters (premium, needs embeddings)

- Tables `kimaai_topics` (`id`, `label`, `description`, `centroid` LONGTEXT, `size`, `embedding_env`, `merged_into`, timestamps) and `kimaai_question_topics` (`chat_id` primary key, `topic_id`, `similarity`, `created_at`).
- An async job per new turn (batched every 15 minutes) embeds the prompt once, assigns it to the nearest topic at the threshold or opens a new one, and updates the centroid as a running mean. Vectors of questions are not stored, only centroids, so the database stays small. Clio and the cookbook use batch k-means [7, 8], which needs every vector kept; MySQL on a shared host cannot hold that cheaply.
- A weekly job merges topics whose centroids are close, and asks the chatbot's model to name topics with at least five members from ten sample questions. Emails, phone numbers and long digit runs are masked before the call. When the embedding environment changes, topics are rebuilt.
- Per topic: volume and change, resolution split, thumbs down, escalation and ungrounded rates, conversions.

### Content-gap report (premium)

Built on the topic store, so the miner no longer embeds 300 prompts per run. Like Kapa [4], it can count only the first question of each conversation (setting), which keeps follow-ups from inflating a topic. Gap score = volume times (share unresolved + share thumbs down + share ungrounded). Each row: topic label, sample questions, why it is flagged, the closest existing page from the vector index, and actions that already exist: create a pinned answer (`PinnedAnswers`), add an eval case (`EvalRunner`), open a new draft post with the topic as title. `KnowledgeGapMiner::report()` reads the store when it is filled and keeps today's behaviour as a fallback, so `wp kimaai gaps` and the MCP tool stay compatible.

### Conversion attribution (premium, integrations)

- Table `kimaai_conversions`: `id`, `session_id`, `type` (`order`, `form`, `lead`, `meeting`), `ref_id`, `value` DECIMAL(18,4), `currency`, `status`, `assist` (`chat`, `product_shown`), `created_at`; `KEY (session_id)`, `KEY (type, created_at)`.
- WooCommerce: on `woocommerce_checkout_order_processed` (classic checkout) and `woocommerce_store_api_checkout_order_processed` (block checkout, before payment; we call `$order->save()` ourselves) [11], read the session cookie (not `woocommerce_update_order`, which fires several times in block checkout); if that session had a visitor message within the window, store `_kimaai_session_id` with `$order->update_meta_data()` (HPOS safe) and insert a conversion. Value counts once the order reaches processing or completed (`woocommerce_order_status_changed`). `assist = product_shown` when the order contains a product the chat showed in that session (from `_kimaai_renderables`). Show WooCommerce's own attribution source next to it.
- Forms: `wpcf7_mail_sent` and `gform_after_submission` read the cookie the same way (CF7 and Gravity Forms submit same-site, so the SameSite Strict cookie is sent [unverified for Gravity Forms AJAX in iframes]).
- Leads and booked meetings from track LD are written as conversions directly.
- Report wording says "orders after a chat", never "orders caused by the chat".

### Admin and settings

"Feedback Analytics" becomes "Insights": tabs Overview, Topics, Content gaps, Conversions, Feedback (today's pages), built from `KpiTiles` and `ChartCard` on the design system primitives; REST `/kimaai/v1/insights/*`, `manage_options`, date ranges as in `FeedbackAnalytics.php`; CSV per tab. Section `insights` in `app/Admin/config/chatbot.php`, same schema style: `insightsSessionIdleMinutes` (30), `topicsEnabled` (premium, `onlyPremium`), `topicsMinSize` (5), `attributionEnabled` (premium, off by default), `attributionWindowDays` (7), `highIntentPages` (shared with LD lead score).

### Privacy

- No new personal data except the link between a session and an order or form entry. That link is included in the LD exporter and eraser (orders found by billing email, then their `_kimaai_session_id`).
- Attribution is off by default and described in the suggested privacy policy text. When the WP Consent API is active, attribution runs only with `wp_has_consent('statistics')`, as WooCommerce does for its own attribution [10]; whether reading the session cookie for analytics needs consent under ePrivacy otherwise is a question for the owner's counsel [unverified].
- Exporter and eraser through the WordPress privacy filters [12]; topics below the minimum size are never shown, as in Clio [7].
- Only masked sample questions go to the model for topic names; aggregates are kept after chat retention deletes the turns, chat-level links are deleted with them.

### Performance

All clustering, labelling, session closing and backfills run in Action Scheduler (group `kimaai`, `$unique = true`) [13] with time budgets; reports read the facts tables with indexes, never the JSON columns. Embedding runs only once per question.

### Tests

- PHPUnit unit with fakes: outcome rules per signal combination, session upsert, online clustering (fixed vectors, threshold, centroid update, merge), PII masking, gap score, attribution window and `product_shown`, order meta via a fake `WC_Order`.
- Integration: migration backfill on a seeded `kimaai_chats`.
- Jest: Insights tabs render data and empty states.
- Playwright in WordPress Playground with WooCommerce: a stubbed chat, then a checkout, then the order appears under Conversions.

## Items

| Item | Title | Acceptance criteria |
| ---- | ----- | ------------------- |
| IN-2 | Chat indexes and conversation facts table with backfill | Indexes on `kimaai_chats`; one session row per conversation, filled live and by chunked backfill; unit and integration tests |
| IN-3 | Outcome job and resolution rate | Hourly job closes idle sessions with the four outcomes; rules unit-tested; Overview tiles show confirmed and assumed apart |
| IN-4 | Insights page shell | Feedback Analytics renamed and tabbed; existing feedback pages unchanged; Jest |
| IN-5 | Topic store and online clustering (premium) | Each new prompt embedded once and assigned; centroids updated; rebuild on embedding change; unit tests with fixed vectors |
| IN-6 | Topic labels and merge job | Weekly job names topics from masked samples; masking unit-tested |
| IN-7 | Topics tab | Volume, change, resolution split, thumbs down and escalation rate per topic; CSV export |
| IN-8 | Content-gap report on the topic store | Gap score and actions (pinned answer, eval case, draft post); `wp kimaai gaps` and MCP tool read the store with fallback |
| IN-9 | WooCommerce attribution | Order meta and conversion row for classic and block checkout; value on paid status; `product_shown`; e2e checkout |
| IN-10 | Form, lead and meeting conversions | CF7, Gravity Forms, LD leads and meetings recorded; Conversions tab |
| IN-11 | Privacy for insights | Exporter and eraser cover session links on orders and forms; policy text; tests |

## Sources

1. Intercom, Fin AI Agent resolutions: <https://www.intercom.com/help/en/articles/8205718-fin-ai-agent-resolutions>
2. Intercom, CX Score: <https://www.intercom.com/help/en/articles/10495092-understand-customer-experience-at-scale-with-the-cx-score>
3. Intercom, Insights and Topics Explorer: <https://www.intercom.com/help/en/articles/10576273-measure-customer-service-with-insights-built-for-the-ai-agent-era> (read from search results only [unverified])
4. Kapa, Coverage Gaps: <https://docs.kapa.ai/analytics/coverage-gaps>
5. Kapa, analytics overview: <https://docs.kapa.ai/analytics/>
6. Chatbase, analytics: <https://www.chatbase.co/docs/user-guides/chatbot/analytics>
7. Anthropic, Clio: <https://www.anthropic.com/research/clio> and paper: <https://arxiv.org/html/2412.13678>
8. OpenAI cookbook, clustering: <https://developers.openai.com/cookbook/examples/clustering>
9. WooCommerce, order attribution meta keys: <https://raw.githubusercontent.com/woocommerce/woocommerce/trunk/plugins/woocommerce/src/Internal/Traits/OrderAttributionMeta.php>
10. WooCommerce, order attribution tracking: <https://woocommerce.com/document/order-attribution-tracking/>
11. WooCommerce, checkout blocks FAQ: <https://developer.woocommerce.com/docs/block-development/cart-and-checkout-blocks/faq/> and order lifecycle hooks: <https://rudrastyh.com/woocommerce/order-lifecycle-hooks.html>
12. WordPress, personal data exporter: <https://developer.wordpress.org/plugins/privacy/adding-the-personal-data-exporter-to-your-plugin/> and eraser: <https://developer.wordpress.org/plugins/privacy/adding-the-personal-data-eraser-to-your-plugin/>
13. Action Scheduler API: <https://actionscheduler.org/api/>
