Scenario performance dashboard
Last updated: September 29, 2026
Overview
The Scenario Performance dashboard shows how each of your transaction monitoring and/or fraud detection scenarios has performed over the last 12 months, so you can find scenarios to tune. It answers two questions for every scenario:
How often does it fire? This is the hit rate: the share of evaluated transactions the scenario flagged. A very high hit rate can mean the scenario is too broad; a very low one can mean it is too restrictive.
How useful are its alerts? This is the true positive rate: the share of closed alerts that analysts confirmed as genuine risk. A high rate means the scenario mostly produces real risk rather than noise.
The ideal scenario has a hit rate that matches your risk appetite and a high true positive rate.
Tuning a scenario in three steps
Find scenarios with a low true positive rate or an unexpected hit rate.
Use the filters to see whether the problem comes from a specific customer segment or scenario configuration.
Adjust the scenario's parameters or segment targeting, then watch the monthly charts over the following weeks to confirm the change worked.
Filtering the dashboard
Every chart covers the current month plus the previous 11 months.
Filter | What it lists | Good to know |
|---|---|---|
Product | Transaction Monitoring and/or Fraud Detection, depending on what your account is entitled to. | All users in the account see the same options. If your account has Fraud Detection but no fraud transactions yet, selecting it shows empty charts. |
Scenario Configuration | Every scenario configuration (bundle) used to evaluate transactions in the last 12 months. | If a configuration has no name, its identifier is shown instead. |
Customer Segment | The highest-priority customer segment used when a scenario evaluated a transaction, over the last 12 months. | Segment is only recorded when a scenario fires, so this filter changes hit rate differently from the others (see below). |
How filters affect hit rate.
Product and Scenario Configuration narrow both the flagged transactions and the total evaluated transactions. Customer Segment narrows only the flagged transactions, because segment information isn't recorded for transactions that didn't trigger the scenario. The result is the share of all evaluated transactions that were flagged in that segment.
The charts
The dashboard has three charts. The first gives a 12-month snapshot per scenario; the other two show how hit rate changes month by month.
1. Hit Rate vs True Positive Rate by Scenario
A scatter chart with one bubble per scenario.
Horizontal and vertical axes: hit rate and true positive rate, each pooled across all 12 months into a single figure.
Bubble size: the number of transactions the scenario flagged. Bigger bubbles create more alert volume for your team.
Hit rate is measured over transactions. True positive rate is measured over closed alerts.
How to read it
Where the bubble sits | What it usually means | What to consider |
|---|---|---|
High hit rate, low true positive rate | Fires often, mostly on noise | Tighten thresholds or narrow segment targeting. Prioritise large bubbles. |
High hit rate, high true positive rate | Fires often and finds real risk | Make sure the team can handle the volume. |
Low hit rate, high true positive rate | Precise, rarely fires | Consider whether it's too restrictive and could be missing risk. |
Low hit rate, low true positive rate | Rarely fires and rarely useful | Review whether the scenario is still needed. |
2. Hit Rate per Scenario by Month
Shows each scenario's hit rate for each month separately, so you can spot trends and see the effect of a configuration change.
Each month's figure uses only that month's transactions. Chart 1 pools all 12 months into one ratio, so the two won't match exactly.
The month comes from the transaction's own date.
3. Hit Rate Evolution
A table listing each scenario that matches your filters, with its hit rate for each of the past 12 months.
"-" means the scenario evaluated no transactions that month.
0% means the scenario evaluated transactions but didn't fire on any of them.
How the metrics are calculated
Hit rate is how often a scenario fires. It measures volume, not accuracy: a scenario that fires on every transaction scores 100%.
Hit rate = transactions the scenario flagged ÷ transactions the scenario evaluated
A transaction counts as flagged (a hit) when the scenario outcome is Allow or Hold, meaning an alert was raised.
Filters applied | Flagged transactions (top) | Evaluated transactions (bottom) |
|---|---|---|
None or Product only | All flagged by the scenario | All evaluated by the scenario |
Scenario Configuration | Flagged within that configuration | Evaluated within that configuration |
Customer Segment | Flagged in that segment | All evaluated by the scenario |
Configuration + Segment | Flagged in that segment, within the configuration | Evaluated within the configuration |
True positive rate is the share of closed alerts that analysts confirmed as genuine risk.
True positive rate = alerts closed as True Positive ÷ (alerts closed as True Positive + alerts closed as False Positive)
Only closed alerts count. Alerts still open or not yet reviewed are left out of both the top and bottom of the ratio.
When there's no data. Both metrics show 0% when there's nothing to divide by. This means a scenario with no closed alerts shows a 0% true positive rate, the same as a scenario whose alerts were all false positives. Check alert volumes before drawing conclusions from a 0%.
FAQs
Why doesn't the monthly chart add up to the 12-month figure in chart 1? Chart 1 is one ratio across all 12 months, not an average of the monthly rates.
A scenario shows 0% true positive rate. Is it performing badly? Not necessarily. It may simply have no closed alerts yet. Check how many alerts it has raised and closed before tuning it.
Why does the Customer Segment filter make hit rates look lower? The segment filter only narrows the flagged transactions, not the total evaluated. You're seeing that segment's share of all evaluated transactions, which will always be lower than or equal to the unfiltered hit rate.