Reducing False Positives
A false positive occurs when VaultAP flags an invoice for review that turns out to be legitimate. Some false positives are inevitable — the system is designed to err on the side of caution. However, a high false positive rate wastes reviewer time and can erode trust in the system.
This guide covers the tools and strategies available to bring false positives down to a manageable level.
Mark as Safe
When a reviewer approves a flagged invoice and determines it was a false positive, they can mark the specific flag as Safe. This teaches VaultAP to account for known-good patterns in future scoring.
Review the flagged invoice
Open the invoice detail view and examine the triggered rules.
Approve the invoice
If the invoice is legitimate, approve it through the normal workflow.
Mark the flag as Safe
Click Mark as Safe next to the specific rule flag that was a false positive. You can mark individual flags — you do not have to mark the entire invoice.
Mark as Safe is contextual. For example, marking a BANK_CHANGE flag as safe for Vendor A does not suppress BANK_CHANGE flags for other vendors. The system learns per-vendor and per-rule patterns.
What Mark as Safe Does
- Future invoices from the same vendor with the same pattern will receive a reduced weight for that rule.
- The rule is not disabled — it still fires, but its effective contribution to the score decreases for that specific context.
- Multiple "Safe" marks for the same vendor-rule combination further reduce the weight, down to a floor of zero contribution.
Tuning Thresholds
If you are seeing too many invoices routed to review, your auto-clear threshold may be too low.
- Raise the auto-clear threshold to let more low-score invoices pass without review. For example, moving from 25 to 35 can significantly reduce review volume.
- Use dynamic thresholds to let the system find the right level automatically based on your data. See Dynamic Thresholds.
Raising thresholds reduces false positives but also increases the chance of missing a genuinely risky invoice. Adjust in small increments and monitor the results.
Adjusting Rule Weights
Some rules may be disproportionately responsible for false positives in your environment.
Identify the frequent offenders
Go to Dashboard > Risk Scoring > Rule Breakdown. This shows which rules fire most often and their approval rate. Rules with a high fire rate and high approval rate are likely producing false positives.
Lower the weight
Reduce the weight of rules that frequently flag legitimate invoices. For example, if ODD_TIMING fires often because a vendor's billing system submits overnight, reduce its weight from 5 to 2 or 0.
Preview and save
Use Preview Impact to verify the effect, then save.
Reviewing Patterns
Periodically review your false positive patterns to identify systemic issues:
- By vendor — Are certain vendors consistently flagged? Consider using Mark as Safe or adjusting vendor-specific settings.
- By rule — Is one rule responsible for most false positives? Lower its weight or refine its configuration.
- By time period — Do false positives spike after certain events (e.g., onboarding new vendors, fiscal year changes)? Temporary threshold adjustments may help.
Common Causes and Solutions
| Cause | Symptoms | Solution |
|---|---|---|
| New vendors | NEW_VENDOR fires on every first invoice from a legitimate vendor. | Lower the NEW_VENDOR weight, or accept it as a one-time flag and use Mark as Safe. |
| Recurring invoices | DUPLICATE_NEAR fires on subscription or retainer invoices billed at the same amount each cycle. | Reduce its weight for organizations with many fixed-price contracts, or Mark as Safe per vendor. |
| Seasonal vendors | FREQUENCY_SPIKE fires on vendors used seasonally, where a burst of invoices is normal. | Lower its weight, or Mark as Safe for known seasonal vendors. |
| Missing PO references | NO_PO_MATCH fires on vendors who legitimately invoice without a purchase order. | Reduce its weight if your organization does not use POs for every vendor. |
| Legitimate bank changes | BANK_CHANGE fires when a vendor genuinely updates their bank details. | Verify and Mark as Safe. Do not disable this rule — bank changes remain a critical fraud signal. |
| Auto-clear too aggressive | Invoices below threshold pass through but later turn out risky. | This is the opposite problem (false negatives). Lower the auto-clear threshold. |
A false positive rate of 10-20% on flagged invoices is typical for a well-tuned system. If your rate is significantly higher, start with the rule breakdown analysis to pinpoint the biggest contributors.