What KPIs or metrics does PLAN D provide for result quality?

Our KPIs are not for marketing but for realistically assessing result quality. They transparently show how well the AI system actually performs its task, traceable, explainable, and comparable over time.

Which KPIs are meaningful depends on the specific use case. That is why we recommend the appropriate metrics, explain the rationale behind their selection, and clarify what they mean in business terms. Examples include:

• Prediction accuracy: How close the AI gets to the expected or known result.
  Examples: R², MAE (Mean Absolute Error), RMSE, MAPE, Accuracy
• Error types and error distribution: What types of errors occur and which of them are critical or tolerable in the business process.
  Examples: Confusion Matrix, False Positives / False Negatives, Precision, Recall, F1-Score
• Model drift: Detection of changes in data or patterns that cause the model to gradually lose predictive power.
  Examples: Data Drift, Prediction Drift, Population Stability Index (PSI), Feature Shift
• Response quality for agents: Assessment of factual correctness, source grounding, and goal achievement of responses.
  Examples: Mean Reciprocal Rank (MRR), Retrieval Precision / Recall (R-P / R-R), Context Coverage (CC), Groundedness Score (GS), Hallucination Rate (HR), Task Success Rate (TSR).
• Cost savings: What impact the AI results have on costs, turnaround times, risks, or revenue.
  Examples: Time saved per process, reduction of manual reviews, avoided error costs, automation rate‍
• Cost-performance ratio: The relationship between computational effort, runtime, infrastructure costs, and achieved model performance.
  Examples: Inference cost per prediction, latency, performance per euro of cloud costs