EWMA and CUSUM for AI agent monitoring

Sentinel uses two statistical algorithms for anomaly detection: EWMA (Exponentially Weighted Moving Average) and CUSUM (Cumulative Sum). Each catches different types of anomalies. Understanding how they work helps you configure them correctly.

EWMA: Exponentially Weighted Moving Average

EWMA maintains a smoothed average that gives more weight to recent observations. It is good for detecting sudden changes in a metric.

How it works

At each time step, the average is updated:

EWMA_t = alpha * value_t + (1 - alpha) * EWMA_(t-1)

Where alpha (0 < alpha < 1) is the smoothing factor:

• High alpha (0.5-0.9): Reacts quickly to changes, more sensitive to noise
• Low alpha (0.1-0.3): Reacts slowly, smooths out noise

An alert is triggered when the current value deviates from the EWMA by more than N standard deviations.

Example

An agent normally makes 5 tool calls per minute. EWMA tracks this:

Minute 1: 5 calls, EWMA = 5.0
Minute 2: 4 calls, EWMA = 4.7
Minute 3: 6 calls, EWMA = 5.1
Minute 4: 50 calls, EWMA = 18.6  <-- Alert: value far from average

Configuration

sentinel: {
  detectors: {
    ewma: {
      alpha: 0.3,           // Smoothing factor
      sigmaThreshold: 3.0,  // Alert at 3 standard deviations
    }
  }
}

CUSUM: Cumulative Sum

CUSUM accumulates small deviations from an expected value. It is good for detecting persistent, gradual changes that EWMA might miss.

How it works

At each time step, the cumulative sum is updated:

S_t = max(0, S_(t-1) + (value_t - expected - slack))

Where:

• expected is the target value (usually the baseline average)
• slack is a tolerance parameter (allows small deviations without accumulation)

An alert is triggered when S_t exceeds the threshold.

Example

An agent's denial rate is normally 5%. It drifts slightly upward:

Minute 1: 6% (drift of 1%), S = 0 (within slack)
Minute 2: 7% (drift of 2%), S = 0.5
Minute 3: 7% (drift of 2%), S = 1.0
Minute 4: 8% (drift of 3%), S = 2.5
...
Minute 10: 9%, S = 5.0  <-- Alert: cumulative drift exceeds threshold

Each individual reading looks normal. But CUSUM accumulates the small deviations until they cross the threshold.

Configuration

sentinel: {
  detectors: {
    cusum: {
      slack: 0.5,      // Allow this much drift per step without accumulation
      threshold: 5.0,  // Alert when cumulative sum exceeds this
    }
  }
}

When to use which

| Anomaly type | Best detector | |-------------|---------------| | Sudden spike | EWMA | | Sudden drop | EWMA | | Gradual drift | CUSUM | | Persistent subtle change | CUSUM | | Intermittent spikes | EWMA with low alpha |

Using both together

Sentinel runs both detectors on every metric. This catches both sudden anomalies (EWMA) and gradual drift (CUSUM):

sentinel: {
  detectors: {
    ewma: { alpha: 0.3, sigmaThreshold: 3.0 },
    cusum: { slack: 0.5, threshold: 5.0 },
  },
  metrics: ['action_rate', 'denial_rate', 'tool_distribution'],
}

An alert from either detector triggers the configured response. In practice, the two detectors rarely fire on the same event, which means they complement each other's blind spots.