I'm looking for an algorithm that can help identify abnormal trends in time-series metrics. The best I've been able to find so far is ARIMA (a completely new concept for me). We offer several services which we monitor active usage against, for any given time of day (typically updated once a minute). Here's a example of one of our services: What I'm interested in is gradual degradation that doesn't get noticed until ... well, it gets noticed. For example, if we zoom in to 12/6 we see: So what I'm specifically interested in here is the trend downward `@17:30~`. Likewise another good example would be: Here we can see the abnormalities on 12/1 and 12/4 during peak. Are there any formal methods of detecting this sort of trend (or rather, abnormality to a historical trend)? Specifically interested in the _trend_ here, not the raw numbers. These services have a historical decreasing usage as time goes on - eventually, these numbers will drop from say 60k peaks to 15k peaks. Is ARIMA the best bet here? e.g., predicting the future curve based on yesterday's curve, setting some threshold, and alerting if that threshold is breached? Or is there a more straightforward way to go about this? Am I fundamentally misunderstanding ARIMA?
Here's kind of what I had in mind for what I'd like to accomplish:
(i.e., taking the curve of one dataset and seeing if another dataset fits within that curve)
