• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

How do I approach creating Dynamic Baselining Program something like in NewRelic?

 
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello Folks,

I am trying to model a very basic program (mostly in Java) that takes input a pair of number :
one is Unix Timestamp and other is some integer or float value. And it outputs some "baselined range" for each time epoch.

(Effectively, create a dynamic, intelligent base lining such that when current value violates, it is identified. Unlike a static threshold which gives false spikes.)

I am unsure what mathematical formula or modeling I should use.

Here is link to what I am trying to build like:
https://blog.newrelic.com/2016/11/16/dynamic-baseline-alerts/

New Re--c Dynamic Baseline Alerts can account for that by producing dynamic thresholds using a mixture of advanced statistics and algorithms to detect small performance anomalies in your applications and services



I pardon if its off topic, but I need some hint towards mathematical algos.
 
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't think you can do that with just a timestamp and an integer value. To calculate an expected value for a given date, you need to know the recent history.

I'm not sure if it can be used to predict the future, but you might want to look at Kalman filters for a basis to improve on. The problem in this scenario with Kalman filters is that they don't account for periodic spikes.

If you want more help with this problem, you might have to find it on a website that's more mathematically inclined.
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Apparently Kalman filters can be used to predict the future. To include periodic spikes, you might have to influence the measurements using weighted measurements from periods in the past, such as a week ago, a month ago, a year ago.
 
Roman Swinson
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Apparently Kalman filters can be used to predict the future. To include periodic spikes, you might have to influence the measurements using weighted measurements from periods in the past, such as a week ago, a month ago, a year ago.



So, After little bit if digging internet i found I need to use model such as ARIMA.

Thanks for the input.
 
reply
    Bookmark Topic Watch Topic
  • New Topic