[perfsonar-user] How to ignore negative latency values in PerfSONAR Dashboard?

[Jeremy Palmer]

Is it possible to make PerfSonar Dashboard ignore negative latency numbers so
they don't show up as CRITICAL/RED squares on the PerfSONAR Dashboard?
Our OWAMP dashboard is configured to return Critical status if latency is >
60ms. However, some of our sites are very close to eachother (less then 0.5ms
one-way) and due to NTP clock skew, they sometimes return negative results
(example: -0.016ms). Unfortunately these negative numbers cause the dashboard to
alarm Critical/red for that check. This confuses our TAC guys and can also mask
real latency issues.
Any way to make the dashboard ignore negative latency numbers so they don't
return CRITICAL status?

[Andrew Lake]

Hi Jeremy,

If you are using the meshconfig, I am not sure there is a way. If you edit your /etc/maddash/maddash-server/maddash.yaml file by hand though, do a text search for "check_owdelay.pl". You should see a command similar to the following:

/opt/perfsonar_ps/nagios/bin/check_owdelay.pl -u %maUrl -w 0.001 -c 0.1 -r 2700 -l -p -s %row -d %col

The -w and -c are the warning and critical thresholds respectively. To make negative values OK add \~: to the warning range. Example below:

/opt/perfsonar_ps/nagios/bin/check_owdelay.pl -u %maUrl -w \~:0.001 -c 0.1 -r 2700 -l -p -s %row -d %col

The \ is to escape the ~ so it isn't expanded to a home directory. The ~ is a special character meaning negative infinity. See the nagios threshold format documentation for more details: https://nagios-plugins.org/doc/guidelines.html#THRESHOLDFORMAT

Thanks,
Andy

Is it possible to make PerfSonar Dashboard ignore negative latency numbers so they don't show up as CRITICAL/RED squares on the PerfSONAR Dashboard?
Our OWAMP dashboard is configured to return Critical status if latency is > 60ms. However, some of our sites are very close to eachother (less then 0.5ms one-way) and due to NTP clock skew, they sometimes return negative results (example: -0.016ms). Unfortunately these negative numbers cause the dashboard to alarm Critical/red for that check. This confuses our TAC guys and can also mask real latency issues.
Any way to make the dashboard ignore negative latency numbers so they don't return CRITICAL status?
--
Jeremy Palmer
Senior Network Engineer
ViaWest, Inc.
Office: 720.891.1045
Fax: [phone]
http://www.viawest.com

[Jeremy Palmer]

Thanks for the tips. I was able to use these suggestion to edit the nagios
commands in maddash.yaml to rid our dashboard of warnings/errors for negative
latency values. I had a few minor issues as follows:

- I had to restart the maddash-server process once I edited the yaml file.

- It would not work when using the \ (backslash) to escape the ~ character (as
in "-w \~:40"). This caused the entire dashboard to go orange and display the
mouseover error: Cannot parse warning range: "\~:40" (see below screenshot).
Loading Image...
I simply removed the \ character in the nagios command and this seems to work fine.

- I had to used this on both the -w (warning) and -c (critical) switches. My
working command is as follows:
command: "/opt/perfsonar_ps/nagios/bin/check_owdelay.pl -u %maUrl -w ~:40 -c
~:60 -r 300 -s %row -d %col"

Jeremy Palmer
Senior Network Engineer
ViaWest, Inc.
Office: 720.891.1045
Fax: [phone]
http://www.viawest.com

Post by Andrew Lake
Hi Jeremy,
If you are using the meshconfig, I am not sure there is a way. If you edit
your /etc/maddash/maddash-server/maddash.yaml file by hand though, do a
text search for "check_owdelay.pl". You should see a command similar to the
/opt/perfsonar_ps/nagios/bin/check_owdelay.pl -u %maUrl -w 0.001 -c 0.1 -r
2700 -l -p -s %row -d %col
The -w and -c are the warning and critical thresholds respectively. To make
/opt/perfsonar_ps/nagios/bin/check_owdelay.pl -u %maUrl *-w \~:0.001* -c 0.1
-r 2700 -l -p -s %row -d %col
The \ is to escape the ~ so it isn't expanded to a home directory. The ~ is a
special character meaning negative infinity. See the nagios threshold format
https://nagios-plugins.org/doc/guidelines.html#THRESHOLDFORMAT
Thanks,
Andy

Post by Jeremy Palmer
Is it possible to make PerfSonar Dashboard ignore negative latency numbers so
they don't show up as CRITICAL/RED squares on the PerfSONAR Dashboard?
Our OWAMP dashboard is configured to return Critical status if latency is >
60ms. However, some of our sites are very close to eachother (less then 0.5ms
one-way) and due to NTP clock skew, they sometimes return negative results
(example: -0.016ms). Unfortunately these negative numbers cause the dashboard
to alarm Critical/red for that check. This confuses our TAC guys and can also
mask real latency issues.
Any way to make the dashboard ignore negative latency numbers so they don't
return CRITICAL status?
--
Jeremy Palmer
Senior Network Engineer
ViaWest, Inc.
Office: 720.891.1045
Fax: [phone]
http://www.viawest.com