[perfsonar-user] NTP errors when running bwctl, and iperf3 times out

[Merzouki, Mheni (IntlAssoc)]

Hi all,

i'm new to using perfsonar, i have the latest perfsonar-toolkit installed on my machine.
i'm trying to test the throughput and bandwidth of my network, i'm doing that by running iperf3 and bwctl as a client for one of the servers provided at http://stats.es.net/ServicesDirectory

- for BWCTL

this is what i get when running bwctl

***@machine:~$ bwctl -a 10 -v -T iperf3 -t 30 -O 4 -c "alt-tic-cdc-pe-b-perfsonar.net.nih.gov:4823"
Messages being sent to syslog(user,err)
bwctl: NTP: STA_NANO should be set. Make sure ntpd is running, and your NTP configuration is good.
bwctl: getaddrinfo(): Name or service not known
bwctl: I2AddrAddrInfo failed
bwctl: Unable to connect to alt-tic-cdc-pe-b-perfsonar.net.nih.gov:4823

i'm getting NTP errors even thought i use the -a option, and my ntp service is active

***@solarflare1:~$ sudo service ntp status
$B!|(B ntp.service - LSB: Start NTP daemon
Loaded: loaded (/etc/init.d/ntp; bad; vendor preset: enabled)
Active: active (running) since Wed 2018-11-28 13:16:02 EST; 10min ago
Docs: man:systemd-sysv-generator(8)
Process: 5800 ExecStop=/etc/init.d/ntp stop (code=exited, status=0/SUCCESS)
Process: 5881 ExecStart=/etc/init.d/ntp start (code=exited, status=0/SUCCESS)
Tasks: 1
Memory: 568.0K
CPU: 63ms
CGroup: /system.slice/ntp.service
$B(&(!(B5893 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 111:117

Nov 28 13:16:02 solarflare1 systemd[1]: Starting LSB: Start NTP daemon...
Nov 28 13:16:02 solarflare1 ntp[5881]: * Starting NTP server ntpd
Nov 28 13:16:02 solarflare1 ntp[5881]: ...done.
Nov 28 13:16:02 solarflare1 systemd[1]: Started LSB: Start NTP daemon.
Nov 28 13:16:02 solarflare1 ntpd[5893]: proto: precision = 0.068 usec (-24)
Nov 28 13:16:02 solarflare1 ntpd[5893]: switching logging to file /var/log/ntpd

- for iperf3

I always get a connection time out, eventhough i can ping the specified server, this is what i get when i run the verbose mode.

***@machine:~$ iperf3 -V -c wash-pt1.es.net
iperf 3.6
Linux solarflare1 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64
iperf3: error - unable to connect to server: Connection timed out

any help or pointers would be very apreciated.
thank you.

Mheni.

[Szymon Trocha]

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

[Merzouki, Mheni (IntlAssoc)]

Hi,
Thank you for your response

I have another machine that is running an iperf3 server, and when I run pscheduler with that machine the task gets submitted but it has a hard time getting the results
***@machine1:~$ pscheduler task throughput --dest A.B.C.D (iperf3 server address)
Submitting task...
Task URL:
https://localhost/pscheduler/tasks/0e640e37-192d-4eb6-a8b5-c5a8511b3795
Running with tool 'iperf3'
Fetching first run...

Next scheduled run:
https://localhost/pscheduler/tasks/0e640e37-192d-4eb6-a8b5-c5a8511b3795/runs/ba19e735-b368-4539-878c-acbc6cbd98c5
Starts 2018-11-29T11:53:10-05:00 (~8 seconds)
Ends 2018-11-29T11:53:29-05:00 (~18 seconds)
Waiting for result...

Problem fetching results: Resource Not found.

No further runs scheduled.

But I’m more interested in running a test with a remote host, so I tried with the example that worked for you, and it looks like it can’t find the tools
***@machine1:~$ pscheduler task throughput --dest wash-pt1.es.net
Submitting task...
Unable to post task: Error getting tools from wash-pt1.es.net: Request timed out
The 'pscheduler troubleshoot' command may be of use in problem
diagnosis. 'pscheduler troubleshoot --help' for more information.

And this is what troubleshooting outputs
***@machine1:~$ pscheduler troubleshoot wash-pt1.es.net
Performing basic troubleshooting of localhost and wash-pt1.es.net.

localhost:

Checking path MTU... Unsafe or unknown: Found only one MTU in trace to localhost
Checking for pScheduler... OK.
Checking clock... Unsynchronized (See check against wash-pt1.es.net)
Idle test.... 13 seconds.... Checking archiving... OK.

wash-pt1.es.net:

Checking path MTU... 1500+
Checking for pScheduler... Failed.
Request timed out

is this a synchronization problem? Do I need to make wash-pt1.es.net my NTP server?
Mheni.
From: Szymon Trocha <***@man.poznan.pl>
Sent: Wednesday, November 28, 2018 5:53 PM
To: Merzouki, Mheni (IntlAssoc) <***@nist.gov>
Cc: perfsonar-***@internet2.edu
Subject: Re: [perfsonar-user] NTP errors when running bwctl, and iperf3 times out

Hi Mheni,

W dniu 28.11.2018 o 19:53, Merzouki, Mheni (IntlAssoc) pisze:
Hi all,

i'm new to using perfsonar, i have the latest perfsonar-toolkit installed on my machine.
i'm trying to test the throughput and bandwidth of my network, i'm doing that by running iperf3 and bwctl as a client for one of the servers provided at http://stats.es.net/ServicesDirectory<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fstats.es.net%2FServicesDirectory&data=02%7C01%7Cmheni.merzouki%40nist.gov%7C7e59608ed1f649b4f43d08d6558458cb%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C0%7C636790424251281675&sdata=cWDPYGqoU6m5neRQMqyq0%2BPM6SnP5ZAqD6lX87XTxa4%3D&reserved=0>

- for BWCTL

this is what i get when running bwctl

Please note that BWCTL was replaced by pscheduler in the latest versions. See http://docs.perfsonar.net/pscheduler_intro.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.perfsonar.net%2Fpscheduler_intro.html&data=02%7C01%7Cmheni.merzouki%40nist.gov%7C7e59608ed1f649b4f43d08d6558458cb%7C2ab5d82fd8fa4797a93e054655c61dec%7C1%7C0%7C636790424251291688&sdata=Lxl7iviRhqYDN8Tu7bNCofbkaJlqmdBcw5JjB78r%2Fxg%3D&reserved=0>

***@machine:~$ bwctl -a 10 -v -T iperf3 -t 30 -O 4 -c "alt-tic-cdc-pe-b-perfsonar.net.nih.gov:4823"
Messages being sent to syslog(user,err)
bwctl: NTP: STA_NANO should be set. Make sure ntpd is running, and your NTP configuration is good.
bwctl: getaddrinfo(): Name or service not known
bwctl: I2AddrAddrInfo failed
bwctl: Unable to connect to alt-tic-cdc-pe-b-perfsonar.net.nih.gov:4823



This server seems to lack DNS entry so calling by hostname will not work and you must use IP address:

[***@psmall-poz1 ~]# host 156.40.197.85
85.197.40.156.IN-ADDR.ARPA domain name pointer atl-tic-cdc-pe-b-perfsonar.net.nih.gov.
[***@psmall-poz1 ~]# host atl-tic-cdc-pe-b-perfsonar.net.nih.gov
;; connection timed out; no servers could be reached
[***@psmall-poz1 ~]#

Try using pscheduler instead of bwctl and iperf when running tests. This is what perfSONAR was designed for.

Please note that the host you picked up may not be allowing public tests although it it visible in the directory. Then it will refuse to connect.



i'm getting NTP errors even thought i use the -a option, and my ntp service is active

***@solarflare1:~$ sudo service ntp status
● ntp.service - LSB: Start NTP daemon
Loaded: loaded (/etc/init.d/ntp; bad; vendor preset: enabled)
Active: active (running) since Wed 2018-11-28 13:16:02 EST; 10min ago
Docs: man:systemd-sysv-generator(8)
Process: 5800 ExecStop=/etc/init.d/ntp stop (code=exited, status=0/SUCCESS)
Process: 5881 ExecStart=/etc/init.d/ntp start (code=exited, status=0/SUCCESS)
Tasks: 1
Memory: 568.0K
CPU: 63ms
CGroup: /system.slice/ntp.service
└─5893 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 111:117

Nov 28 13:16:02 solarflare1 systemd[1]: Starting LSB: Start NTP daemon...
Nov 28 13:16:02 solarflare1 ntp[5881]: * Starting NTP server ntpd
Nov 28 13:16:02 solarflare1 ntp[5881]: ...done.
Nov 28 13:16:02 solarflare1 systemd[1]: Started LSB: Start NTP daemon.
Nov 28 13:16:02 solarflare1 ntpd[5893]: proto: precision = 0.068 usec (-24)
Nov 28 13:16:02 solarflare1 ntpd[5893]: switching logging to file /var/log/ntpd

- for iperf3

I always get a connection time out, eventhough i can ping the specified server, this is what i get when i run the verbose mode.

***@machine:~$ iperf3 -V -c wash-pt1.es.net
iperf 3.6
Linux solarflare1 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64
iperf3: error - unable to connect to server: Connection timed out

any help or pointers would be very apreciated.
thank you.


This server (and others) may not run iperf server. pscheduler test works fine:

[***@psmall-poz1 ~]# pscheduler task throughput --dest wash-pt1.es.net
Submitting task...
Task URL:
https://localhost/pscheduler/tasks/e6f917dc-492a-479e-88ac-81f86298c83a
Running with tool 'iperf3'
Fetching first run...

Next scheduled run:
https://localhost/pscheduler/tasks/e6f917dc-492a-479e-88ac-81f86298c83a/runs/71607017-94ca-4855-8c41-1ef8e4d9166a
Starts 2018-11-28T22:47:45Z (~3 seconds)
Ends 2018-11-28T22:48:04Z (~18 seconds)
Waiting for result...

* Stream ID 5
Interval Throughput Retransmits Current Window
0.0 - 1.0 107.45 Mbps 0 2.73 MBytes
1.0 - 2.0 525.73 Mbps 0 25.49 MBytes
2.0 - 3.0 817.89 Mbps 1 28.29 MBytes
3.0 - 4.0 943.34 Mbps 0 28.29 MBytes
4.0 - 5.0 856.70 Mbps 0 28.29 MBytes
5.0 - 6.0 751.07 Mbps 0 28.29 MBytes
6.0 - 7.0 779.16 Mbps 1 28.29 MBytes
7.0 - 8.0 773.10 Mbps 1 28.29 MBytes
8.0 - 9.0 772.01 Mbps 0 28.29 MBytes
9.0 - 10.0 770.66 Mbps 0 28.29 MBytes

Summary
Interval Throughput Retransmits
0.0 - 10.0 709.66 Mbps 3

No further runs scheduled.
[***@psmall-poz1 ~]#

Regards,

[Szymon Trocha]

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

[Andrew Lake]

Hi,

Is DNS configured properly on your host? Try running “host perfsonar.net”
to see if you get a result. That would potentially explain all your issue
including the NTP synchronization. Also be sure to check the firewalls as
Szymon indicated.

Thanks,
Andy


On November 29, 2018 at 2:36:55 PM, Szymon Trocha (
***@man.poznan.pl) wrote:

W dniu 29.11.2018 o 18:28, Merzouki, Mheni (IntlAssoc) pisze:

Hi,

Thank you for your response



I have another machine that is running an iperf3 server, and when I run
pscheduler with that machine the task gets submitted but it has a hard time
getting the results

****@machine1:~$ pscheduler task throughput --dest A.B.C.D (iperf3 server
address)*

Submitting task...

Task URL:

https://localhost/pscheduler/tasks/0e640e37-192d-4eb6-a8b5-c5a8511b3795

Running with tool 'iperf3'

Fetching first run...



Next scheduled run:

https://localhost/pscheduler/tasks/0e640e37-192d-4eb6-a8b5-c5a8511b3795/runs/ba19e735-b368-4539-878c-acbc6cbd98c5

Starts 2018-11-29T11:53:10-05:00 (~8 seconds)

Ends 2018-11-29T11:53:29-05:00 (~18 seconds)

*Waiting for result...*



*Problem fetching results: Resource Not found.*



No further runs scheduled.



But I’m more interested in running a test with a remote host, so I tried
with the example that worked for you, and it looks like it can’t find the
tools

* ***@machine1:~$ pscheduler task throughput --dest
wash-pt1.es.net <http://wash-pt1.es.net>*

Submitting task...

*Unable to post task: Error getting tools from wash-pt1.es.net
<http://wash-pt1.es.net>: Request timed out*

The 'pscheduler troubleshoot' command may be of use in problem

diagnosis. 'pscheduler troubleshoot --help' for more information.



And this is what troubleshooting outputs

****@machine1:~$ pscheduler troubleshoot wash-pt1.es.net
<http://wash-pt1.es.net>*

Performing basic troubleshooting of localhost and wash-pt1.es.net.



localhost:



Checking path MTU... Unsafe or unknown: Found only one MTU in trace to
localhost

Checking for pScheduler... OK.

* Checking clock... Unsynchronized (See check against wash-pt1.es.net
<http://wash-pt1.es.net>)*

Idle test.... 13 seconds.... Checking archiving... OK.



wash-pt1.es.net:



Checking path MTU... 1500+

* Checking for pScheduler... Failed*.

Request timed out



is this a synchronization problem? Do I need to make *wash-pt1.es.net
<http://wash-pt1.es.net> *my NTP server*?*

-

Hi,

Synchronization is definitely worth checking (ntpq -p) but I have doubts
about you host or network firewall which may not permit perfSONAR traffic
so the requests time out. This may also be the reason for NTP not
synchronising as it cannot connect to NTP servers.

Please verify http://docs.perfsonar.net/manage_security.html

And also what type of installation do you have? Full latest toolkit or one
of the bundles?

Regards
--
Szymon Trocha
Poznań Supercomputing & Networking Center
General NOC phone [phone] | noc.pcss.pl
Personal desk phone [phone]
Wysłaliśmy do Ciebie ten e-mail w odpowiedzi na Twoje zapytanie lub w
związku z oferowaną usługą. Przesłanie korespondencji do Centrum
Zarządzania PCSS lub zgłoszenie telefoniczne jest równoznaczne z wyraÅŒeniem
zgody na przetwarzanie danych osobowych przez Instytut Chemii
Bioorganicznej Polskiej Akademii Nauk w Poznaniu adres: ul. Z. Noskowskiego
12/14, 61-704 Poznań. Szczegółowe informacje znajdują się w naszej Polityce
prywatności <http://noc.pcss.pl/index.html#PolitykaPrywatnosci>. | This
message has been sent as a part of communication with PSNC NOC or your
service request sent to us. For more information read our Privacy Policy
<http://noc.psnc.pl/index.html#PrivacyPolicy>.