Fiber to the shed: testing the fiber optic transceivers / 2021-01-15

I wanted to get an idea whether the network over the fiber optic transceivers is reliable. So at the moment our dining room table looks like a network lab.

For testing networks there is iperf. I found out the Raspberry Pi 3B+ can't keep up with 100 Mbit/second UDP packets, so I searched for a speed where the Pi performs ok. This turns out to be 30 mbit, at higher speeds there is packet loss. I also had to reduce packet size to avoid fragmentation which costs CPU. I use IPv6 because that's what I'm used to. It turned out later the maximum speed without loss is higher with IPv4 than with IPv6.

Server on the raspberry pi:

koos@raspberrypi:~ $ iperf --version
iperf version 2.0.9 (1 June 2016) pthreads
koos@raspberrypi:~ $ iperf -s -V -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:  160 KByte (default)
------------------------------------------------------------

Test without fiber optic transceivers in the path. Layer 2 route: virtual machine - host machine - utp - network switch - utp - network switch - utp - raspberry pi

koos@wozniak:~$ iperf --version
iperf version 2.0.9 (1 June 2016) pthreads
koos@wozniak:~$ iperf -V -u -b30M -i 10 -t 120 -M 10 -l 1400 -c ..
------------------------------------------------------------
Client connecting to .., UDP port 5001
Sending 1400 byte datagrams, IPG target: 373.33 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
..
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec
[  3] Sent 321430 datagrams
[  3] Server Report:
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec   0.004 ms    0/321430 (0%)

Test with fiber optic transceivers in the path. Layer 2 route: virtual machine - host machine - utp - network switch - utp - network switch - utp - fiber optic transceiver - fiber - fiber optic transceiver - utp - raspberry pi

koos@wozniak:~$ iperf -V -u -b30M -i 10 -t 120 -M 10 -l 1400 -c ..
------------------------------------------------------------
Client connecting to .., UDP port 5001
Sending 1400 byte datagrams, IPG target: 373.33 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec
[  3] Sent 321430 datagrams
[  3] Server Report:
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec   0.007 ms    0/321430 (0%)

Trying with IPv4 shows that packet loss starts to occur above 45 mbit. This is an interesting difference.

But the important conclusion is that there is no packet loss over the fiber path. There may be a bit more latency, but that's not a surprise. As a last test I looked at purely ping traffic using IPv6.

Without fiber in the path:

koos@wozniak:~$ ping -c 100 -i 0.2 -q ..
PING ..(.. (2001:xxxx)) 56 data bytes

--- .. ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 20192ms
rtt min/avg/max/mdev = 0.567/0.680/0.866/0.063 ms

With fiber in the path:

koos@wozniak:~$ ping -c 100 -i 0.2 -q ..
PING ..(.. (2001:xxxx)) 56 data bytes

--- .. ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 20191ms
rtt min/avg/max/mdev = 0.625/0.738/0.828/0.046 ms

This also shows a bit more latency over fiber.

The extra latency is probably due to the fiber optic transceivers containing a network switch.