News items for tag ipv6 - Koos van den Hout

Our child plays minecraft regularly. The start was with the Microsoft minecraft edition but recently the java edition became available too without paying again.

I have set up the bedrock server for the Microsoft minecraft edition to make it possible to play with other people outside the house. So the most recent request was to do this for the java edition too.

I don't know much about minecraft but I can do enough with just some websearching and finding a howto. So I started with How to Set Up a Dedicated Minecraft Server on Linux which seems to be a way to try to sell dedicated servers but I have enough server hardware here at home so I just used the same virtual machine which ran the minecraft bedrock server.

It turned out the default-jdk resulted in openjdk-11 getting installed and this resulted in not being able to run the latest minecraft java server. I switched to openjdk-17-jre-headless because I only need the runtime and I never want to run the graphical stuff, so that saved a lot in needed libraries and other overhead.

The server started fine, but the minecraft java edition couldn't connect to it when trying to connect by name, but gave no usable error message. That's a different rant. I checked on the server side and saw the listening socket in dual-stack mode.

With tcpdump I soon found out the minecraft java edition starts with the IPv4 address and gives up when that fails. The solution was to remove the IPv4 address (A record) from the name, flush the dns cache and after that it worked. This does mean that when friends want to connect that are behind ISPs that only support legacy Internet addresses they will have a different problem.

Years and years after writing proposals to start doing something with IPv6 at work I noticed the first systems actually having IPv6 connectivity in production networks.

Finally getting there! I wonder when workstations will start having IPv6 connectivity.

Someone mailed me a few days ago with an interesting question. So I typed a reasonably long answer. But upon sending this answer I received the following error message:

   ----- The following addresses had permanent fatal errors -----
<????????@outlook.com>
    (reason: 550 5.7.1 Unfortunately, messages from [45.83.232.134] weren't sent. Please contact your Internet se...ail.live.com/mail/troubleshooting.aspx#errors. [HE1EUR04FT003.eop-eur04.prod.protection.outlook.com])

   ----- Transcript of session follows -----
... while talking to outlook-com.olc.protection.outlook.com.:
>>> MAIL From:<?????? .at. idefix.net> SIZE=4837 BODY=8BITMIME
<<< 550 5.7.1 Unfortunately, messages from [45.83.232.134] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [HE1EUR04FT003.eop-eur04.prod.protection.outlook.com]
554 5.0.0 Service unavailable

Trying to get my IPv4 address allowed didn't work. The form for getting IP addresses whitelisted did not allow for IPv6 addresses, but then again outlook.com has no IPv6 addresses listed for its MX record. I would think microsoft would do something with IPv6 to support innovations in Internet but I guess they only do that to win contracts.

After a while I got a response with a ticket number and an hour later a response that looked like maybe a person had taken a look at it, with "Our investigation has determined that the above IP(s) do not qualify for mitigation." So that leaves me with possible mails from outlook[.]com that I can't answer, making me look bad because I don't seem to reply at all.

I'm convinced the mail setup is correct on my end. The domain idefix.net has an SPF record and the mail was sent out via the approved route.

The only solution I can think of at the moment is blocking mail from outlook.com at the protocol level with an error message pointing at a webpage what the problem is, so when someone sends an e-mail from outlook[.]com to one of my domains they will get an error message with an embedded hint what they should do, namely We cannot reply to your mail, please send us mail from a different domain, see https://idefix.net/mailreject.html for an explanation. About the same as microsoft does, although the careful reader might have noticed the error code S3150 is not mentioned at http://mail.live.com/mail/troubleshooting.aspx#errors.

I needed a virtual machine with ubuntu so I did the base installation and also configured unattended-upgrades and sendmail to get the results. But I noticed after a while I never saw any mail from that machine.

Problem soon found:

mailer=relay, pri=30131, relay=postbode.idefix.net. [82.95.196.202], dsn=4.0.0, stat=Deferred: Connection timed out with postbode.idefix.net.

The machine wasn't even trying to reach the mailserver over IPv6! On the internal network with servers it will fail over IPv4 because of the portforwarding rule for the port from the outside IP to the mailserver but I never expected an internal machine to try IPv4.

Somehow this seems default for sendmail 8.15.2 in Ubuntu 20.04. I could find someone else asking this: No IPv6 outbound from Sendmail starting with 20.04 but no answers how/why.

At first I suspected systemd-resolved as the old saying goes that all sendmail problems are caused by DNS. But disabling that didn't fix the problem.

I now have the IPv6 address hardcoded in the configuration, that works.

dnl FEATURE(`msp', `postbode.idefix.net', `25')
FEATURE(`msp', `[2001:980:14ca:1::23]', `25')

I also found out the option ResolverOptions=+WorkAroundBrokenAAAA was set but not causing this.

I wanted to get an idea whether the network over the fiber optic transceivers is reliable. So at the moment our dining room table looks like a network lab.

For testing networks there is iperf. I found out the Raspberry Pi 3B+ can't keep up with 100 Mbit/second UDP packets, so I searched for a speed where the Pi performs ok. This turns out to be 30 mbit, at higher speeds there is packet loss. I also had to reduce packet size to avoid fragmentation which costs CPU. I use IPv6 because that's what I'm used to. It turned out later the maximum speed without loss is higher with IPv4 than with IPv6.

Server on the raspberry pi:

koos@raspberrypi:~ $ iperf --version
iperf version 2.0.9 (1 June 2016) pthreads
koos@raspberrypi:~ $ iperf -s -V -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size:  160 KByte (default)
------------------------------------------------------------

Test without fiber optic transceivers in the path. Layer 2 route: virtual machine - host machine - utp - network switch - utp - network switch - utp - raspberry pi

koos@wozniak:~$ iperf --version
iperf version 2.0.9 (1 June 2016) pthreads
koos@wozniak:~$ iperf -V -u -b30M -i 10 -t 120 -M 10 -l 1400 -c ..
------------------------------------------------------------
Client connecting to .., UDP port 5001
Sending 1400 byte datagrams, IPG target: 373.33 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
..
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec
[  3] Sent 321430 datagrams
[  3] Server Report:
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec   0.004 ms    0/321430 (0%)

Test with fiber optic transceivers in the path. Layer 2 route: virtual machine - host machine - utp - network switch - utp - network switch - utp - fiber optic transceiver - fiber - fiber optic transceiver - utp - raspberry pi

koos@wozniak:~$ iperf -V -u -b30M -i 10 -t 120 -M 10 -l 1400 -c ..
------------------------------------------------------------
Client connecting to .., UDP port 5001
Sending 1400 byte datagrams, IPG target: 373.33 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec
[  3] Sent 321430 datagrams
[  3] Server Report:
[  3]  0.0-120.0 sec   429 MBytes  30.0 Mbits/sec   0.007 ms    0/321430 (0%)

Trying with IPv4 shows that packet loss starts to occur above 45 mbit. This is an interesting difference.

But the important conclusion is that there is no packet loss over the fiber path. There may be a bit more latency, but that's not a surprise. As a last test I looked at purely ping traffic using IPv6.

Without fiber in the path:

koos@wozniak:~$ ping -c 100 -i 0.2 -q ..
PING ..(.. (2001:xxxx)) 56 data bytes

--- .. ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 20192ms
rtt min/avg/max/mdev = 0.567/0.680/0.866/0.063 ms

With fiber in the path:

koos@wozniak:~$ ping -c 100 -i 0.2 -q ..
PING ..(.. (2001:xxxx)) 56 data bytes

--- .. ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 20191ms
rtt min/avg/max/mdev = 0.625/0.738/0.828/0.046 ms

This also shows a bit more latency over fiber.

The extra latency is probably due to the fiber optic transceivers containing a network switch.

I have one Apache server exposed to the outside world for IPv6 clients (because of a history in hostnames going back to the 20th century). So after enabling OCSP for haproxy I decided to have a look at OCSP stapling for Apache 2.4. That's even easier than haproxy since Apache 2.4 will fetch the ocsp data itself. I followed Apache 2.4 SSL/TLS Strong Encryption: How-To OCSP Stapling and it works.

So now the current score at the Qualys SSL server test for koos.idefix.net is A+ both via IPv4 and IPv6.

I decided to have a look whether I can set up the static routes like those needed to get ads-b data out to plane finder via the dhcp server. This works a lot better than having to set those routes by hand after a reboot.

This can be done with the rfc3442 classless static routes extension in DHCP, which isn't supported out of the box by isc dhcpd. But there is support in the dhclient configuration on raspbian, so I only had to add the server side.

All the samples I could find for adding this to the server side added arrays of bytes which is harder to read/comprehend. I had a look at the dhcp-options manpage which showed the option to add a structured record with IPv4 addresses.

Main configuration adding the option:

option rfc3442-classless-static-routes code 121 = array of { integer 8, ip-address, ip-address };
# netmask bit count, destination, via

Specific host configuration using the option with the current address for pfclient-upload.planefinder.net. Which may change!

        host joy {
            hardware ethernet b8:27:eb:ae:ad:47;
            option rfc3442-classless-static-routes 32 80.84.58.2 10.42.2.1;
        }

This pushes route to 80.84.58.2/32 via 10.42.2.1.

Hosts that get this option via dhcp should ignore the default router option so if you need that too you will need to add a route for 0.0.0.0/0. In my specific usecase I don't want to set a default IPv4 route.

I have a Synology NAS at home running DSM, so I had a look at the certificate options. According to the documentation it can get a LetsEncrypt certificate so I tried that. And it worked... which wasn't what I expected.

Some testing later found out port 80 tcp was open for every IPv6 address at home. That's now fixed and limited to those few IPv6 addresses that need to be reachable from the outside world.

Browsing the opinions about allowing outside access to the webserver on the Synology versus not allowing it showed me some differing opinions, but an article listening some malware and ransomware targetting Synology systems made me decide to close the system. Looking at the nginx configuration on the Synology gives me the idea some of the web-accessible functionality is available via port 80.

Several of the machines here at home have IPv4 to the outside world disabled, simply to find every ancient service or program that still lives in the old world. Today I found one of those while installing dehydrated to automatically renew Let's Encrypt certificates.

Indeed, github has no IPv6 support. It tries to be a modern service, but lacks an AAAA record.

The solution is simple: use a webproxy to solve this. The only reason I still have a squid webproxy running is to be able to access IPv4-only http/https services from those hosts, so setting the http proxy in the global git config helped. I'm just surprised github doesn't support IPv6.

Update: After some searching I found Github users have been asking about IPv6 connectivity since at least 2018 and the "solution" is that they currently don't support IPv6 and the request is on some list.

Trying to get devuan updates, I see:

Err:5 http://nl.mirror.devuan.org/merged ascii Release
  404  Not Found [IP: 2001:878:346::116 80]
Err:6 http://nl.mirror.devuan.org/merged ascii-security Release
  404  Not Found [IP: 2001:878:346::116 80]
Err:7 http://nl.mirror.devuan.org/merged ascii-updates Release
  404  Not Found [IP: 2001:878:346::116 80]

While nl.mirror.devuan.org has no shortage of IPv6 and IPv4 addresses:

;; ANSWER SECTION:
nl.mirror.devuan.org.   78083   IN      CNAME   deb.devuan.org.
deb.devuan.org.         78083   IN      CNAME   deb.roundr.devuan.org.
deb.roundr.devuan.org.  845     IN      AAAA    2001:638:a000:1021:21::1
deb.roundr.devuan.org.  845     IN      AAAA    2a01:4f8:140:1102:2b76:955d:b48f:bdf3
deb.roundr.devuan.org.  845     IN      AAAA    2001:878:346::116
deb.roundr.devuan.org.  845     IN      AAAA    2a01:4f8:162:7293::14
deb.roundr.devuan.org.  845     IN      AAAA    2800:a8:c001::a
deb.roundr.devuan.org.  845     IN      AAAA    2a01:4f9:2a:fa9::2
deb.roundr.devuan.org.  845     IN      AAAA    2001:590:3803::31:151
deb.roundr.devuan.org.  845     IN      AAAA    2001:4ca0:4300::1:19
deb.roundr.devuan.org.  845     IN      AAAA    2a02:2a38:1:400:422a:422a:422a:422a
deb.roundr.devuan.org.  845     IN      AAAA    2a0a:e5c0:2:2:400:c8ff:fe68:bef3

;; ANSWER SECTION:
nl.mirror.devuan.org.   78063   IN      CNAME   deb.devuan.org.
deb.devuan.org.         78063   IN      CNAME   deb.roundr.devuan.org.
deb.roundr.devuan.org.  824     IN      A       46.4.50.2
deb.roundr.devuan.org.  824     IN      A       130.225.254.116
deb.roundr.devuan.org.  824     IN      A       190.64.49.124
deb.roundr.devuan.org.  824     IN      A       31.220.0.151
deb.roundr.devuan.org.  824     IN      A       200.236.31.1
deb.roundr.devuan.org.  824     IN      A       131.188.12.211
deb.roundr.devuan.org.  824     IN      A       141.84.43.19
deb.roundr.devuan.org.  824     IN      A       37.187.111.86
deb.roundr.devuan.org.  824     IN      A       5.196.38.18
deb.roundr.devuan.org.  824     IN      A       95.216.15.86
deb.roundr.devuan.org.  824     IN      A       185.38.15.81

I always get the error for 2001:878:346::116 when connecting. This site seems to have a problem with the devuan mirror at the moment, so I'd like to use another one, but apt keeps going back to the same source. This has to do with IPv6 address destination selection (RFC 3484 / RFC 6724).

A good explanation at IPv6 Destination Address Selection – what, why, how - Karl Auer with:

Rule 9, “use longest matching prefix“, will prefer the candidate destination address that shares the greatest number of contiguous leading bits with the source address that would be chosen for it. Such an address is likely to be topologically closer to the source address.

Indeed that address is close to my home network addresses:

2001:0878:0346:0000:0000:0000:0000:0116
2001:0980:14ca:0001::/64

So the "roundr" round robin isn't very round for IPv6 users.

Workaround: reject the address that is giving me problems:

# ip -6 route add unreachable 2001:878:346::116
# apt update
Get:1 http://nl.mirror.devuan.org/merged ascii InRelease [25.6 kB]
Get:2 http://nl.mirror.devuan.org/merged ascii-security InRelease [25.6 kB]
Get:3 http://nl.mirror.devuan.org/merged ascii-updates InRelease [25.6 kB]
Get:5 http://nl.mirror.devuan.org/merged ascii-security/main Sources [185 kB]
Hit:4 http://packages.roundr.devuan.org/merged ascii InRelease
Get:6 http://nl.mirror.devuan.org/merged ascii-security/main amd64 Packages [480 kB]