News items for tag security - Koos van den Hout

On my to-do list was the idea to look at OCSP stapling for haproxy. OCSP is Online Certificate Status Protocol which wraps the revocation status of a certificate in the certificate negotiation. This speeds up the TLS setup a bit since the client doesn't have to make an extra connection to the OCSP responder of the certificate issuer and it adds a bit of privacy because the certificate issuer doesn't see which client requests the status of a certificate.

Finding the right way to get the ocsp updates to haproxy was a bit of work, eventually made some modifications to the script in HAProxy OCSP stapling. I also used the remarks in OCSP stapling with HAProxy. From pitfall to euphoria because I saw the "OCSP single response: Certificate ID does not match any certificate or issuer" error message. I had to restart haproxy first to make it enable ocsp processing (because now each server certificate has its own .ocsp file) and now it accepts the "set ssl ocsp-response" command.

Update: I'm not completely happy yet: after a certificate was renewed haproxy complained about the .ocsp file being out of date. Which is fully correct, since that .ocsp file was about a previous version of the certificate. This needs more work. Ideally I would check the validity of the .ocsp file before deciding to renew it. And fetch the new ocsp data before reloading a renewed certificate.

Anyway, the 'TLS setup' part of connecting to sites like idefix.net goes from 20-21 milliseconds to 5-8 milliseconds. Not a blinding fast improvement but all bits help and I like to have optimal security and privacy.

I had a look at some weblogs and after removing the entries caused by webbots most of the rest of the traffic was attacks. All on stuff I don't have (usually wordpress), but one thing was noticeable:

37.59.47.61 - - [13/Oct/2020:00:17:34 +0200] "GET ////nette.micro?callback=shell_exec&cmd=ifconfig HTTP/1.1" 404 747 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
37.59.47.61 - - [13/Oct/2020:00:17:41 +0200] "GET /////nette.micro?callback=shell_exec&cmd=ifconfig HTTP/1.1" 301 715 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"
37.59.47.61 - - [13/Oct/2020:00:17:43 +0200] "GET /nette.micro?callback=shell_exec&cmd=ifconfig HTTP/1.1" 404 747 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"

From what I've found about the 'nette microframework' there are callbacks, but none of those is called shell_exec.

Yesterday I read about changes at LetsEncrypt that influence LetsEncrypt intermediate certificates and DANE and had a look at my own DANE record set up in december 2019.

I decided to change the 'usage' value to 1, meaning 'EE match validated by public CA' because it's linked to a known public CA, and the old value 3 meaning 'private EE' wasn't completely true because it's linked to a known public CA.

But I received a notification this morning, with:

Only certificate usages DANE-TA(2) and DANE-EE(3) are supported with SMTP.

With references to rfc 7672 section 3.1.1 and further which makes a valid point about CA validation in SMTP sessions.

So the validation chain is purely based on DNSSEC.

After mention of the internet.nl tests at work I tested my webserver with the test from internet.nl and got a failed for the cipher order test. I do have the 'best' configuration according to the Mozilla SSL Configuration Generator but the test at internet.nl disagrees on this point because of the ordering of the ciphers. So with a lot of checking I now have:

ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256

Which is not the order Mozilla suggests, but gives me an A+ on the Qualys SSL Server test and a good result on the standards test at internet.nl.

I also found out generating my own Diffie-Hellman parameters is not good for parameter sizes of 2048 bits and up. I changed to a known-good group of 4096 bits.

Sometimes hobby and work intertwine when I'm not expecting it.

I set up a domainname and added a dummy website for something related to amateur radio. I have no idea if it will go anywhere, but I thought I'd get the web configuration right. The domain name isn't published anywhere.

But, to my surprise:

178.174.174.11 - - [20/May/2020:09:14:35 +0200] "GET /.git/HEAD HTTP/1.0" 404 594 "-" "-"
178.174.174.11 - - [20/May/2020:09:14:35 +0200] "GET /.git/HEAD HTTP/1.0" 404 594 "-" "-"
178.174.174.11 - - [20/May/2020:09:14:53 +0200] "GET /.git/HEAD HTTP/1.0" 404 594 "-" "-"
178.174.174.11 - - [20/May/2020:09:14:53 +0200] "GET /.git/HEAD HTTP/1.0" 404 594 "-" "-"
81.92.203.216 - - [20/May/2020:09:15:12 +0200] "GET /.git/HEAD HTTP/1.0" 404 594 "-" "-"
2a00:d680:30:50::67 - - [24/May/2020:16:54:36 +0200] "GET /wp-login.php HTTP/1.1" 404 594 "http://******.*******.**/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"

I added the domain name and requested a LetsEncrypt certificate on 11 May 2020, I set up the webserver correctly on 19 May 2020. The only 'publication' of the name is via the certificate transparancy log. Somehow this is enough for the first probes for possible security issues.

Looking in the haproxy logs finds even more requests on 15 and 18 May 2020. Part of the requests are via http, not https.

Keeping encryption settings correct needs a lot of testing to make sure things are right. With external-facing webservices this is easy with the Qualys SSL scan, but for other services than https or services not facing outward a local tester is needed. This local tester is sslscan, a commandline tool but which depends on the shared openssl libraries which have insecure protocols disabled to helps disabling those deprecated protocols.

But to test services the client needs to support those old protocols to do the test correctly.

So I built a static version of sslscan with static openssl using the instructions at https://github.com/rbsec/sslscan. And that works for the full testing range!

I have a Synology NAS at home running DSM, so I had a look at the certificate options. According to the documentation it can get a LetsEncrypt certificate so I tried that. And it worked... which wasn't what I expected.

Some testing later found out port 80 tcp was open for every IPv6 address at home. That's now fixed and limited to those few IPv6 addresses that need to be reachable from the outside world.

Browsing the opinions about allowing outside access to the webserver on the Synology versus not allowing it showed me some differing opinions, but an article listening some malware and ransomware targetting Synology systems made me decide to close the system. Looking at the nginx configuration on the Synology gives me the idea some of the web-accessible functionality is available via port 80.

I think I found the most probable reason of the earlier problem with DNSSEC signed subzones. I was trying this with a domain for which I don't have control over one of the secondary nameservers.

In one of my showerthought moments I decided to try another domain where I have that full control (just less nameservers) and was able to make it all validate correctly after some tries. Forgetting one or more of all the steps needed to correctly create a domain with DNSSEC and getting the delegation right will give errors.

So I guess running a nameserver with all DNSSEC options disabled hinders validation.

I noticed the news about LetsEncrypt revoking a lot of certificates on 4 March 2020 and did some checking to find out eventually that one of my certificates is in that set. Users have been notified of this problem... when their account had a contact e-mail address. By default dehydrated doesn't set an e-mail address so none of my instances used one. I do like to get informed so I searched how to update the contact info. The data is in /etc/dehydrated/config field CONTACT_EMAIL but I needed some searching before I found the method to get the update passed on to LetsEncrypt.

Some searching later found Update registration email address - Issue #425 dehydrated which shows that a simple dehydrated --account does the magic.

A few days ago I changed the configuration of haproxy to stop accepting TLSv1.0 and TLSv1.1. With the upcoming deprecation of TLSv1.0 and TLSv1.1 this seemed the right SSL configuration. Today I remembered there is one directly reachable Apache server, so I had a look at the settings there and checked the results with the Qualys SSL Labs SSL Server test where I noticed some ciphers listed as 'weak'. And seeing different results between my haproxy and apache servers, which I did not expect as I used the same settings for SSLCipherSuite in Apache and ssl-default-bind-ciphers in haproxy.

The last issue was caused by the fact that Apache2.4.25 in Devuan ascii uses libssl 1.0.2 and haproxy 1.7.5 uses libssl 1.1.0. I'm not sure that's an ideal configuration but it's what I work with.

With the output of openssl ciphers -v I get a list of cipher names. But this is with libssl1.1.0 so the output lists ciphers that Apache doesn't have access to (yet). The good part is that Apache ignores ciphers that aren't available, so the net result is a running and working configuration.

The current result is for Apache 2.4.25:

SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLHonorCipherOrder on
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256

And for haproxy 1.7.5:

ssl-default-bind-options force-tlsv12 no-tls-tickets
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256

The fun part is that I can test the SSL negotiation with sslscan locally but sslscan is linked against openssl 1.0.2 so it misses some of the newer options. And I also test with the Qualys SSL Labs ssl test but that takes a while.

The too long; didn't read version of finding the right configuration options

And later I found I could have saved a lot of time researching options using the Mozilla SSL Configuration Generator. I don't completely agree with the suggestions there because I want to generate my own dhparams. Using 'well-known Diffie-Hellman paramaters' has security risks. But otherwise all the suggestions for ciphers are very usable and save me a lot of time.

In 2020 the support for TLSv1.0 and TLSv1.1 will end so the famous qualys SSL test is giving capped grades. I decided to get with the times and limit my outside web ports to TLSv1.2 so now I am back at an A+ grade.

Eventually this will start to cause problems as Devuan stable doesn't have an openssl with TLSv1.3 support yet.

Suricata is running and detecting attacks, but it was causing a 100% cpu load after a restart of the ppp connection (the DSL here uses PPP over Ethernet).

The errors point at the problem starting when the ppp connection restarts:

21/1/2020 -- 00:59:36 - <Error> - [ERRCODE: SC_ERR_AFP_READ(191)] - Error reading data from iface 'ppp0': (100u) Network is down
21/1/2020 -- 00:59:37 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't set fanout mode, error Invalid argument

Which also starts to fill the system log with:

Jan 21 00:59:42 xxxxxxxx kernel: [11347441.726755] device ppp0 left promiscuous mode
Jan 21 01:00:13 xxxxxxxx kernel: [11347472.055712] device ppp0 entered promiscuous mode
Jan 21 01:00:13 xxxxxxxx kernel: [11347472.071533] device ppp0 left promiscuous mode
Jan 21 01:00:13 xxxxxxxx kernel: [11347472.091653] device ppp0 entered promiscuous mode

The interesting part is that this causes higher power usage about five and a half hours later.

Solution: restart suricata in an /etc/ppp/ip-up.d/ script.

Today I needed blocks of random letters to practise sending morse. What better tool to create those blocks than good old pwgen with the right settings:

$ pwgen -0 -A 5 12
ahhud eizaa kuoku ahyoo aequi epiis eiwei eimap sohsh papai ikeit oucho

And the trick for generating groups of five digits is a bit longer:

$ pwgen -r abcdefghijklmnopqrstuvwxyz -A 5 12
97228 85996 98876 38451 06091 98556 53369 73632 29509 29032 89601 16078

Or both letters and digits:

$ pwgen -A 5 12
sa7la oc7ko an5ne axae6 vohz6 aez5i eh3qu sha5m inai8 eor3a fuv1o ro6ha

Use better parameters with pwgen to generate actual passwords.

I tried adding subzones with DNSSEC by adding the DS record to the parent zone, but in both tries I got errors from DNSViz. Different errors even: in one case the signature on the DS record was seen as invalid and in another case there was no signature at all. The errors are reproducable, even after waiting for caches to empty.

I kept seeing warnings about an expired signature when running named-checkzone or dnssec-signzone and it took some searching before I found the reason.

Recently I removed the records with type SPF from my zones since the recommended approach is to use TXT records with SPF data. The RRSIG records for the SPF records were left in the signed zonefile, but not updated so they expired and started to give warnings.

The SPF records were for names that had other data too which seems to trigger this. Removing a record completely (no RRTYPEs left for the name) removes all signatures.

The things in DNSSEC I haven't tested yet are a signed subzone, a ZSK rollover and a KSK rollover. Those will eventually happen too.

Now I have DNSSEC running ok on my domains I can start looking at security innovations that rely on DNSSEC.

The first one is DANE for the mailserver, in which the public key signature is published in DNS record secured with DNSSEC to give a separate path to verify the public key during the SMTP session.

The public key of the mailserver is also signed by LetsEncrypt as described in Automating Let's Encrypt certificates further and Automating Let's Encrypt certificates with DNS-01 protocol so there are two completely independent paths to verify the identity of the mail server.

To find the public key of the mailserver for a given domain:

$ dig +short idefix.net mx
10 postbox.idefix.net.
$ dig +short _25._tcp.postbox.idefix.net tlsa
3 1 2 2B55764A99A47AEC5B66D8EB4E741F2646BF6352CABC9BE3F37D2F42 0BD7EF56B5BE3058E7B10964BA963777364443057E45599E07A82375 7A812F1A7014356A

I found the tlsa tool from package hash-slinger by Paul Wouters to create these records. This can be both from the protocol which has certain risks (if that connection is intercepted) or from the public key file. Or via the web tool Generate TLSA Record by Shumon Huque.

TLSA records are generically linked to a TCP or UDP port. The next step will probably be to start adding records for other public services with TLS like https. There was a time that some people were convinced DANE was going to replace certificate authorities for https, but at this moment it is very limited. I have added TLSA records for https (tcp/443) for camp-wireless.com and www.camp-wireless.com for now and I'm testing with these. For now one of my favourite checkers isn't convinced.

This does increase the chances for things to go wrong. With the tlsa program it is possible to verify records too, so I can use this to verify TLSA records.

$ tlsa --verify -6 --starttls smtp --port 25 postbox.idefix.net
SUCCESS (Usage 3 [DANE-EE]): Certificate offered by the server matches the TLSA record (2001:980:14ca:1::23)

Although this certificate is a valid LetsEncrypt certificate, DNS-based Authentication of Named Entities (DANE) does not support usage 1 (check the certificate public key and verify certificate chain to a known root) for SMTP with STARTTLS, so it is usage 3 (just check the certificate public key). The tlsa program does not check this specifically, but the web checker at DANE TLSA Server checker found the issue, so I corrected that.

I use selector 1 to just check the public key because the complete certificate changes with every LetsEncrypt renewal. My choice for mtype 2 (sha512) is just a wish for a strong hashing algorithm.

This also makes the link between service configuration and DNS contents a lot stronger. Maybe this needs secure automated updates.

Met alle tips voor het maken van veilige wachtwoorden en die alleen beschikbaar hebben vanuit een wachtwoordmanager loop ik nu tegen websites aan die vragen om een wachtwoord maar vervolgens moet je dat wachtwoord ineens op een fysiek andere plek dan achter je eigen computer intikken.

De eerste keer dat ons dat overkwam was bij een camping van staatsbosbeheer op Ameland. We hadden ons via de website ingeschreven en bij het aanmelden op de camping zelf bleek er een aanmeldscherm te zijn waar je met e-mail adres en wachtwoord moest inloggen. Maar we gebruiken voor dat soort websites altijd gegenereerde wachtwoorden die we niet weten. Met veel zoeken naar de hoek van de camping met een beetje mobiele data dekking konden we bij onze wachtwoordkluis en konden we het wachtwoord opzoeken. Want het aanmeldscherm is omdat de beheerder van de camping er ook maar een uur per dag is.

De tweede keer was bij de bibliotheek in Utrecht. Als je daar in de bibliotheek zelf een reservering wilt maken moet je ook inloggen op een computer met gebruikersnaam en wachtwoord. En ook daar konden we het niet snel opzoeken, maar daar konden ze ons helpen aan de hand van de bibliotheekpas.

Some things noticed by Suricata IDS are amusing to me. When looking at lines like:

11/15/2019-13:14:35.001691  [**] [1:2402000:5363] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.156.73.11:46843 -> 82.95.196.202:41505
11/15/2019-13:15:06.794357  [**] [1:2402000:5363] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 88.214.26.102:42131 -> 82.95.196.202:8703
11/15/2019-13:15:06.794357  [**] [1:2403384:53195] ET CINS Active Threat Intelligence Poor Reputation IP group 85 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 88.214.26.102:42131 -> 82.95.196.202:8703
11/15/2019-13:15:20.065796  [**] [1:2403393:53195] ET CINS Active Threat Intelligence Poor Reputation IP group 94 [**] [Classification: Misc Attack] [Priority: 2] {UDP} 93.174.95.106:27221 -> 82.95.196.202:7
11/15/2019-13:15:32.845110  [**] [1:2402000:5363] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.156.73.21:44503 -> 82.95.196.202:43935
11/15/2019-13:16:23.399397  [**] [1:2402000:5363] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 185.175.93.27:58989 -> 82.95.196.202:53166

All 'Dshield Block Listed' and 'Poor Reputation IP' traffic is port scans to ports that are blocked. So it's not a surprise those IPs have a poor reputation or a Dshield listing.

After hearing about intrusion detection systems a few times I decided to give one a try at home. Although a lot of attacks are blocked I sometimes see weird attacks and it would be nice to have a better idea of what exactly the attack was.

Yes, I have weird interests sometimes. I'm glad I have an ISP (xs4all) where I can select the option 'give me the completely unfiltered Internet connection' so I even see SMB protocol attempts.

I first tried 'snort' but that doesn't deal with PPP interfaces by default. It can be recompiled to accept those but I did not want that. The next option I heard about is 'Suricata' which is running at the moment.

I was amused by the reports of DDoS-like NTP traffic. Those are caused by the NTP statistics gathering. I know NTP can be abused for generating DDoS traffic but all security reports about NTP servers I manage have been false positives.

Anyway it's running and complaining a lot about the traffic it sees. For example the IPv6 port scan/network mapping attempts I noticed two months ago are still active.

11/13/2019-15:06:59.703451  [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 240e:00f7:4f01:000c:0000:0000:0000:0003:6050 -> 2001:0980:14ca:0001:020d:56ff:fece:ffe1:5901
11/13/2019-15:08:39.645780  [**] [1:2002911:6] ET SCAN Potential VNC Scan 5900-5920 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 240e:00f7:4f01:000c:0000:0000:0000:0003:5167 -> 2001:0980:14ca:0001:020d:56ff:fece:ffe6:5901

It seems the TCP reflective SYN attacks are continuing. In researching my options I saw the option to use a netmask with the iptables recent module.

This helps a bit with the attacks trying to flood an entire block. I've updated the filtering to work by the /24, start a check on a SYN from such a block, end when an ACK flies by and start dropping when the rate is over 10 per 2 minutes.

iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,ACK SYN -m recent --update --seconds 120 --hitcount 10 --name tcpsyn --mask 255.255.255.0 --rsource -j LOGDROP
iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,ACK SYN -m recent --set --name tcpsyn --mask 255.255.255.0 --rsource
iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,ACK ACK -m recent --remove --name tcpsyn --mask 255.255.255.0 --rsource

LOGDROP is a rule to drop packets and ratelimit the logging of dropped packets, to avoid turning a network attack into a disk attack.

But I have to be careful not to make services hard to reach for legitimate clients. The above is working, and during attacks I don't see a single SYN_RECV socket.