News items for tag homeserver - Koos van den Hout

2020-07-30 Backup to a remote webdav server using rclone 4 days ago
After the earlier issues with backing up to a remote webdav server I let the problem rest but made sure my backups were in order from time to time.

Until I came across a mention about rclone which especially mentions copying to various cloud services. Since I am trying to backup to a webdav server based on owncloud I had a look and this is a supported configuration in rclone. So I installed rclone to give it a try.

From the devuan distribution I got rclone version 1.35 which seemed to have problems with the specific owncloud server. So I had a look and newer .deb packages are available on the Rclone download page. This worked better.

On the first run rclone was convinced a lot of the files were modified locally since I transfered them with fusedav+rsync, so those were refreshed. But now it is all synchronized correctly the changes are minimal and the runtime isn't very long. I do make sure my uplink isn't filled completely so I limit the bandwidth. Command:
$ rclone --bwlimit 1M -v sync /camera/ owncloudservice:backuptest/camera/

Tags: , ,
2020-07-16 Time to grow the diskspace for the home server 2 weeks ago
There were some ideas for one or more new virtual machines in the homeserver conway 2017 and the current volume group is almost full. Time to order some new diskspace because there's also some upcoming Devuan upgrades where I'd like to keep a snapshot of the 'before' situation so I can go back if everything breaks.

So I ordered 2 960 Gb SSDs. They will run in a mirror anyway. I was wondering whether to add them to the current volume group or take the 2 256 Gb SSDs out of the volume group. I decided to take those two out: there will be enough space after the upgrade and it can save some power. This does mean the new SSDs will also be set to be bootable and I will have to do a move of the volume group.

The order of changes:
  • Shut down machine
  • Install 2 new disks
  • Boot up machine
  • Partition 2 new disks with boot partition, make bootable with UEFI
  • Test boot from new disk
  • Make raid-1 device from the rest of the space on both disks
  • Add new raid-1 to volume group
  • Move volume group away from old raid-1
  • Remove old raid-1 from volume group
  • Unlink old raid-1
  • Shut down machine
  • Remove 2 old disks
  • Boot up again
Quite a number of steps, this will take some time.
Read the rest of Time to grow the diskspace for the home server

Tags: , ,
2020-05-14 After years of rants, Windows can still surprise me in a positive way 2 months ago
Windows 10 discovering CUPS printers Microsoft Windows does fall straight into the "does not work well with others" category for me, but today Windows 10 on my work laptop managed to give me a positive surprise.

I wanted to print something at home, and my home network is set up to publish CUPS printers via multicast DNS, both via IPv4 and IPv6 so Linux devices on the network see the printer right away. On selecting "Add a printer" in Windows 10 it just showed me the main home printer as an option and sending something to the printer worked the first time. I did notice the default paper size was still Letter although I have set up A4 everywhere, so that was the only thing left to adjust.

Now for the screenshot I removed the printer and tried to add it again and I notice the availability isn't very consistent. I do see a lot of mdns traffic when I start adding a printer!

Tags: , ,
2020-05-05 Internal documentation of my home network 3 months ago
A few times I had to lookup something again about the way things work in my setups. I made a remark before that I should set up a documentation wiki at home to keep this information somewhere central.

Right before I started with the homeserver conway I set up Mediawiki on a webserver. First on the previous homeserver greenblatt but as soon as web production was migrated to the new server I ran it on the web production server virtual machine.

So for a lot of 'how did I' questions there are answers, and some future plans. Also for plans on the house and on amateur radio related things.

People who know me from work will just say this is an extension of the trail of MediaWiki based documentation systems I left behind, and they are right.

Tags: , ,
2020-04-29 Seeing when it's time to walk to the laserjet printer 3 months ago
I have an aged laserjet 4100 DTN printer at home and it sometimes takes a while to print something. The logs from cups will state that it has been sent to the printer but the printer will still show processing.

Solution: ask the printer for the active pagecounter. This will be updated after the page has really been output, so it will only change when the printer is done with the page.
$ snmpget -v1 -c internal laserjet
iso. = Counter32: 738042

Tags: , ,
2020-04-04 Found the probable reason of the DNSSEC subzones problem 4 months ago
I think I found the most probable reason of the earlier problem with DNSSEC signed subzones. I was trying this with a domain for which I don't have control over one of the secondary nameservers.

In one of my showerthought moments I decided to try another domain where I have that full control (just less nameservers) and was able to make it all validate correctly after some tries. Forgetting one or more of all the steps needed to correctly create a domain with DNSSEC and getting the delegation right will give errors.

So I guess running a nameserver with all DNSSEC options disabled hinders validation.

Tags: , ,
2020-03-08 Updating the Fritz!box 7360v1: still no PPPoE passthrough 4 months ago
A while ago I noticed a mention of new firmware for the Fritz!box 7360v1. As I want a separate PPPoE process to have full control of my Internet connection I hoped the PPPoE passthrough option would become available, since this would be a firmware version later than 6.30, but no.

At least the upgrade went fine without having to use the recovery options. So the 'in case of emergency' settings have been kept forwarding the necessary ports via IPv4.

Tags: , ,
2020-02-17 Tweaking the SSL cipher settings for 2020 5 months ago
Encrypt all the things meme A few days ago I changed the configuration of haproxy to stop accepting TLSv1.0 and TLSv1.1. With the upcoming deprecation of TLSv1.0 and TLSv1.1 this seemed the right SSL configuration. Today I remembered there is one directly reachable Apache server, so I had a look at the settings there and checked the results with the Qualys SSL Labs SSL Server test where I noticed some ciphers listed as 'weak'. And seeing different results between my haproxy and apache servers, which I did not expect as I used the same settings for SSLCipherSuite in Apache and ssl-default-bind-ciphers in haproxy.

The last issue was caused by the fact that Apache2.4.25 in Devuan ascii uses libssl 1.0.2 and haproxy 1.7.5 uses libssl 1.1.0. I'm not sure that's an ideal configuration but it's what I work with.

With the output of openssl ciphers -v I get a list of cipher names. But this is with libssl1.1.0 so the output lists ciphers that Apache doesn't have access to (yet). The good part is that Apache ignores ciphers that aren't available, so the net result is a running and working configuration.

The current result is for Apache 2.4.25:
SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLHonorCipherOrder on
And for haproxy 1.7.5:
ssl-default-bind-options force-tlsv12 no-tls-tickets
The fun part is that I can test the SSL negotiation with sslscan locally but sslscan is linked against openssl 1.0.2 so it misses some of the newer options. And I also test with the Qualys SSL Labs ssl test but that takes a while.

The too long; didn't read version of finding the right configuration options

And later I found I could have saved a lot of time researching options using the Mozilla SSL Configuration Generator. I don't completely agree with the suggestions there because I want to generate my own dhparams. Using 'well-known Diffie-Hellman paramaters' has security risks. But otherwise all the suggestions for ciphers are very usable and save me a lot of time.

Tags: , , ,
2020-02-10 Getting with the times and limiting the webserver to TLSv1.2 5 months ago
In 2020 the support for TLSv1.0 and TLSv1.1 will end so the famous qualys SSL test is giving capped grades. I decided to get with the times and limit my outside web ports to TLSv1.2 so now I am back at an A+ grade.

Eventually this will start to cause problems as Devuan stable doesn't have an openssl with TLSv1.3 support yet.

Tags: , , ,
2020-01-30 Backup to a remote webdav server, first success! 6 months ago
I found a completely different option for transferring files from linux to a remote webdav filesystem: fusedav. Mounting the remote 'cloud' disk with fusedav and synchronizing files with rsync is starting to work.

I decided to split my backups into two categories: first there are file collections that usually only grow, like digital camera pictures and audio project files. This takes the most diskspace and doesn't really need versioning.

The second category is configuration files, homedirs, mail and other things that change and where I may need an older version. This is where backups based on amanda work better.

I mount the filesystem with:
$ fusedav -u koos -p topsecret https://webdav.cloudprovider/remote.php/webdav/ /home/koos/webdavmount/
And the rsync command to backup to this mount:
$ rsync -av --progress --bwlimit=512K --size-only --timeout=30 /camera/2003/ webdavmount/camera/2003/
This looks scriptable so it can run on a regular basis with just a status update to me.

Reliability is still an issue. I added the --timeout=30 parameter to make rsync abort when the bytes stop flowing.
Read the rest of Backup to a remote webdav server, first success!

Tags: , ,
2019-12-24 First tries with DNSSEC on subzones: no success 7 months ago
I tried adding subzones with DNSSEC by adding the DS record to the parent zone, but in both tries I got errors from DNSViz. Different errors even: in one case the signature on the DS record was seen as invalid and in another case there was no signature at all. The errors are reproducable, even after waiting for caches to empty.

Tags: , ,
2019-12-19 Removing an RRTYPE for a DNS name causes an expired RRSIG for that record 7 months ago
I kept seeing warnings about an expired signature when running named-checkzone or dnssec-signzone and it took some searching before I found the reason.

Recently I removed the records with type SPF from my zones since the recommended approach is to use TXT records with SPF data. The RRSIG records for the SPF records were left in the signed zonefile, but not updated so they expired and started to give warnings.

The SPF records were for names that had other data too which seems to trigger this. Removing a record completely (no RRTYPEs left for the name) removes all signatures.

The things in DNSSEC I haven't tested yet are a signed subzone, a ZSK rollover and a KSK rollover. Those will eventually happen too.

Tags: , ,
2019-12-14 Moved the first domain registration to TransIP 7 months ago
The machine moved so I had to do the whole update dance with the glue records again. Since the IPv6 glue records 'vanished' when I added DNSSEC to I decided to move to a different registrar where IPv6 glue records and DNSSEC are normal and don't require an extra support call.

Since I have an account with TransIP anyway for the stack storage service I just had to add (and pay for) domain services. Interesting bit is that TransIP says I have to pay again next year. According to the registry the domain is registered until 11 august 2024 at the moment.

Adding DNSSEC gave problems at first, the format they expect is from the public part of the key signing key, which is a different format from the file which gets generated by dnssec-signzone. After some tries and searching I found the right source and format. The error message was about the Key Tag which was confusing as that is a number where there isn't much to go wrong.

Tags: ,
2019-12-12 Adding the first TLSA records for secured services 7 months ago
Encrypt all the things meme Now I have DNSSEC running ok on my domains I can start looking at security innovations that rely on DNSSEC.

The first one is DANE for the mailserver, in which the public key signature is published in DNS record secured with DNSSEC to give a separate path to verify the public key during the SMTP session.

The public key of the mailserver is also signed by LetsEncrypt as described in Automating Let's Encrypt certificates further and Automating Let's Encrypt certificates with DNS-01 protocol so there are two completely independent paths to verify the identity of the mail server.

To find the public key of the mailserver for a given domain:
$ dig +short mx
$ dig +short tlsa
3 1 2 2B55764A99A47AEC5B66D8EB4E741F2646BF6352CABC9BE3F37D2F42 0BD7EF56B5BE3058E7B10964BA963777364443057E45599E07A82375 7A812F1A7014356A
I found the tlsa tool from package hash-slinger by Paul Wouters to create these records. This can be both from the protocol which has certain risks (if that connection is intercepted) or from the public key file. Or via the web tool Generate TLSA Record by Shumon Huque.

TLSA records are generically linked to a TCP or UDP port. The next step will probably be to start adding records for other public services with TLS like https. There was a time that some people were convinced DANE was going to replace certificate authorities for https, but at this moment it is very limited. I have added TLSA records for https (tcp/443) for and for now and I'm testing with these. For now one of my favourite checkers isn't convinced.

This does increase the chances for things to go wrong. With the tlsa program it is possible to verify records too, so I can use this to verify TLSA records.
$ tlsa --verify -6 --starttls smtp --port 25
SUCCESS (Usage 3 [DANE-EE]): Certificate offered by the server matches the TLSA record (2001:980:14ca:1::23)
Although this certificate is a valid LetsEncrypt certificate, DNS-based Authentication of Named Entities (DANE) does not support usage 1 (check the certificate public key and verify certificate chain to a known root) for SMTP with STARTTLS, so it is usage 3 (just check the certificate public key). The tlsa program does not check this specifically, but the web checker at DANE TLSA Server checker found the issue, so I corrected that.

I use selector 1 to just check the public key because the complete certificate changes with every LetsEncrypt renewal. My choice for mtype 2 (sha512) is just a wish for a strong hashing algorithm.

This also makes the link between service configuration and DNS contents a lot stronger. Maybe this needs secure automated updates.

Tags: , ,
2019-10-17 Tested incremental DNSSEC signing 9 months ago
I noticed some really unused records in one zone which is now DNSSEC signed. For example I still had to point at my Google+ page.

So I removed them and did the signing after increasing the serial number. Indeed the records that had no update kept their original signature and the records that where changed (such as the SOA because of the serial number) were signed with new signatures.

Tags: , ,
2019-10-16 The signatures for the first DNSSEC signed zone expired, and I signed the rest 9 months ago
Today I was reminded of the first zone I signed with DNSSEC and did the check again with DNSViz. And I saw a lot of error messages. Some searching found that I let all the signatures expire (after the default time of 30 days).

Solution: re-sign the zone and have a careful look at when I need to sign the zones again. Officially just in time for expiry time of the signature (default 30 days) minus TTL of the record.

Obviously this process has to be automated. In the first go I decided to force new signatures after 21 days. But I tested some things later and decided to go for more regular checks of the ages of the signatures and refresh the signatures that are about to expire. This is usually reserved for 'big' zones with lots of resolvers querying them but I decided to implement this myself to avoid problems, and learn more about DNSSEC.

The magic signing command is now:
    named-checkzone $* $^
    ./ $^
    dnssec-signzone -S -K /etc/bind/keys -g -a -r /dev/random -D -S -e +2592000 -i 604800 -j 86400 -o $* $^
    rndc reload $*
    touch $@
The expiry is set with -e at 30 days, the checkinterval with -i at 7 days and the jitter factor with -j at 1 day.

Now there is a special part in the Makefile to be called from cron on a regular basis. It won't produce any output when there is nothing to update.
    @for zone in $(SIGNEDZONES); do if [ `find $${zone}-signedserial -mtime +7 -print` ]; then touch $${zone}-zone ; $(MAKE) --no-print-directory $${zone}-signedserial; fi ;done
The Make variable SIGNEDZONES is filled with the zonenames of the zones that have to be kept DNSSEC signed. File structure for each forward zone is as listed in first zone with valid DNSSEC signatures.

So now almost all my domains are DNSSEC signed. A learning experience and a good level of security.

Tags: , , ,
2019-09-11 First zone with valid DNSSEC signatures 10 months ago
My previous test with DNSSEC zone signing showed a problem with entropy in virtual machines. Today I had time to reboot the home server running the virtual machines including the virtual machine with the nameserver, based on bind9.

Now I can create DNSSEC signatures for zonefiles at high speed (0.028 seconds) with enough entropy available. My first test is with which is a domainname for redirecting to Camp Wireless but since that variant was mentioned somewhere I had to generate the redirects to the right version.

The next step was to upload the DS records for the zone to my registrar and get them entered into the top level domain. This failed on the first attempt, the DS records have to be entered very carefully at the registrar.

I tested the result with dnsviz for and found an error in the first try: I updated the serial after signing the zone. So the soa record wasn't signed correctly anymore.

I updated my zonefile Makefile to do the steps in the right order:
        named-checkzone $* $^
        ./ $^
        dnssec-signzone -S -K /etc/bind/keys -g -a -r /dev/random -D -S -o $* $^
        rndc reload $*
        touch $@
For the zone the original data is in, the DNSSEC signatures in And make will abort when one of the commands gives an error level, so it will for example stop completely when I make a typo in the zonefile which will make named-checkzone fail. The -D option creates a file to be used with $INCLUDE in the original zonefile. This does create a circular dependency: named-checkzone will fail when the -signedserial file isn't available on the first run. So the first run will have to be manually.

So now the zone is signed correctly. The next developments will be to find out how to monitor this extensively so I won't be surprised by problems and to redo the signing from time to time to make DNSSEC zone walking very hard.

And when I trust all of this I will implement it on other domain names that I manage.
Read the rest of First zone with valid DNSSEC signatures

Tags: , , ,
2019-07-05 I tested the randomness setup 1 year ago
Doing some more reading on haveged made me decide to test the actual randomness of my setup with haveged and randomsound which I created to fix the lack of entropy for dnssec signing operations so I booted the same testing virtual machine which can tap from the host /dev/random. I ran rngtest until it was time to shut down the laptop which was showing the output. The result:
$ rngtest < /dev/random 
rngtest 2-unofficial-mt.14
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

rngtest: starting FIPS tests...
^Crngtest: bits received from input: 4999640
rngtest: FIPS 140-2 successes: 249
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=303.011; avg=543.701; max=5684.774)bits/s
rngtest: FIPS tests speed: (min=43.251; avg=64.587; max=84.771)Mibits/s
rngtest: Program run time: 9194254192 microseconds
I ratelimited the virtio-rng-pci driver from the host, so the test took a really long time. Given earlier tries with dnssec-signzone this is fast enough.

No need to buy a hardware random generator, although they are way cool and it would be an idea to have a source of correctness (NTP) next to a source of randomness.

Update: I ran rngtest on /dev/urandom and I had to ask for a really big load of blocks to get failures. The first test with 249 blocks gave the same result as above, just a lot higher bit rate. So now I know less about the correct randomness of my setup but at least the test shows that I can safely run dnssec-signzone which was the original idea.

Tags: , , ,
2019-07-04 First tests with dnssec show a serious lack of entropy 1 year ago
I was looking at the options for implementing DNSSEC on the domains I have, and started doing this on a domain name that is just used for web redirects, so I won't break anything serious when I make an error. And I am looking at monitoring options at the same time.

Looking for usable documentation I found DNSSEC signatures in BIND named - which shows and explains a lot of the options for doing this with bind9, including full automation. I want to take steps I understand, so I will start with careful minimal automation on a domain name that I can 'break'.

Following that documentation I created a key-signing key (KSK) and a zone-signing key (ZSK). I used the /etc/bind/keys directory which is the standard location.

The first dnssec-signzone action took 54 minutes. After waiting for a bit I started wondering what was happening and it turned out to be a problem with entropy: the signing uses a lot of data from /dev/random. I have the virtio-rng module loaded but the host wasn't making randomness available to the guest operating system. The host server does run randomsound to get more entropy since there is no hardware random number generator available.

Documentation on how to 'forward' randomness from the host to the client virtual machine: Random number generator device - Domain XML format

So I did some tests with a test virtual machine with a similar configuration. The results:
  • Just software kernel rng in the virtual machine: 54 minutes.
  • Offering virtio-rng randomness from the host from /dev/urandom running randomsound: less than 1 second.
  • Offering virtio-rng randomness from the host from /dev/random running randomsound: 11 minutes 10 seconds.
  • Offering virtio-rng randomness from the host from /dev/random running randomsound and haveged: less than 1 second.
Installing haveged which gathers entropy from hardware processes fixes the whole problem.

Now to implement the same settings for the virtual machine running the production nameserver and I'll be able to take the next step.

Tags: , , ,
2019-06-19 Looking at the wrong side of a mirrored disk 1 year ago
Due to recent kernel updates I rebooted the home server and ran into only older kernels available. Some searching later I found out it booted from another disk than the disk the update manager was maintaining /boot on.

The solution was to mirror the /boot partition by hand and change the EFI boot setup to try a boot from both disks, so the machine will still boot when one half of the mirror is completely unavailable. I did buy mirrored disks to have the machine available with one disk unavailable.

Changing the EFI boot setup with efibootmgr was somewhat complicated, but I got it all done. How to add a second disk found via Partitioning EFI machine with two SSD disks in mirror - Unix & Linux stackexchange and understanding the numbers in the efibootmgr -v output via "efibootmgr -v" output question.

The ideal solution would be to have /boot and /boot/efi on mirrored partitions without metadata (so they are readable too from the efi loader as an unmirrored partition). According to what I read this is possible in Linux with devicemapper but there is not a lot of experience shared.

Tags: , ,
  Older news items for tag homeserver ⇒
, reachable as PGP encrypted e-mail preferred.

PGP key 5BA9 368B E6F3 34E4 local copy PGP key 5BA9 368B E6F3 34E4 via keyservers pgp key statistics for 0x5BA9368BE6F334E4 Koos van den Hout
Other webprojects: Camp Wireless, wireless Internet access at campsites, The Virtual Bookcase, book reviews