2021-11-22 Resizing a filesystem through several layers
For work I use a supplied laptop with Windows 10. For some of my work I want to have a Linux environment available so I have VirtualBox with a Linux virtual machine running. And because some of the work I do on that Linux virtual machine I use full-disk encryption. And the installation was done with the encrypted lvm setting. Resizing the filesystem because it was getting full turned out to be a lot of steps! After stopping the virtual machine I wanted to resize the disk from the VirtualBox media manager but that gave an error. After that I tried the commandline, giving about the same error:> "\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifymedium rotterdam.vdi --resize 32768 0%... Progress state: VBOX_E_NOT_SUPPORTED VBoxManage.exe: error: Failed to resize medium VBoxManage.exe: error: Resizing to new size 34359738368 is not yet supported for medium 'C:\Users\hout0101\VirtualBox VMs\rotterdam\rotterdam.vdi' VBoxManage.exe: error: Details: code VBOX_E_NOT_SUPPORTED (0x80bb0009), component MediumWrap, interface IMedium VBoxManage.exe: error: Context: "enum RTEXITCODE __cdecl handleModifyMedium(struct HandlerArg *)" at line 816 of file VBoxManageDisk.cppIt turns out the .vdi is the wrong type for dynamic resizing. Solution: clone it! The new .vdi will have the dynamic type automatically and there is a "before" .vdi now on disk to revert to if anything goes wrong.> "\Program Files\Oracle\VirtualBox\VBoxManage.exe" showhdinfo rotterdam.vdi UUID: f832b0b4-8738-491d-bd9c-291d755a4af7 Parent UUID: base State: created Type: normal (base) Location: C:\Users\hout0101\VirtualBox VMs\rotterdam\rotterdam.vdi Storage format: VDI Format variant: fixed default Capacity: 26067 MBytes Size on disk: 26070 MBytes Encryption: disabled Property: AllocationBlockSize=1048576 In use by VMs: rotterdam (UUID: 2454dadb-a82d-4d74-bbea-8dcf2b2d1bf1) > "\Program Files\Oracle\VirtualBox\VBoxManage.exe" clonehd rotterdam.vdi rotterdam-2.vdi 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Clone medium created in format 'VDI'. UUID: 835e2f75-c19d-4e98-865e-d7acf1359fc7 > "\Program Files\Oracle\VirtualBox\VBoxManage.exe" showhdinfo rotterdam-2.vdi UUID: 835e2f75-c19d-4e98-865e-d7acf1359fc7 Parent UUID: base State: created Type: normal (base) Location: C:\Users\hout0101\VirtualBox VMs\rotterdam\rotterdam-2.vdi Storage format: VDI Format variant: dynamic default Capacity: 26067 MBytes Size on disk: 26069 MBytes Encryption: disabled Property: AllocationBlockSize=1048576 > "\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifymedium rotterdam-2.vdi --resize 32768 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%I moved the old .vdi out of the way and added the new .vdi to the virtual machine and started it again. This worked fine, but the root volume wasn't any bigger (yet). Next steps: enlarge the extended partition and the Linux partition in it on disk using parted. You really have to know what you are doing here, so I'm not just going to give a cut-and-paste sample. Now I can resize the encrypted and mounted volume! With the right passphrase.# cryptsetup resize /dev/mapper/sda5_cryptAnd grow the 'physical' (ahem) volume:# pvresize /dev/mapper/sda5_cryptResize the logical volume:# lvextend /dev/rotterdam-vg/root -l +1674And finally resize the mounted filesystem:# resize2fs /dev/mapper/rotterdam--vg-rootAnd the filesystem has grown, and looks good in a fsck on the next boot. So solid state disk → Windows filesystem → vdi file → VirtualBox → disk in Linux virtual machine → partition → lukscrypt → logical volume manager → volume → filesystem.
2021-09-28 Debugging a systemd issue .. without having to curse
Today I ran into an issue related to systemd and I decided to try to fix it without too much cursing. The result was a number of google searches ending up on unix.stackexchange.com but eventually I fixed the problem. At work we use splunk for security monitoring and one of the indexers failed to start the splunk processes after a reboot. On browsing the systemd boot log with journalctl -b -l I discovered that the main issue was that creating files in /opt/splunk failed. This was due to an interesting race condition: splunk may start as soon as target network.target has been reached, but mounting /opt over iscsi also needs network.target to start. So the unit file has been updated to:[Unit] Description=Systemd service file for Splunk, generated by 'splunk enable boot-start' After=network.target opt.mountThe next problem was the systemctl start Splunkd.service failing in some intricate way. I had a look at the logging and saw that it was actually trying to restart the service and failed at killing one of the old processes. It turned out the /opt/splunk/var/run/splunk/splunkd.pid file had old contents and one of the PIDs in that file was now in use by a kernel thread. Those you can't kill, the restart failed and therefore the service did not start at all. Solution: remove the .pid file.
2021-04-08 Stopping with NTP servers at work
For almost 20 years I was involved with the running of NTP time servers at work. But the hardware aged and my job is no longer in systems administration and not in the department actually housing the timeservers. So, time to stop doing it. The pool ntp server has been retracted, DNS names removed and soon I will make one final trip to shut down hardware one last time and remove it from racks. The end for ntp.cs.uu.nl and others. I still run an NTP server at home which is available in the IPv6 NTP pool. That server also compared itself to one of the servers at work so it has been reconfigured. I added a few upstream servers and made sure all of them are reachable via IPv6. The log of NTP service at cs.uu.nl was kept, here is the final version:
Date Event 8 Apr 2021 DNS names for ntp service at cs.uu.nl removed 2 Apr 2021 Announcement posted to system administration mailing list that ntp service at cs.uu.nl will stop 24 Sep 2014 A second stratum-1 ntp appliance is brought on-line, galileo.cs.uu.nl 28 Nov 2011 Fixed the networking for stardate, the full time lab is up and running. 23 Nov 2011 The antenna cable connectors are soldered on which results in a working setup after a few tries. Stardate is better at reporting the state of the power to the GPS antenna, but has no working network. Huygens has working network and serves time to metronoom. 22 Nov 2011 The server ntp.cs.uu.nl is active at its new IP. Our own GPS reference doesn't work yet: we still need to solder the right connectors on the antenna cable. The server is added to the ntp pool and traffic starts to flow a few hours later. 15 Nov 2011 The ntp servers are moved to their new location 14 Nov 2011 The ntp servers are switched off 13 Nov 2011 We retract ntp.cs.uu.nl at its current address from the pool because the serverroom will move physically, the ntp equipment will move to a different location and the IP will change to deal with the traffic better 18 Sep 2011 Stats for doei.cs.uu.nl, five years after withdrawing it from the ntp pool 19 Sep 2010 Stats for doei.cs.uu.nl, four years after withdrawing it from the ntp pool 4 Mar 2010 The turkish adsl provider ttnet falls off the Internet for a few hours, traffic falls from 2000 packets/second to 100 packets/second in that time 22 Jan 2010 We volunteer ntp.cs.uu.nl for the turkish part of the ntp pool. Traffic explodes, peaks over 5000 packets/second 18 Sep 2009 Stats for doei.cs.uu.nl, three years after withdrawing it from the ntp pool 28 Jul 2009 ntp.cs.uu.nl back at full speed in the ntp pool, firewall configuration fixed 15 Jul 2009 rear doors of racks closed again 2 Jul 2009 10:00 serverroom airco has problems with high temperatures (28-30 C), we open rear doors of racks which makes the temperature go down a bit in the racks but the airco still has hard work Mar 2009 ntp.cs.uu.nl tuned down in the ntp pool to avoid firewall issue 18 Sep 2008 Stats for doei.cs.uu.nl, two years after withdrawing it from the ntp pool 17 Jan 2008 huygens.cs.uu.nl has a GPS reception failure, fixed with a software update 18 Sep 2007 Stats for doei.cs.uu.nl, a year after withdrawing it from the ntp pool 11 Mar 2007 airco failure serverroom 5 Mar 2007 all ntp servers moved to one rack close together for temperature stability 20 Jan 2007 airco failure serverroom 9 Jan 2007 huygens.cs.uu.nl added as stratum-1 23 Dec 2006 airco failure serverroom 29 Nov 2006 powerfailure in our building 1 Nov 2006 metronoom.dmz.cs.uu.nl takes over as ntp.cs.uu.nl and joins pool.ntp.org ~ 24 Oct 2006 antenna cable to stardate.cs.uu.nl reconnected ~ 6 Oct 2006 ntpd on stardate disabled: free running clock starts to differ too much from correct time ~ 25 Aug 2006 antenna cable from stardate.cs.uu.nl disconnected because of building and recabling activities 1 Aug 2006 doei.cs.uu.nl leaves pool.ntp.org 19 Aug 2003 doei.cs.uu.nl joins pool.ntp.org 10 Jan 1999 stardate.cs.uu.nl set up as stratum-1 with GPS time reference
2021-03-03 Checking the TLS setup for my webservers
I'm currently following the course The Best TLS and PKI Training Course in the World and learning even more about the workings of encryption, TLS and certificates. One of the things I learned is to balance security with performance. And I directly used this new insight on my own webservers. The connection which brought you this page from https://idefix.net/ is still encrypted but I saved a few milliseconds on the encrypted setup by switching from a big (4096 bit) RSA private key to a 384 bit ECDSA key which are comparable in cryptographic strength. But the calculations with the ECDSA key are less CPU intense. And yes, I have statistics on page loading times before and after the changeover of the key. It was a good moment to change private keys anyway, the old keys were more than a year old. This is one of those areas where I like having my knowledge hands-on. Actually understanding what is happening and why.
I'm currently following the course 2020-12-05 Playing with a fully programmable LED strip
At work there is a sort-of competition for the best christmas decorations in the office. At the end of last year I considered doing something with programmable LEDs to 'participate' in this competition in 2020. This year turned out somewhat different, but slowly my son is also somewhat interested in electronics, soldering and making the computer do something. So I set out to find fully programmable LED strips. I found a good comparison of LED strips in a Youtube video: LED Strips, what's the difference? WS2811, WS2812B, 2812Eco, WS2813, WS2815, SK6812, SK9822. which compares the several available types and their pros and cons. After viewing this video and for my limited experiment I thought the WS2812B based LED strip would be the best choice. The next hurdle was controlling it and I found Connect and Control WS2812 RGB LED Strips via Raspberry Pi which has pointers to the right code. I am not following the advice on that page about working with mains power cables. That looks dangerous. I ordered a WS2812B based LED strip and a matching power supply for 5V 40Watt from a Dutch webshop and got it in a few days later. I was amused by the warning the webshop gave that a LED strip like this is for advanced users only because you have to add a controlling device and do all the programming. That is exactly what I intended to do! Programming is in Python3, and I haven't written any Python code before. But with a lot of google searches and looking at samples I got the idea right. I now have the LED strip blinking in exactly the patterns I want, including a nice pattern for a christmas tree. And it blinks 'MERRY CHRISTMAS' in morse code, because why not!
2019-03-19 Time to update putty
An interesting bit of news: SSH client gets patched after RSA key exchange memory vuln spotted.The fixes implemented on PuTTY over the weekend include new features plugging a plethora of vulns in the Telnet and SSH client, most of which were uncovered as part of an EU-sponsored HackerOne bug bounty.Get your updated putty at the PuTTY download page. Update: Interesting visual change in putty: informational lines from the client are now prefixed by a putty logo. This could make it harder to mislead the user in certain attacks.
2018-10-01 Getting distracted on shodan
This morning I was looking on shodan for open remote desktop servers in the work network since RDP was mentioned as an attack vector in the latest GANDCRAP ransomware. Searching for '3389' on shodan found something completely different: an open industrial control system (ICS) for tankstation gauges.IN-TANK VOORRAAD TANK PRODUCT VOLUME TC VOLUME VULVOL HOOGTE WATER TEMP 1 UL 98 9757 9693 10283 939.2 0.0 20.09 2 EURO 2...According to The Internet of Gas Station Tank Gauges -- Take #2 - Rapid7 this was already a reported issue in January 2015 and according to their research it may be possible to do bad things with this access. The above is from a gas station I can find on google maps. Oh I found the way to search for open remote desktop servers on shodan: port:3389.
2018-09-24 Windows 10 WiFi can't deal with password changes
The work laptop is now "upgraded" to Windows 10. I wasn't sure about it as I saw Windows 7 as less annoying but it's the corporate choice. And after I changed the password for my eduroam wifi-account it just gives an error and does not connect to the wireless network. The obvious choice to show the option to enter a new password does not pop up (unlike Android which came with that suggestion right away). Even the "network troubleshooter" doesn't come with the source of the connection problem let alone the obvious solution. The Windows 10 "solution" is to just forget the network and discover it again. I'm glad this isn't a network where I need special options and a certificate to log in.
2018-08-17 Trying (and failing) to correlate security logs
Since activating sendmail authentication with secondary passwords I see a number of attempts to guess credentials to send mail via my system. This is not very surprising, given the constant attack levels on the wider Internet. For work I am looking at log correlation and monitoring and with that in mind I noted that finding the right information from sendmail where and when the attempt came from is quite hard since there are several processes busy and it's hard to correlate the logging. The failed attempt is logged by saslauthd in /var/log/auth.log:Aug 16 12:28:57 greenblatt saslauthd[32648]: pam_unix(smtp:auth): check pass; user unknown Aug 16 12:28:57 greenblatt saslauthd[32648]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= Aug 16 12:28:59 greenblatt saslauthd[32648]: do_auth : auth failure: [user=monster] [service=smtp] [realm=idefix.net] [mech=pam] [reason=PAM auth error] Aug 16 12:29:00 greenblatt saslauthd[32649]: pam_unix(smtp:auth): check pass; user unknown Aug 16 12:29:00 greenblatt saslauthd[32649]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= Aug 16 12:29:02 greenblatt saslauthd[32649]: do_auth : auth failure: [user=monster] [service=smtp] [realm=] [mech=pam] [reason=PAM auth error]This is probably related to this sendmail log information:Aug 16 12:28:56 greenblatt sm-mta[20716]: STARTTLS=server, relay=62.82.128.182.static.user.indesat.com [62.82.128.182] (may be forged), version=TLSv1/SSLv3, verify=NO, cipher=DHE-RSA-AES256-SHA, bits=256/256 Aug 16 12:29:02 greenblatt sm-mta[20716]: w7GASspx020716: 62.82.128.182.static.user.indesat.com [62.82.128.182] (may be forged) did not issue MAIL/EXPN/VRFY/ETRN during connection to MSP-v6But I can't be sure as there are multiple 'did not issue MAIL/EXPN/VRFY/ETRN' messages in the logs. So I can't build a fail2ban rule based on this.
2017-03-10 Improving the Internet security one service at a time
Items with tag work before 2017-03-10At work we (indirectly) get the scanning results from Shadowserver which now includes open VNC servers which is yet another service we don't really want left open to the Internet in general. A few were found which are now actively chased after to get them firewalled/disabled. I really like the concept of shadowserver. ISPs that want the information can get full overviews of insecure services and botnet activity on their network. A full overview of what shadowserver looks for can be found at The scannings will continue until the Internet improves - Shadowserver.