News items for tag onewire - Koos van den Hout

After the problems with detaching and attaching the USB 1-wire interface from a kvm virtual machine to fix an interference issue showed up again I decided to move the USB 1-wire interface to a different machine, one where kvm virtualisation isn't in the mix. The closest available machine that can deal with the 1-wire interface is a Raspberry Pi which also has other monitoring tasks.

This move worked fine and the 1-wire temperatures are showing up again in influxdb. I decided not to update the rrdtool temperature database. I will have to find time to migrate the rrdtool history to influxdb. Ideally there will be some aggregation for older measurements but I'd like an "infinite" archive of a daily average.

I have a DS2490 USB 1-wire interface on the home server conway which is rerouted to one of the virtual machines so that that virtual machine can read the sensors on the 1-wire network. This rerouting works when the machine is started, the DS2490 USB 1-wire shows up in the virtual machine fine. From time to time this DS2490 USB 1-wire interface gets confused when I am transmitting on the radio so the solution is to detach it from the virtual machine, unplug it from the server, plug it in again and attach it to the virtual machine again. Today this had to be done and I got an unexpected error message:

root@conway:~# virsh attach-device --live gosper /etc/onewire-for-gosper.xml
error: Failed to attach device from /etc/onewire-for-gosper.xml
error: internal error: unable to execute QEMU command 'device_add': failed to find host usb device 2:8

In logfile /var/log/libvirt/libvirtd.log:

2022-09-24 21:16:38.655+0000: 10923: error : qemuMonitorJSONCheckError:395 : internal error: unable to execute QEMU command 'device_add': failed to find host usb device 2:8

To be complete about it: usb device 2:8 is exactly the right one!

root@conway:~# lsusb | grep 2490
Bus 002 Device 008: ID 04fa:2490 Dallas Semiconductor DS1490F 2-in-1 Fob, 1-Wire adapter

This seems to be new since I upgraded the homeserver to Devuan beowulf giving me versions:

| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                  Version         Architecture Descripti
+++-=====================================-===============-============-=========
ii  libvirt-clients                       5.0.0-4+deb10u1 amd64        Programs 
ii  libvirt-daemon                        5.0.0-4+deb10u1 amd64        Virtualiz
un  libvirt-daemon-driver-storage-gluster                  (no descr
un  libvirt-daemon-driver-storage-rbd                      (no descr
un  libvirt-daemon-driver-storage-zfs                      (no descr
ii  libvirt-daemon-system                 5.0.0-4+deb10u1 amd64        Libvirt d
ii  libvirt-glib-1.0-0:amd64              1.0.0-1         amd64        libvirt G
ii  libvirt0:amd64                        5.0.0-4+deb10u1 amd64        library f

First idea: AppArmor

The first search result that came up was Bug #1552241 “libvirt-bin apparmor settings for usb host device” : Bugs : libvirt package : Ubuntu. So I tried changing the /etc/apparmor.d/abstractions/libvirt-qemu file. After a few tries and reading the warnings in the rest of the file I made sure the source was AppArmor by completely disabling it. The error did not go away so I reverted the libvirt-qemu rules to the original settings, restarted AppArmor and kept debugging.

Second idea: usb rights

Based on QEMU USB passthrough broken after Ubuntu 18.04 upgrade I added udev rules to make sure group libvirt-qemu had read and write rights on the usb device, with /lib/udev/rules.d/51-qemu-usb-passthrough.rules containing:

SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="04fa", ATTRS{idProduct}=="2490", MODE="0664", GROUP="libvirt-qemu"

And doing the

root@conway:~# udevadm control --reload-rules

And verifying the resulting rule:

root@conway:~# udevadm test -a -p  $(udevadm info -q path -n /dev/bus/usb/002/008)
calling: test
version 3.2.9
This program is for debugging only, it does not run any program
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

[..]

GROUP 110 /lib/udev/rules.d/51-qemu-usb-passthrough.rules:1
MODE 0664 /lib/udev/rules.d/51-qemu-usb-passthrough.rules:1
handling device node '/dev/bus/usb/002/008', devnum=c189:135, mode=0664, uid=0, gid=110
[..]

Indeed the right groupid, but still the same error message when trying the attach-device command.

Interesting find: it's specific to the virtual machine that had the device before

Small update: I can attach the USB device to a different host and detach it from that host again. I just can't attach it to the 'original' host again.

I also posted this question on serverfault: Can't live-attach a USB device to a kvm virtual host again after upgrades.

Update: After a complete reboot of the homeserver the USB 1-wire interface worked again (as I could imagine). But after another interference problem it's now in the same state again. I did change the definition in both the virthost configuration and the xml file from managed='no' to managed='yes' before the reboot but that hasn't helped. Contents of the /etc/onewire-for-gosper.xml file now:

    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x04fa'/>
        <product id='0x2490'/>
      </source>
    </hostdev>

I moved the new ntp server to the shed today. I found a nice case for it: an actual wooden box. I climbed on the roof of the shed to find a place for the GPS antenna (with magnetic base). Parts of the enclosures around our solar panels are from ferrous metals, so I found a place with an ok view of the sky to place the antenna and led the cable to a ventilation shaft to get it inside the shed. I made sure the cable was going up in the ventilation shaft first to avoid having a drip loop on one of our bicycles.

Although I did most work on the w1retap configuration before I couldn't get it running at first. I kept seeing the error message:

koos@henkp:~ $ LD_LIBRARY_PATH=/usr/local/lib/w1retap w1find DS2490-1
Error 119: Failed to set libusb configuration

It took some serious searching to find a hint: that is caused by the usb device file access rights. Solution is to install the 45-w1retap.rules that comes with w1retap into /etc/udev/rules.d.

At the moment weather data is being fetched on the Raspberry but the wifi between shed and house is so bad that the data stays there. I'm not sure how that can be fixed. It turns out the external wi-fi dongle I bought was listed as having 5 GHz support, but the reviews of the chipset used say it doesn't. The congestion in the 2.4 GHz band makes it very difficult to reach the pi. Doing a ping test over longer time gives me 91% packet loss.

I dug up a different 2.4 GHz antenna from the junkbox and suddenly the connection is stable with a lot less packet loss. This antenna is directional and now pointing right at my access point.

Now the weather data is collected and forwarded to the server for Weather station Utrecht Overvecht.

NTP didn't seem to work on the first try, I'm not seeing any data for the GPS_NMEA server. This works again after a powerdown/up.

For more than 12 years now(!) the house has temperature sensors using the 1-wire protocol. I recently redid some of the wiring between floors and I finally got around to rerouting the 1-wire network via this new route.

I also added a temperature sensor in the big room in the attic, we are thinking of using that room more often. To get an idea of how good that idea is we wanted to get an idea of the temperatures up there and that's what I have 1-wire sensors for! I soldered an 18b20 sensor to the end of a 4-wire flat phone cable, added it to the network and it's measuring.

So now 12 environmental temperatures are measured every 5 minutes: 9 in the house, one in the weather hut, one in the shed and one on the roof of the shed.

I also updated the 1-wire projects overview with how I use 4-wire flat phone cable in RJ45 connectors for 1-wire network. I had to look up how I did that previously before I could start adding new cables!

I was looking at some onewire temperature stats and noticed the first stats being aged out. I started monitoring temperatures with 1-wire sensors in January 2007 using rrdtool. I set up round robin archives with an expiry in 11 years, and those 11 years have passed now for the first measurements.

The new hardware for the homeserver has no external serial ports, so I could not use the old serial / 1-wire interface that has been doing the home monitoring for years. But I had a spare USB DS2490 interface. So I plugged this into the server and wanted to forward the USB device to the guest VM that runs all the monitoring.

First I had to blacklist all the loaded drivers to have the device available to kvm as-is. In /etc/modprobe.d/local-config.conf:

blacklist w1_smem
blacklist ds2490
blacklist wire

Next step was to attach the device to the right vm. I followed the hints at How to auto-hotplug usb devices to libvirt VMs (Update 1) and edited the definition for the vm to get the host device like:

    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x04fa'/>
        <product id='0x2490'/>
      </source>
    </hostdev>

But that did not get the usb device attached to the running VM and I did not feel like rebooting it. So I created an extra file with the above and did a

root@conway:~# virsh attach-device --live gosper /tmp/onewire.xml 
Device attached successfully

And then I had to do the same blacklisting as above in the virtual machine. After doing that I detached and attached it from the VM without touching it with simply:

root@conway:~# virsh detach-device --live gosper /tmp/onewire.xml 
Device detached successfully

root@conway:~# virsh attach-device --live gosper /tmp/onewire.xml 
Device attached successfully

After that I had to set up rules for the telemetry user to have enough access to the USB device:

#
SUBSYSTEMS=="usb", GOTO="usb_w1_start"
GOTO="usb_w1_end"
LABEL="usb_w1_start"
ATTRS{idVendor}=="04fa", ATTRS{idProduct}=="2490", GROUP="telemetry", MODE="0666"
LABEL="usb_w1_end"

And now it all works:

telemetry@gosper:~$ digitemp_DS2490 -a
DigiTemp v3.7.1 Copyright 1996-2015 by Brian C. Lane
GNU General Public License v2.0 - http://www.digitemp.com
Found DS2490 device #1 at 002/003
Jan 01 21:53:11 Sensor 10A8B16B0108005D C: 9.500000
Jan 01 21:53:12 Sensor 28627F560200002F C: 17.062500
Jan 01 21:53:14 Sensor 10BC428A010800F4 C: 19.562500
Jan 01 21:53:15 Sensor 1011756B010800F1 C: 11.937500
Jan 01 21:53:16 Sensor 10B59F6B01080016 C: 16.312500
Jan 01 21:53:17 Sensor 1073B06B010800AC C: 18.687500
Jan 01 21:53:18 Sensor 102B2E8A010800F0 C: 29.250000
Jan 01 21:53:20 Sensor 28EF71560200002D C: 16.687500

Working house temperatures again!

Since the powerfailure that caused problems for the weatherstation computer ritchie and the conclusion that even after the bios upgrade the serial ports kept failing there was no 'inside the shed' temperature.

But this week I needed a better view of the temperature inside the shed as we're using it to keep some meat cool. So I heated up the soldering iron and the heatshrink gun and made a cable with two DS18B20 sensors in it. I decided that if I started on measuring temperatures inside the shed I also wanted the temperature near the roof.

The interesting bit was adding the two sensors to the w1retap configuration. It seems the whole 1820 family of temperature sensors needs to be set up as a 'DS1820' and w1retap will find out how to read it. Resulting configuration:

28F24C5602000054:DS1820:Tempinside:TEMP0:⁰C::
286B545602000031:DS1820:Temproof:TEMP0:⁰C::

and now I have logging of the temperatures:

2017-12-29T16:28:00+0100 Tempinside 2.812500 ⁰C
2017-12-29T16:28:00+0100 Temproof 2.687500 ⁰C

And it helps us to determine when we need to make space in our fridge and move some other things to the shed to keep them somewhat cool.

I visited the Hobby boards website just out of interest and noticed they are going out of business. With the growing interest in home automation I thought they would be doing great but I guess 1-wire networks aren't as popular as the wireless home automation options (which don't seem to be designed for security and privacy).

I still have a number of temperature sensors so I can add those easily. I have ordered another humidity sensor since that is something I want to measure indoor in the crawlspace under our house.

In mijn 1-wire projecten was ook nog het plan om de watt-uur pulsen van de electricteitsmeter te gaan tellen. Met een schakeling met lichtgevoelige leds en een teller is het me nooit gelukt de pulsen betrouwbaar te tellen.

Ik ontdekte de YouLess die het tellen van zo'n ledje of van een draaischijfmeter terdege opgelost heeft en die niet begint met de data naar een externe dienst te sturen voor ik er naar kan kijken. Er is gewoon een simpele interface waarmee de tellingen van de YouLess via het netwerk uit te lezen zijn en die gebruik ik natuurlijk en maak er interne statistieken van.

Gemeten electriciteitsgebruik per kwartier

Van die statistieken maak ik ook een grafiek met een nauwkeurigheid per kwartier. Die keuze is omdat de 'slimme meter' per kwartier uitgelezen kan worden en ik wil wel eens weten wat je er dan uit kan halen. Heel veel dus! Uit de kwartierwaarden is een hoop te halen over onze dagelijkse gewoonten, zeker als je ze over een lange termijn beschikbaar hebt. Voor de meeste pieken kan ik een verklaring bedenken en als eenmaal bekend is welke actie welke hoeveelheid energie gebruikt kan je daarna andere pieken ook weer uitleggen. Opvallend is dat zelfs bij de resolutie van 15 minuten een kort maar duidelijk gebruik zoals de waterkoker of een lange termijn wijziging zoals het licht aan of uit schakelen prima te zien is.

The last year in counted lightning strikes, showing clearly that I got active on 20meter psk31 in October 2014. The 'blips' before that are real thunderstorms.

I thought of this ages ago: Moving the lightning strike detector to the shed but only today got around to it because we had some serious chances of a thunderstorm earlier today which showed up on the lightning strike detector but the graph was completely screwed up again after I tried some psk31 digital mode transmissions on the 20 meter amateur band (14.000 to 14.250 MHz for me).

So now it is in the shed. Moving it to a lower position does mean I will not get readings for thunderstorms as far away as I used to but I'd rather have usable readings at this moment. First tests with transmitting psk31/psk63 on 20 the meter amateur band after I changed it look like it doesn't count the transmissions anymore. Now to wait for the next good thunderstorm to see how that gets counted. Some blips are showing up.

Update 2015-07-14: The first result seems to be that using the lights in the shed (tubelights with starters) shows up clearly. Using the radio still has no effect. I now await the first thunderstorm for more results.

Update 2015-07-28: No thunderstorm has been reported by the KNMI weather institute thunderstorm archive within a short distance of my sensor. I guess the maximum range is quite limited now.