News items for tag statistics - Koos van den Hout

Time for an overview of what happened in amateur radio in 2022 for me. Like previous years I will look back at the plans and what happened. Looking back at Closing 2021 in amateur radio the following results are clear:

• The morse exam finally happened and I passed it.
• More morse contacts in contests and in general
• 18 new countries/entities in the log
• More countries/entities in morse in the log
• Satellite contacts: none
• Used the improving propagation

The plans for 2023:

• Try to get more countries/entities, especially in morse. I am working towards DXCC in morse: 100 entities confirmed.

And one thing is both a result of 2022 and an item for 2023: I ordered a new radio: a Yaesu FT-911A, HF, VHF, UHF all mode at the end of 2022 and it was delivered last week. That will be a separate post.

After the problems with detaching and attaching the USB 1-wire interface from a kvm virtual machine to fix an interference issue showed up again I decided to move the USB 1-wire interface to a different machine, one where kvm virtualisation isn't in the mix. The closest available machine that can deal with the 1-wire interface is a Raspberry Pi which also has other monitoring tasks.

This move worked fine and the 1-wire temperatures are showing up again in influxdb. I decided not to update the rrdtool temperature database. I will have to find time to migrate the rrdtool history to influxdb. Ideally there will be some aggregation for older measurements but I'd like an "infinite" archive of a daily average.

I saw an upgrade of Grafana available, which turned out to be 9.0.0. When upgrading to 9.0.0 I get...

An unexpected error happened
TypeError: Object(...) is not a function

t@[..]public/plugins/grafana-clock-panel/module.js:2:15615
WithTheme(undefined)

So maybe the grafana-clock-panel plugin isn't compatible with 9.0.0 somehow.

Downgrading to 8.5.6 and reloading everything makes it work again.

Update: I checked the grafana-clock-panel plugin and noticed it hadn't been updated. So I did that update and retried grafana 9.0.0, and that made everything run smoothly again.

Na het aanpassen van het netwerk naar de schuur naar gigabit was ik natuurlijk ook de monitoring van de SolarEdge inverter met modbus/tcp aan het testen. En toen viel me even iets op, de inverter stond in mode THROTTLING en dat was me nog niet eerder opgevallen.

De uitvoer is dan

$ ./sunspec-status -v se-schuur -m 0
INVERTER:
             Model: SolarEdge  SE2200
  Firmware version: 3.2537
     Serial Number: xxxxxxxx

            Status: THROTTLING

 Power Output (AC):          342 W
  Power Input (DC):          348 W
        Efficiency:        98.50 %
  Total Production:     3964.313 kWh
      Voltage (AC):       237.40 V (49.94 Hz)
      Current (AC):         1.53 A
      Voltage (DC):       378.80 V
      Current (DC):         0.92 A
       Temperature:        42.75 C (heatsink)

Ik kon niet vinden wat de reden was van het terugregelen van het uitgangsvermogen. Ik log nu wel de statuswaarde van de inverters om te zien of dit vaker voorkomt.

Update: Achteraf denk ik dat dit gekomen is omdat ik de omvormer in de schuur gereboot had om het juiste IPv4 adres te krijgen voor monitoring. Dit was op een best wel zonnig moment. Na de reboot was ik snel aan het testen of de modbus/tcp monitoring het deed naar het nieuwe adres, en de omvormer gaat niet in een klap voluit electriciteit leveren maar brengt dat langzaam op gang.

After reverting to Grafana 8.4.7 for a while because alerts were failing in Grafana 8.5.0 I had a look at the available version today and saw version 8.5.2. I assumed the problem with DataSourceNoData errors was fixed by now and did the upgrade.

Indeed the alerts are seeing data fine now and I trust they will work when needed.

I installed Grafana from their debian repository, so I get updates via the normal apt update / apt dist-upgrade process. Since upgrading to version 8.5.0 the alerts were all firing because of 'DatasourceNoData' errors. According to Alert Rule returned no data (after upgrade to 8.5.0) #48128 other people are seeing this too.

For now I downgraded to version 8.4.7 where things work fine and I'll see if a newer version shows up.

I've been playing with grafana for about a year since starting with updating my statistics gathering and I keep seeing new options and updates in grafana.

Grafana recently got some new options for alerting and I am trying a few of those. Alerts for things that are a real problem and can cause other problems are a good start. Based on some earlier problems I keep an eye on some filesystems that are over 90% full.

Today I read Three DDoS attacks on my personal website found via Three DDoS attacks on my personal website : r/homelab reddit and this made me wonder about overloads on my webserver. The easiest way to detect problems with web serving I could think of is to look at the queue size in haproxy which is monitored in influxdb/grafana anyway for nice graphs of website traffic.

I did have a time with too high queues for backend webservers. But that was when the backend server was completely broken due to a filesystem problem so that was a logical reason.

It would be nice if I could iterate alerts, like 'for the root filesystem of every monitored system'. Or at least copy them changing only the system name in the rules and alerts.

I noticed I didn't do a "Closing 2021 in amateur radio" post yet, so time to catch up. Looking back at the Review of 2020 in amateur radio with plans for 2021 I can say:

• Practising morse has happened! Just no exam yet, but that is mainly due to the current circumstances
• Satellite contacts: none.
• Morse and phone in contest: yes!
• New qsl cards ordered and in use

And the plans for 2022:

• More and more morse, and that exam. There is an exam date now and it will be possible to get the wanted 'CW included' on my radio amateur identification
• Again satellites
• In contests: try to get more morse and phone contacts.
• Use the better propagation to get contacts on different bands

More detailed statistics over 2021

And I had to check my own notes again how I got these numbers last year, so I'm adding the sql queries I typed at the mysql/mariadb client. With the database behind cqrlog available I can make all kinds of queries.

By month

The influence of months with (digital) contests isn't as strong as in previous years.

+-------+-----+
| month | cnt |
+-------+-----+
|     1 | 234 |
|     2 | 204 |
|     3 | 238 |
|     4 | 161 |
|     5 | 131 |
|     6 | 111 |
|     7 | 211 |
|     8 |  19 |
|     9 | 232 |
|    10 | 204 |
|    11 | 191 |
|    12 | 101 |
+-------+-----+

Query: select month(qsodate) as month,count(id_cqrlog_main) as cnt from cqrlog_main where year(qsodate)=2021 group by month order by month;

By band

No real surprises there. And the feeling that 10 meter was improving isn't showing in the statistics yet.

+------+-----+
| band | cnt |
+------+-----+
| 40M  | 699 |
| 20M  | 849 |
| 17M  | 151 |
| 15M  |  40 |
| 10M  | 243 |
| 2M   |  51 |
| 70CM |   4 |
+------+-----+

Query: select band,count(id_cqrlog_main) as cnt from cqrlog_main where year(qsodate)=2021 group by band order by freq;

By mode

Almost double the number of morse contacts compared to the previous year.

+-------+-----+
| mode  | cnt |
+-------+-----+
| JT65  |   2 |
| PSK31 |   3 |
| FM    |  19 |
| FT4   |  35 |
| PSK63 | 226 |
| CW    | 240 |
| SSB   | 267 |
| RTTY  | 386 |
| FT8   | 859 |
+-------+-----+

Query: select mode,count(id_cqrlog_main) as cnt from cqrlog_main where year(qsodate)=2021 group by mode order by cnt;

Na een poosje blijkt ook het getal 65534 (0xFFFE) in sunspec modbus antwoorden een vorm van 'geen geldige uitlezing' te kunnen zijn. Ik heb de scripts die de gegevens ophalen en verwerken richting influxdb hier op aangepast.

Het lokaal uitlezen van de SolarEdge omvormers via Modbus/TCP ging op den duur toch weer haperen, zeker over wifi.

Ik begreep uit wat beschrijvingen dat het bijwerken van de firmware in de omvormer hierbij zou moeten helpen dus dat heb ik vandaag gedaan. Ik had eerder naar een van de omvormers ook een netwerkkabel gelegd maar nog niet de omvormer open gemaakt om die van binnen aan te sluiten. Nu moest de omvormer ook open om de firmware bij te werken dus heb ik gelijk het ontbrekende stukje UTP kabel gemaakt en aangesloten, en de wifi module verwijderd.

De SolarEdge handleiding voor het bijwerken van omvormer firmware had na alle stappen om de omvormer veilig open te maken een aantal stappen voor het kiezen van een upgrade maar de omvormer 'zag' gelijk bij het inschakelen dat er een kaartje met firmware in zat en startte vanzelf de upgrade.

Op beide omvormers is de upgrade gelukt. De omvormer waar ik ook de netwerkaansluiting omgezet heb van wifi naar bedraad wil momenteel geen gegevens doorgeven naar SolarEdge monitoring (ik zie wel verkeer naar buiten gaan) en er staat ook geen S_OK op het display. Ik denk dat dat komt door de wijziging netwerkaansluiting. Volgens wat bronnen kan dat even duren, maximaal 2 dagen.

Ondertussen kan ik nu via Modbus/TCP de nodige gegevens goed uitlezen, en ik heb wat veranderingen aan het monitoring script gemaakt zodat er ook data van de afzonderlijke omvormers naar een influxdb database gaat.