News items for tag statistics - Koos van den Hout

2021-03-01 Updating my statistics gathering
Grafana dashboard sample For years and years I have been using rrdtool to gather and graph statistics at home. I started gathering home temperatures around 2008 but I see NTP statistics gathering from 2003 and my last mrtg graphs were created in October 2002. So that suggests I've been using rrdtool since that date.

Anyway, I'm looking at newer options. After some asking around I installed influxdb and started gathering data. I adjusted some of my data gathering scripts around rrdtool to also put the data in influxdb.

The easiest data to gather and graph was the load average, available entropy and number of processes for a number of systems at home. So that dashboard has been built and allows selection of the wanted computer.

My first conclusion is just collecting data and thinking what kind of graphs to create later is a lot easier with influxdb. With rrdtool the round robin database is designed around the graphs you want. In this case I just start gathering data and when data has come in start playing with possible graphs from that data.

The next challenge is to set the rules for maintaining the old data. One of the triggers to look at other options was that I was at the end of a nearly 11-year cycle of stored temperatures in rrdtool, and I wanted to keep that history if possible.

I don't have to keep every measurement forever, but with storage being cheap I think I will keep daily averages forever when this is 'production'.

Tags: , ,
2021-01-01 Closing 2020 in amateur radio with a plot of the number of contacts and a review
QSO count plot up to December 2020 Time to plot the number of contacts in 2020 and a review. I made no specific resolutions for 2020 but looking back there were positive developments.
  • The Kenwood TS480-SAT is at a remote location with good antennas for most of the HF bands. This enabled me to work new countries and get more voice and morse contacts in the log.
  • I was active on amateur satellites a few times, including from Austria.
  • The morse speed improved and I got on the air more with morse. Including a few morse contests.
  • I tried to follow the Bulgarian Saints 2020 stations and I had at least one contact with one of the stations in 10 out of the 12 months of 2020. In 8 months I had at least one contact in morse with the station of that month. So I earned the Bulgarian Saints diploma 2020.
  • In general I made more contacts in this year than in any other year. The endfed antenna is now mounted outside in such a way I can leave it there, which makes getting on the radio for a few contacts easier. There were also more special event stations active this year.
  • I had radio contacts with several new countries.
  • The box with outgoing QSL cards is now empty!
  • I'm active as QSL manager for my local club, this is fun and my part of keeping the club running.
Plans for 2021:
  • Keep practicing morse, try to pass the morse exam.
  • More satellite contacts. Weather permitting...
  • Morse and phone in contests.
  • Order new QSL cards and keep on sending them.
Read the rest of Closing 2020 in amateur radio with a plot of the number of contacts and a review

Tags: , , , ,
2020-11-28 Getting the DSL linespeed with both current and attainable speed from the Fritz!Box 7360/7590
I noticed I documented my original script to fetch the upstream and downstream DSL speed from the Fritz!Box 7360 but never documented the additional steps I took later to add the attainable upstream and downstream speeds to the current upstream and downstream speeds.

After switching to the Fritz!Box 7590 I missed my VDSL statistics so I dug up the scripts I had for the 7360 and tested whether they still work. And yes they do, so no changes there. The complete script:
#!/usr/bin/perl -w

use strict;

my ($fritzuser,$fritzpass);

$fritzuser="root";
$fritzpass="VerySecretPassword";

system("wget --user=$fritzuser --password=$fritzpass --post-file=wanifcfgrequest.xml --header=\"Content-Type: text/xml\" --header=\"SOAPAction: \\\"urn:dslforum-org:service:WANDSLInterfaceConfig:1#GetInfo\\\"\" --no-check-certificate http://fritz.koos.koffie.dot:49000/upnp/control/wandslifconfig1 -O wanifcfganswer.xml -o getfritz.log");

open(STATUSFILE,"){
        if (/(\d+)<\/NewUpstreamMaxRate>/){
                $upstream = $1;
        }
        if (/(\d+)<\/NewDownstreamMaxRate>/){
                $downstream = $1;
        }
        if (/(\d+)<\/NewUpstreamCurrRate>/){
                $upstreamcur = $1;
        }
        if (/(\d+)<\/NewDownstreamCurrRate>/){
                $downstreamcur = $1;
        }
}

if (defined $upstream and defined $downstream){
        my $line=sprintf("N:%d:%d:%d:%d",$downstream*1000,$upstream*1000,$downstreamcur*1000,$upstreamcur*1000);
        print $line."\n";
}
This does need the wanifcfgrequest.xml file:
<?xml version="1.0" encoding="utf-8"?>
<s:Envelope
   xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
   s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
   <s:Body>
      <u:GetInfo xmlns:u="urn:dslforum-org:service:WANDSLInterfaceConfig1">
      </u:GetInfo>
   </s:Body>
</s:Envelope>
And I get a usable wanifcfganswer.xml:
<?xml version="1.0"?>
 <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:GetInfoResponse xmlns:u="urn:dslforum-org:service:WANDSLInterfaceConfig:1">
<NewEnable>1</NewEnable>
<NewStatus>Initializing</NewStatus>
<NewDataPath>Fast</NewDataPath>
<NewUpstreamCurrRate>33032</NewUpstreamCurrRate>
<NewDownstreamCurrRate>111049</NewDownstreamCurrRate>
<NewUpstreamMaxRate>35725</NewUpstreamMaxRate>
<NewDownstreamMaxRate>117129</NewDownstreamMaxRate>
<NewUpstreamNoiseMargin>50</NewUpstreamNoiseMargin>
<NewDownstreamNoiseMargin>60</NewDownstreamNoiseMargin>
<NewUpstreamAttenuation>80</NewUpstreamAttenuation>
<NewDownstreamAttenuation>110</NewDownstreamAttenuation>
<NewATURVendor>41564d00</NewATURVendor>
<NewATURCountry>0400</NewATURCountry>
<NewUpstreamPower>498</NewUpstreamPower>
<NewDownstreamPower>513</NewDownstreamPower>
</u:GetInfoResponse>
</s:Body>
</s:Envelope>
This works without any change both on the Fritz!Box 7360 and the Fritz!Box 7590.

Tags: , ,
2020-09-02 An update to the home 1-wire network
For more than 12 years now(!) the house has temperature sensors using the 1-wire protocol. I recently redid some of the wiring between floors and I finally got around to rerouting the 1-wire network via this new route.

I also added a temperature sensor in the big room in the attic, we are thinking of using that room more often. To get an idea of how good that idea is we wanted to get an idea of the temperatures up there and that's what I have 1-wire sensors for! I soldered an 18b20 sensor to the end of a 4-wire flat phone cable, added it to the network and it's measuring.

So now 12 environmental temperatures are measured every 5 minutes: 9 in the house, one in the weather hut, one in the shed and one on the roof of the shed.

I also updated the 1-wire projects overview with how I use 4-wire flat phone cable in RJ45 connectors for 1-wire network. I had to look up how I did that previously before I could start adding new cables!

Tags: , , ,
2020-06-24 Moved the shed roof temperature sensor from under the roof to above the roof
I wanted to do some things in the shed with cabling, some things were not ideal after the solar power installation.

One of the things was that I had a temperature sensor to measure the temperature above the roof, but with all things that happened with cables it ended up hanging below the roof. I moved it back up in the ventilation pipe and with the current temperatures it started measuring 4 degrees Celcius higher immediately.

Tags: , ,
2020-06-15 A new home timeserver on order
After earlier tries to have a nice GPS-based timeserver for my home network I noticed a simple but very effective GPS 'hat' for the Raspberry Pi, the Raspberry Pi GPS Hat from Uputronix. While the Pi's are already taking over the home network just one more could be a nice addition. In the longer run this will probably replace the shed computer.

So I ordered a Pi with an added dual-band WiFi adapter, a case, the GPS hat and a GPS antenna. The GPS hat has PPS support so I will get the time correct. With the instructions from 5 minute guide to making a GPS Locked Stratum 1 NTP Server with a Raspberry Pi it should be easy. If this all works I may even add the resulting Pi to the IPv6 NTP Pool.

Update 2020-06-16: SOS Solutions came back with some bad news: the uputronix Pi GPS Hat isn't available anymore. I'm now looking at the comparable adafruit hardware which is somewhat more expensive, but offers the same options.

Update 2020-06-18: And the adafruit hardware is also not available soon. I cancelled the GPS unit part of the order and I'm looking at sourcing a GPS module for the Pi from another source. The GPS hat which sossolutions no longer sells is originally from uputronics where a newer version of the Raspberry Pi GPS/RTC Expansion Board is listed as available on the site. Based on a ublox chipset which allows me access to a lot of the GPS data.

Tags: , , ,
2020-05-01 Time for a plot of the number of contacts in amateur radio after a busy month
QSO count plot up to April 2020 Time for a new plot of the number of radio contacts. As usual contest months are quite visible and January is for me the month with the most contests.

But April 2020 is also quite visible. This last week I had a lot of time for radio due to holiday and not going anywhere. And other radio amateurs also had the time to be active, so there were a lot of new calls to get in the log. Combined with a good 10 meter band opening this added to a high number of contacts for a month with only one contest.

Tags: , ,
2020-02-16 Reconsidering data gathering and processing
For years and years I've been doing a lot of data gathering and storing the data using rrdtool. Data such as temperatures from lots of places, from mainboard CPU sensors to an outside weather station, other weather data, web traffic data, house electricy and gas usage, solar power. I started doing this with mrtg in 2001 and switched to rrdtool.

There are some improvements to this system, such as maintaining the rrd files on one machine and doing measurements on other machines in the form of timestamped files to be transported to the machine with rrd via rsync-over-ssh. This allows the central database to do a catch-up of decentrally gathered data after an interruption.

All in all there are two disadvantages at the moment: the system isn't very flexible, adding a datasource means making the big decision about how much data to keep how long and what I want to look at.

Diskspace isn't as constrained as it once was, I may want to keep some data forever and I may want to zoom in to a period a bit longer ago. So I'm looking at different solutions. For one dataset I already added an alternate datastore: the electricity and gas meter readings get copied to a postgres database once a day so I can look at the daily readings forever.

So the search is on for the ideal solution. For gathering and transporting data I am looking at mqtt, a lightweight protocol to gather and transmit data. This also makes it easier to have multiple data collectors look at one source so I can test with a few things first before I make a real switch to any new system.

Tags: ,
2020-01-01 Closing 2019 in amateur radio, time to plot the number of contacts and look back
QSO count plot up to December 2019 Time for a new plot of the number of radio contacts. Months with contests are quite visible. After the peak in number of contacts in July there was first a holday and after that no big peaks in number of contacts. December 2019 jumps out a bit again due to the FT8 roundup on 8/9 December in which I made 66 contacts and later in the month the troposperic ducting allowing contacts over interesting distances in the 70 centimeter and 2 meter band added to a sprint at the end. In 2019 I made a few more contacts than in the previous record year 2017.

Looking back at my amateur radio resolutions for 2019 I think most came true.

If I look at them one by one:
  • Keep learning morse! - I'm still working on my morse, but there is measurable improvement. I have learned the full set for the Belgian CW exam and I'm working on accuracy and speed.
  • Get more countries on more HF bands in the log - More countries and more slots on HF are in the log. I also use the club station to achieve that goal. The ARRL DXCC Award shows that I'm getting somewhere.
  • Moonbounce on 2 meter - I've listened on the right frequencies to the moon on 2 meter. Nothing heard.
  • Those digimode contests, and maybe a few phone contests - I participated in two phone contests and a number of digimode contests. No serious improvement in scores.
  • Operate HF outside - I operated HF outside. Not as much as I would like.
  • At least one satellite contact - Multiple satellite contacts have been made!
Now I have to think about 2020, but the year is still young.

Tags: , , , , ,
2019-08-05 Time for a new plot of the number of radio contacts
QSO count plot up to July 2019 Time for a new plot of the number of radio contacts. Months with contest(s) stand out again as they elevate the number of contacts. In July 2019 I participated in the DL-DX RTTY Contest 2019 and the IARU-HF Championship 2019. That last one has added a few countries to my list of countries confirmed in phone modes.

Tags: , ,

IPv6 check

Running test...
, reachable as koos+website@idefix.net. PGP encrypted e-mail preferred. PGP key 5BA9 368B E6F3 34E4 local copy PGP key 5BA9 368B E6F3 34E4 via keyservers

RSS
Meningen zijn die van mezelf, wat ik schrijf is beschermd door auteursrecht. Sommige publicaties bevatten een expliciete vermelding dat ze ongevraagd gedeeld mogen worden.
My opinions are my own, what I write is protected by copyrights. Some publications contain an explicit license statement which allows sharing without asking permission.
Other webprojects: Camp Wireless, wireless Internet access at campsites, The Virtual Bookcase, book reviews
This page generated by $Id: newstag.cgi,v 1.35 2021/11/09 13:09:49 koos Exp $ in 0.021599 seconds.