News items for tag virtualbookcase - Koos van den Hout

2022-08-26 Limiting URLs to scan with wapiti
I wanted to use wapiti as scanner to check for other vulnerabilities in The Virtual Bookcase after receiving a report about a cross-site scripting vulnerability. Wapiti is open source and free, which is a fitting price for scanning a hobby project site.

I quickly ran into wapiti taking hours to scan because of the URL structure of the site: all /book/detail/x/y URLs map to one handler that deals with the X and Y parameters in SQL queries. Yes those queries are surrounded by very defensive checking and I use positional parameters. Everything to avoid SQL injection and becoming the next Little Bobby Tables.

Wapiti has no simple method that I can find to crawl for a list of URLs and stop at that to allow for selecting the list of URLs to scan. But it has an option to minimize crawling and import a list of additional URLs to scan so I used that option to get at the same result.

Gathering URLs was done with wget:
$ wget --spider -r http://developer.virtualbookcase.com 2>&1 | grep '^--' | egrep -v '\.(css|jpg|gif|png)' | awk '{ print $3}' > developer.virtualbookcase.com-urls.txt
After that I sorted the file with URLs and threw out a lot of them, making sure all the scripts with several variants of input were still tested.

With that list I start wapiti with some special options. It still needs a starting url at -u so I give it the root but I limit the crawling with the depth parameter -d 1 and the max files parameter --max-files-per-dir 50. Then I add the additional urls from the earlier scan with the -s parameter. It's a lot of tweaking but it does the trick.
$ wapiti -u http://developer.virtualbookcase.com/ -d 1 --max-files-per-dir 50 -s developer.virtualbookcase.com-urls.txt -o ~/wapiti/ -v 2
No vulnerabilities were found. I found one PHP warning which only triggered in the kind of corner case a web vulnerability scanner causes, or an attacker. So I fixed that corner case too.

Tags: , , ,
2022-08-25 D'oh!!! A cross-site scripting vulnerability in one of my own sites
I received a responsible disclosure report of a vulnerability in The Virtual Bookcase.

I will directly admit I haven't done a lot of maintenance on this site in the past few years but I want to keep my sites secure.

The report came via openbugbounty.org and has no details about the vulnerability, so I am not 100% sure where the reported vulnerability is. But based on the report text XSS (Cross Site Scripting) and a peek in the access-log looking for specific requests I found I made a beginner mistake in dealing with a search query: displaying it as-is within an HTML context. I immediately fixed that error in the site.

Now I wonder why it took so long for me to realize the error of my ways or for someone to notice it!

Checking the logs some more finds huge amounts of attempts at SQL injection, which is a vulnerability I am very aware of and where I put up standard defenses. But this is the first time a security researcher made me aware of the cross-site scripting vulnerability.

Update: I contacted the reporter about the vulnerability who responded quickly inquiring about the possible bounty for finding the bug. As this is a site that hasn't delivered any income in years the best I can do is a mention in the credits of the site or on a separate hall of fame.

Update: I also started a vulnerability scanner on the site myself, to find any other vulnerabilities I might have missed. This scanner is going through the development site at the moment. Like many other scanners it doesn't see by default how certain urls all map to the same PHP script.

I already committed a few minor updates to improve handling of corner cases in not set variables and other things popping up in the scan.

Update 2022-09-23: I realized the reporter has never responded with the actual bug information.

Tags: , , ,
2020-01-03 No longer amazon.com associate on The Virtual Bookcase
I received a message from amazon that The Virtual Bookcase no longer qualifies as an amazon.com associate. That was no big surprise as I haven't done a lot of maintenance on the site and haven't added a lot of content in the last years.

The only serious maintenance was for the migration to the new web server where php 7.0 is the standard version. I wish to some day migrate to perl but haven't found time yet.

So I removed all amazon affiliate links I could find. This also means I can't use the amazon.com API anymore.
Read the rest of No longer amazon.com associate on The Virtual Bookcase

Tags: , ,
2018-01-03 Fixing stuff in The Virtual Bookcase for PHP 7
After spending an evening fixing scripts on The Virtual Bookcase to make them run in PHP 7 and make them safer at the same time I came to the conclusion that I still don't like php.

My conclusion is that if I want to maintain sites I'd rather redo them in perl. I noticed any serious maintenance on the scripts of The Virtual Bookcase was 9 years ago (!). That was also when I had the habit of writing maintenance scripts in perl and web code in php. The upside is that a part of the page-generating code is already available in perl.

But a rewrite is a task for another day. For now the site works cleanly in PHP 7 (and 5) and I can go on to the next task for moving the homeserver.
Read the rest of Fixing stuff in The Virtual Bookcase for PHP 7

Tags: , , , ,
2017-08-19 Moving virtualbookcase.com to https
I received a notification from the google webmaster program that chrome browser would start showing security warnings on http://www.virtualbookcase.com/ due to the search box there.

The simple solution: make the site correctly available via https and redirect to the https version. I found out I already started doing the first bit and therefore the conversion was easy. Now with encrypted connections: The Virtual Bookcase.

Tags: , , ,
2015-03-27 Overly interested Amazon EC2 nodes
On Camp Wireless and The Virtual Bookcase I see the following pattern in the access logs:
2620:108:700f::36bc:aade - - [27/Mar/2015:13:27:11 +0100] "GET / HTTP/1.1" 302 298 "-" "curl/7.36.0"
2406:da00:ff00::36e2:d963 - - [27/Mar/2015:13:27:38 +0100] "GET / HTTP/1.1" 302 298 "-" "curl/7.36.0"
Constant requests, 2 or 3 per minute from Amazon EC2 IPv6 addresses just requesting the / using curl. Over the day I now see 1334 unique addresses with at most 5 requests from one url.

The same pattern as described in Stange stream of HTTP GET requests in apache logs, from amazon ec2 instances - Server Fault with no real answer to the why.

It's not a problematic amount of traffic, I'd just like to understand what is happenning!

Tags: , , , , ,
2014-08-04 IPv6 visitor stats 2014
Time to count IPv6 visitor percentage to different websites again:
SiteJuly 2009July 2010July 2011July 2012July 2014
http://idefix.net/ my homepage 1% 2% 2% 3% 4%
http://netwerk.pcgg.nl/ hcc!pcgg netwerkgroep 2% 2% 2% 3% 1%
http://weather.idefix.net/ weather maps < 1% 5% 6% 7% 6%
http://bbs.idefix.net/ BBS files 1% 1% 1% 3%
http://webcam.idefix.net/ the webcam < 1% 1% < 1% 2% 2%
http://www.virtualbookcase.com/ The Virtual Bookcase < 1% 1% 1% 4% 87%
http://www.camp-wireless.org/ Camp Wireless < 1% 1% 1% 3% 70%
http://weatherstation.idefix.net/ Weather station Utrecht Overvecht 1%
Interesting numbers. Results for The Virtual Bookcase and Camp Wireless are totally skewed thanks to some IPv6 bot constantly checking the site from constantly changing IPv6 addresses .. but without privacy extensions enabled.

Method: unique IPv6 addresses seen in the whole month / total unique addresses seen in the whole month.

Tags: , , , , , ,
2012-08-03 (#)
Time to count IPv6 visitor percentage to different websites again:
SiteJuly 2009July 2010July 2011July 2012
http://idefix.net/ my homepage 1% 2% 2% 3%
http://weather.idefix.net/ weather maps < 1% 5% 6% 7%
http://netwerk.pcgg.nl/ hcc!pcgg netwerkgroep 2% 2% 2% 3%
http://bbs.idefix.net/ BBS files 1% 1% 1%
http://webcam.idefix.net/ the webcam < 1% 1% < 1% 2%
http://www.virtualbookcase.com/ The Virtual Bookcase < 1% 1% 1% 4%
http://www.camp-wireless.org/ Camp Wireless < 1% 1% 1% 3%
Interesting numbers. Websites with a more 'general' audience are now also growing this year.

Method: unique IPv6 addresses seen in the whole month / total unique addresses seen in the whole month.

Tags: , , , , , , , ,
2012-01-26 (#)
After the updates to virtualbookcase.com last weekend I noticed one other area of serious bit rot: character set damage. The pages were still charset=US-ASCII and this went noticeably wrong in HTML For Dummies® by Ed Tittel and Steve James. But there are only a few places where I had to change that, and the databases are all UTF8 anyway.

Funny is how the amount of visitors is slowly growing since this weekend. A few improvements and a few new links from my homepage, and suddenly more visitors.

Tags: , ,
2012-01-22 (#)
I got around to posting reviews for books I read recently: But I noticed quite some bit rot on The Virtual Bookcase. Some scripts needed updating before I could add a book or post a review. And the reviews queue had been filled with lots of spam.

Tags: , ,

IPv6 check

Running test...
, reachable as koos+website@idefix.net. PGP encrypted e-mail preferred. PGP key 5BA9 368B E6F3 34E4 local copy PGP key 5BA9 368B E6F3 34E4 via keyservers

RSS
Meningen zijn die van mezelf, wat ik schrijf is beschermd door auteursrecht. Sommige publicaties bevatten een expliciete vermelding dat ze ongevraagd gedeeld mogen worden.
My opinions are my own, what I write is protected by copyrights. Some publications contain an explicit license statement which allows sharing without asking permission.
Other webprojects: Camp Wireless, wireless Internet access at campsites, The Virtual Bookcase, book reviews
This page generated by $Id: newstag.cgi,v 1.37 2022/02/15 21:48:19 koos Exp $ in 0.036169 seconds.