2008-03-11 (#) 10 years ago
Did some work on The Virtual Bookcase. First of all Amazon notified me that amazon web services was going to disable version 3 of the API which I was still using (yes, a year after the first notification that it was going to be ended .. not that much time for virtualbookcase at the moment). So time to do some PHP programming and redo the stuff for version 4. I also noticed that the bot for the russian search site Yandex was causing high amounts of hits but would not even return a direct link to www.virtualbookcase.com when searching on virtual bookcase at yandex. So I wanted to disable Yandex in robots.txt but I could not easily find the right name to put in robots.txt because all the help at Yandex is also in Russian. Finally I found some hints at this page describing robots.txt in russian that the right User-Agent probably is Yandex. And the robots.txt for cisco.com agrees.