Linux Today: Linux News On Internet Time.

Editor's Note: Snooping the Internet With Netcraft

Apr 11, 2009, 00:01 (6 Talkback[s])
(Other stories by Carla Schroder)

by Carla Schroder
Managing Editor

Netcraft is a fascinating site all full of information about the Web. If you're not familiar with common Linux/UNIX networking tools, you might be a bit taken aback at how much you can learn about what other people are doing without ever leaving your comfortable computer chair.

One of Netcraft's most popular features is "What's that site running?" Look on the top left of Netcraft.com. Enter any URL, for example linuxtoday.com, and you learn all sorts of snoopy stuff in a nicely-organized Site Report. The Linux Today Site Report reveals that LT runs on Linux and Apache. The history only goes back to 2008, so let's see if snooping on LinuxPlanet turns up anything interesting. The linuxplanet.com report (not www.linuxplanet.com) shows that LinuxPlanet was running on Solaris/Apache until 2003, when it moved to Linux/Apache.

Linux Planet runs on Linux/Apache, but has an embarrassing uptime average- 19.29 days between reboots. Maybe the admins or devs have been doing a lot of work. Maybe someone clumsy trips over the power cord a lot.

We see that Linux Today has no uptime bragging rights either, with a current uptime of only 42 days. There is an uptime graph which shows a 90-day moving average of 57.55 days between reboots. However, that doesn't really tell you anything because LT resides on several load-balanced servers. So uptime has little relationship to availability. (Measuring uptimes is tricky anyway, due to peculiarities in the Linux kernel,as the uptime FAQ explains.)

Fun and Games With Market Share

The April 2009 Web Server Survey page is full of fun data, such as a summary from August 1995 - April 2009. Apache has been dominant since 1996. Netcraft sorts Web server share into two categories: all sites and active sites. All sites include both parked hostnames and active sites. The active sites category is quite a bit smaller, and presents a more accurate real-world picture. Microsoft famously inflated IIS' (Internet Information Server) share in 1996 when GoDaddy migrated from Apache to IIS, and the bulk of the transferred domains were parked, rather than active sites.

According to the April survey, there were a total of 231,510,169 sites that Netcraft was able to find. 45.95% ran Apache, and 29.27% ran IIS. Number three at 12.49% is qq.com, a Chinese domain powered by QZHTTP, that contains almost 29 million sites. This appeared suddenly in the February Netcraft survey. What is QZHTTP? According to The Chinese (Web Servers) Are Coming, most likely it is rebranded Apache.

Rounding out the survey are Google and nginx.

When we move down the page to the active domains, the numbers change:

Apache  37,268,162  46.52%
Microsoft  30,116,432  37.59%
Google  6,131,966  7.65%  
nginx  2,602,109  3.25%
qq.com  0  0.00%
Where did all the qq.com domains go? Netcraft does not say.

Million Busiest Sites

This is where Apache really struts its stuff:
Apache  663,649  67.56% 
Microsoft 185,763  18.91% 
nginx  31,064  3.16%
Google  16,088  1.64%

Most Reliable Hosting Company Sites

Most Reliable Hosting Company Sites in March 2009 displays the top Web hosts and their operating systems. FreeBSD is consistenly dominant in the top 10, and Linux always makes a respectable showing. Amazingly, one company that runs Windows Server 2003 is number 5, which in my opinion is a testament to the amazing skill of their server admins.