web 2.0 image clusters

at http://livemocha.com we use a cluster of lighttpd servers to handle the requests for image, audio/video files. the reason to use lighttpd instead of apache is to do with the superior capability of lighttpd to handle large number of concurrent requests with low latency.

the apache server will use one process to serve a request. while the request is being served, that process will not accept any other requests. any other request needs to be served by a different apache process.

for a low volume server, this is not a problem. there are enough apache processes lieing around to serve the concurrent requests.

it does become a problem with high volume. a typical web page may have 20-30 images, a css file, some javascript files. the browser will first make a request to apache (assuming apache is the web server) for the main web page. then as the browser parses the page, it will issue additional requests to the image files etc found on the page.

if these requests go back to apache, they will each have to be served by one apache process one at a time. the KeepAlive option at apache **may** provide some relief by keeping the TCP connection open at apache so that the browser can keep using that for the images. but this doesn't change the fact that one apache process will always block other requests while it is serving the image.

the image request is mostly I/O bound - meaning most of the time to fulfill the request will be spent waiting for the image file to be fetched from a hard drive, perhaps in some network share. in the case of livemocha, most images were served off a NFS server. So it is quite wasteful to have a process sitting around doing nothing but waiting for I/O to complete.

Having a process waiting around is bad because it takes memory. There is only a finite amount of it in the server, and when there are more processes than that can fit memory the O/S will have to swap a processes into disk to make room for another process. When the O/S context switches to the swapped process, it has to be brought back into memory again, possibly resulting in another process being swapped, so on and so forth. this is a vicious cycle that can and will bring the server to its knees - this phenomena is known as the "slashdot effect".

lighttpd can serve multiple requests from one process. it uses asynchronous I/O available in linux 2.6 kernels (ex: epoll) to multiplex multiple requests in one process. this uses very low memory and can perform well under the "slashdot effect".

nginx is another server that performs very well using similar technologies. we decided on lighttpd mainly because some other key sites were known to use it. from benchmarks, nginx seemed to be superior.

Ubuntu 8.04 - setup wireless to start on boot

Make sure your /etc/network/interfaces file looks like this:

auto lo
iface lo inet loopback
auto wlan0
iface wlan0 inet dhcp
wireless-essid your-essid
wireless-ap 00:25:43:DE:AC:2B
wireless-mode Managed

How do you find your essid and access point (ap)? In my case, I cheated by using the "Network configuration" icon on the top left. If you click on it, it will show the available interfaces, just pick the correct one. Then issue the iwconfig command on the shell, and you should see the essid and ap.

ex:

thushara@gini-sisila:~$ iwconfig
lo no wireless extensions.

eth0 no wireless extensions.

wlan0 IEEE 802.11b+/g+ ESSID:"my-essid" Nickname:"acx v0.3.36"
Mode:Managed Frequency:2.437 GHz Access Point: 00:25:43:DE:AC:2B
Bit Rate:24 Mb/s Tx-Power=15 dBm Sensitivity=1/3
Retry min limit:7 RTS thr:off
Power Management:off
Link Quality=31/100 Signal level=3/100 Noise level=0/100
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:0 Missed beacon:0

You can also find these parameters by logging into your wireless router's admin interface. You need to wire the computer to the router (or use another computer connected to the router) for this.