Java : fetching web pages using NIO - timeouts

I'm developing a call back framework on top of Java NIO, that will allow for easier implementations of asynchronous page downloads.

As you start adding sockets to a single selector, you get into request time-out issues. This happens because you normally add all the sockets to the selector and then wait for sockets that are ready. Some sockets, after waiting for a few seconds for the request, will timeout at the remote end.

The HTTP response code for this is 408 - it is easy to handle this by removing the SelectionKey from the selector, and adding the socket back to the selector. Since, the application will now be dealing with fewer sockets, the next time, the socket will most probably not time out.

The latency is mostly in domain name resolution. I will next try doing name resolution totally outside socket registration, so I don't eat into the connection time with the slow DNS lookup calls. This should reduce the number of 408 responses, and I should be able to add more sockets to the selector.