Sunday, April 17, 2005

Inside look into Microsoft TCP/IP connection limit workaround

I couldn't find out any information on how this was hacked. I assume that a binary differ was used against the orginal tcpip.sys vs the patched one. Halvar Flake's bindiff plugin with IDA-Pro, quite probably.

However, after analyzing the fix, I found it instructive to disassemble tcpip.sys to find out what the hack accomplished.

Basically, tcpip.sys (XP SP2) introduces a pointer value in the image that points to the maximum number of concurrent connections. In a couple of places in the code a check is made to see if the current number of connections exceed this value.

The maximum number of connections is set with this instruction:

mov dword ptr [_ActiveOpenProgressThreshold], 0Ah

The hack changes this to:

mov dword ptr [_ActiveOpenProgressThreshold], 0FFFFFEh

The disassembly makes more sense if you have symbols for tcpip.sys. You can obtain this by setting the symbol path to Microsoft public symbol servers:

set _NT_SYMBOL_PATH=SRV*c:\symbols*

No comments: