many options of lsof

this is handy reference to the ways of using lsof.

Mac and launchd - removing programs that refuse to be killed

Mac uses the launchd daemon to start processes on boot time, much like the init.d scripts in Linux. Except, launchd has a configuration file (with an extention .plist) for each such process, and these files are in a few different places. So if you want to stop an auto launch, you will be hunting around, which I just did.

The official doc states all the locations a plist file.

In my case, I was using a educational prototype from UW called vanish. This spawns a couple of processes using launchd. So if I kill the processes, they come right back up. I found the plist files under ~/LaunchAgents and removing them did the trick.

Java ByteBuffer : how does this work?

The first time you encounter the ByteBuffer, you may run into some surprises. The function that flips most folks is in fact ByteBuffer.flip(). To understand the flip() and to be not flipped by it and other such idioms, we will look at what this class is and how it should be used.

Basically, a ByteBuffer allows us to read data repeatedly from some input, like a non-blocking socket. ByteBuffer keeps track of the position of the last byte written, so you don't need to. You can keep writing to the same ByteBuffer and rest assured that previous data will not be over-written.

This is pretty handy in asynchronous I/O (using Java NIO package) as data from asynchronous sockets don't always arrive all at once. We need to map buffers to sockets and keep reading until there is no more data from the remote end.

So what about this flip()? Well, the way the ByteBuffer class was designed, data is read to the buffer starting at position and upto limit. Data is written starting at position and upto limit as well. So, if you followed that, after reading some data from a socket, the ByteBuffer position would be advanced, and reading now will not get any data as position is at the end of the buffer. So flip() basically sets the position to 0 (start), and limit to the position (previous position to be exact, or rather the end of useful input).

So think about read and write operations on the ByteBuffer manipulating data within position and limit and you will see more clearly the need to flip once in a while.

Happy flipping!

Reading UTF-8 data from asynchronous sockets to the file system

Using asynchronous sockets, data is generally read into ByteBuffer objects. The general pattern is to read multiple times until there is no more data, and each time when the ByteBuffer is full, transfer to a larger buffer, like a ByteArrayOutputStream.

Now if you want to manipulate data collected (which is now in the ByteArrayOutputStream) as a String, it has to be decoded. This can be done using the CharsetDecoder object like this:

ByteArrayOutputStream outStrm;

// read data to outStrm using nio

CharBuffer charBuffer = CharBuffer.allocate(outStrm.size());
byte[] ba = outStrm.toByteArray();
ByteBuffer byteBuffer = ByteBuffer.wrap(ba);
Charset charset = Charset.forName( "UTF-8" );
decoder = charset.newDecoder();
CoderResult res = decoder.decode(byteBuffer, charBuffer, true);
res = decoder.flush(charBuffer);
String out = charBuffer.flip().toString();

However, all this decoding does is translating UTF-8 characters to their respective code points. As a result, we can't save this data to a file (OutputStream) correctly.

If you were to print the out string to the display, it is not guaranteed to print valid UTF-8 characters. Of course it will work for the single byte characters, but not necessarily for the multi-byte characters. Ex: 0xca a0 represents a non-breaking space with a code point of 0xA0. The above decoding will decode this to the code point 0xA0, but if you now write this to an output stream, it will not be stored as UTF-8, as the decoding stripped the UTF-8 and replaced it with code points.

So the correct approach is to simply write the byte buffer to an output stream like this:

outStrm.writeTo(System.out);

This will present UTF-8 characters to the output stream and thus the file will be saved as correct UTF-8 data.

A lenient URL decoder for Java

The URLDecoder class in the JDK insists on doing a strict parsing of escape characters in an encoded URL string. Sometimes, the application might want to decode correctly escaped string sequences and leave incorrect sequences intact. In fact the Sun documentation states that this aspect of decode handling is implementation dependent. However, Sun's implementation is strict - it throws an exception when it encounters improper escape sequences rather than treating them like regular text.

I couldn't find a lenient implementation so hand-crafted this from the original source for URLDecode class found here.

Following is the lenient decode.

    public static String decodeLenient(String s, String enc)
           throws UnsupportedEncodingException {

       boolean needToChange = false;
       StringBuffer sb = new StringBuffer();
       int numChars = s.length();
       int i = 0;

       if (enc.length() == 0) {
           throw new UnsupportedEncodingException("URLDecoder: empty string enc parameter");
       }

       while (i < numChars) {
           char c = s.charAt(i);
           switch (c) {
               case '+':
                   sb.append(' ');
                   i++;
                   needToChange = true;
                   break;
               case '%':
                   /*
                  # * Starting with this instance of %, process all
                  # * consecutive substrings of the form %xy. Each
                  # * substring %xy will yield a byte. Convert all
                  # * consecutive bytes obtained this way to whatever
                  # * character(s) they represent in the provided
                  # * encoding.
                  # */

                   // (numChars-i)/3 is an upper bound for the number
                   // of remaining bytes
                   byte[] bytes = new byte[(numChars - i) / 3];
                   int pos = 0;

                   while (((i + 2) < numChars) &&
                           (c == '%')) {
                       String hex = s.substring(i + 1, i + 3);
                       try {
                           bytes[pos] =
                                   (byte) Integer.parseInt(hex, 16);
                           pos++;
                       } catch (NumberFormatException e) {
                           sb.append(new String(bytes, 0, pos, enc));
                           sb.append("%");
                           sb.append(hex);
                           pos = 0;
                       }
                       i += 3;
                       if (i < numChars)
                           c = s.charAt(i);
                   }

                   sb.append(new String(bytes, 0, pos, enc));

                   // A trailing, incomplete byte encoding such as
                   // "%x" will be treated as unencoded text
                   if ((i < numChars) && (c == '%')) {
                       for (; i<numChars; i++) {
                           sb.append(s.charAt(i));
                       }
                   }

                   needToChange = true;
                   break;
               default:
                   sb.append(c);
                   i++;
                   break;
           }
       }

       return (needToChange ? sb.toString() : s);
   }
}

Remove intrusive bing popups from content sites

Today as I was pursuing the moneycentral.com site, in my quest to acquire the financial knowledge of the Wall Street robber barons, I stumbled on the latest feature from Bing : annoying hover over popups that show a search listing.

They are rather annoying for two reasons:
1) They appear over the text you are reading
2) It is very difficult to navigate around them as all you need to do is to have the mouse cursor on the link area for the helpful popup to appear

So I modified my previous GreaseMonkey script to handle this as well. Fortunately it is easy as this popup is fired off an anchor tag with an attribute of "itxtdid". All we need to do is remove the attribute, and the popup gets disabled.

Below is the new script. Enjoy.

// ==UserScript==
// @name           test
// @namespace      http://userscripts.org/thushara/
// @include        *
// ==/UserScript==
var allLinks, thisLink;
allLinks = document.evaluate(
 '//a[@href]',
 document,
 null,
 XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
 null);
for (var i = 0; i < allLinks.snapshotLength; i++) {
 thisLink = allLinks.snapshotItem(i);
 // do something with thisLink
 if (thisLink.href.substring(0,19)=="http://www.bing.com") {
   thisLink.removeAttribute("href");
 }
 if (thisLink.hasAttribute("itxtdid")) {
   thisLink.removeAttribute("itxtdid");
 }
}

Ubuntu 8.04 - sound on flash player with firefox

If sound works generally, but not inside Flash Player (in FireFox), try this:

thushara@agni:~$ sudo apt-get install libflashsupport
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
libflashsupport
0 upgraded, 1 newly installed, 0 to remove and 131 not upgraded.
Need to get 8326B of archives.
After this operation, 65.5kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com hardy/universe libflashsupport 1.9-0ubuntu1 [8326B]
Fetched 8326B in 0s (14.9kB/s)
Selecting previously deselected package libflashsupport.
(Reading database ... 99009 files and directories currently installed.)
Unpacking libflashsupport (from .../libflashsupport_1.9-0ubuntu1_i386.deb) ...
Setting up libflashsupport (1.9-0ubuntu1) ...

Processing triggers for libc6 ...
ldconfig deferred processing now taking place
thushara@agni:~$


restart FireFox and sound should be available...