It's been a long time since I've done much programming at the machine level. These days, most of the code I write is in C or Java and I have little need for low-level control over hardware. Even the Linux drivers I've written contain no assembly code. But this weekend I got a little taste of low-level hacking again. I'd forgotten how much fun it could be.
VMware has a virtual disk format of its own, but it's not quite what we want, so we interpose on all of its disk accesses using LD_PRELOAD. Unfortunately, for the llseek() system call, VMware does an end-run around the system C library and executes the system call directly. This prevents our interposition from working.
Lucky for us, Mendel Rosenblum, one of the original authors of VMware, is in our research group. For GSX 1.0, he just changed the sources to use llseek() in the C library and recompiled. But we upgraded to GSX 2.0 recently and he hadn't yet got around to recompiling it for us, so we were stuck for a bit. I decided that going ahead and fixing it myself would be a good adventure. Without benefit of sources, I'd have to make it a binary patch.
Finding the system call in the binary was easy. The opcode to make a system call is "int $0x80", which assembles to CD 80 and appeared in the binary only a dozen times or so. llseek() is system call 0x8c, so examining each occurrence narrowed it down to two places that needed to be fixed up. Using "objdump -d /usr/bin/vmware" to get a full disassembly of the VMware binary, I confirmed that both spots contained ordinary system call sequences.
The next step was to devise a way to call out to my own code instead of invoking the syscall. My first thought was to add llseek() to the binary's dynamic symbol table and PLT and so on, and invoke it directly through that. I went off and grabbed a copy of the ELF specification and read it through. Afterward, I examined the vmware binary with objdump and readelf and concluded that this was impractical since it would require relocating everything in the binary.
My second thought, and the one that I ended up using, was that I could use dlsym() to dynamically find llseek(). The binary already imported dlsym(), so this approach didn't require any additions to the dynamic symbol table. I wrote up a stub to substitute, assembled it, pasted it into the binary using a hex editor, and fired up the result.
It of course crashed. But I was expecting this, so next I ran it under gdb and did some judicious breakpointing and single-stepping and cursing. Some hours later, I had a working 32-byte patch that stubbed through to my preload library, and I went to bed about 4:00 a.m. That was last night. Today I modified it to call through to the interposition library, tested it, tweaked it a bit for easy adaptability to other versions of VMware, wrote a couple of useful scripts to install it into a given binary, and checked it into the CVS. Success!
In other news, I'm getting paid twice as much during the summer due to weird university rules. It's like free money. Consequently, I went out yesterday to do a little shopping for new shirts at the Stanford Shopping Center, an expensive shopping center on campus. Entirely by coincidence, all of the major department stores there were having big sales. I went to Nordstrom, Bloomingdales, and Macy's. Nordstrom had absolutely beautiful clothing, but even with money burning a hole in my pocket I couldn't see myself paying $98 for one shirt. Bloomingdales and Macy's were much more reasonable and I ended up with 4 shirts that I liked for about $100 total.
I spent a good bit of time this weekend cleaning out the apartment. It's a four-bedroom apartment, but since Bernard left for Kenya on Tuesday I've been the only one here. He will be back in fall, but the other two have moved out for good. I cleaned out the big storage room, breaking down all the empty boxes and throwing away anything that wasn't Bernard's or mine, and I swept everything carpeted. I also threw away everything in the bathroom that wasn't his or mine. Yesterday I went through the refrigerator and threw out everything old or disgusting, and today I pulled out all of the shelves and the drawers and washed them in the sink, and cleaned out the cupboards too. Now I just need to mop the floors, which are in awful shape too.