Hotboot and SxS

How many times do I need to fix hotboot? I fixed it once before, but after recompiling Ruby 1.9 at some point, it stopped working again. Now with my performance sweep out of the way, I debugged this yet again. The way I fixed this before relied heaviliy on calling _get_osfhandle and _open_osfhandle in the right places to convert between CRT filehandles and Windows HANDLEs.

I used the Win32API class to do this:

GET_OSFHANDLE = Win32API.new("msvcrt", "_get_osfhandle", ["I"], "I")
OPEN_OSFHANDLE = Win32API.new("msvcrt", "_open_osfhandle", ["I", "I"], "I")

# and to call them:
def Hotboot.handleFromFileDesc(fd)
    return GET_OSFHANDLE.Call(fd)
end # end function handleFromFileDesc

def Hotboot.fdFromHandle(handle, flags=0)
    return OPEN_OSFHANDLE.Call(handle, flags)
end # end function fdFromHandle

What I found was that after recompiling Ruby 1.9, it no longer links against msvcrt.dll, but instead against msvcr90.dll. This is a big problem, because those two functions modify private DLL state about filehandles. If my code is using one dll's version, and Ruby is using another's, then it won't see each other's effects as they should. Of course, to fix this I need to make my code use msvcr90.dll.

The naive approach, changing "msvcrt" to "msvcr90" doesn't work, because msvcr90.dll is a SxS module. Just making that change leads to the infamous R6034 error. I spent a bunch of time following red herrings about adding manifest files, copying the CRT dll locally, and turning on loader snaps, all the while getting ever more frustrated.

It was while I was writing up a question to ask on stackoverflow that I got back into thinking up other ideas. I thought: well, if LoadLibrary is having trouble because of SxS configuration, maybe I shouldn't try to load it, but instead reference the instance that's already loaded. After all, by the time this code is called, ruby has been running for a while, and certainly will have msvcr90.dll already loaded.

But even using GetModuleHandle wouldn't work, so I started thinking it had something to do with where I was loading the library from. Previously I wasn't specifying a path to the library, just giving the name. Now I tried giving the full path to the dll in the sxs folder, and for whatever reason, it worked!

So I wrote a little program to call EnumProcessModules on my ruby instance, find the full path of the dll I need, and print it out. I parsed this output and passed this full path to Win32API.new. Life is good again.

DL

As I was searching around I happened to see someone mention that Win32API was being deprecated after Ruby 1.9. That's not cool... I use that! It turns out that Win32API is just a wrapper around a Ruby library called DL. And somehow that link is the only reasonable reference/guide for it that I've been able to find. How crazy is that?

DL is cool. You already saw how I was loading the functions before. Now see how it's done after, with DL.

module CRT
    extend DL::Importer

    # ...

    modpath = Utilities.findLoadedModuleFullPath("msvcr90.dll")
    dlload(modpath)

    extern("int _get_osfhandle(int)")
    extern("int _open_osfhandle(int, int)")

    #...
end

# and to call them:
def Hotboot.handleFromFileDesc(fd)
    return CRT._get_osfhandle(fd)
end # end function handleFromFileDesc

def Hotboot.fdFromHandle(handle, flags=0)
    return CRT._open_osfhandle(handle, flags)
end # end function fdFromHandle

I actually kind of like the new way using DL, especially the part where you just write out the function signature instead of what looks like packed array templates.