So Your Ruby Gem Won't Install, Huh?

I'm one of like three people in the world that does anything with Ruby on Windows, so I run into all kinds of fun things, because everything was written and tested on Linux. Fun, fun. For instance, a pretty common problem I've hit is not being able to install some gem.

Usually it will be some configuration issue or compiler error in code that wasn't quite cross-platform, so I'll need to go in and edit a Makefile and recompile an extension. In some cases, the gem comes with both a native extension and a Ruby wrapper, so all parts need to be installed. Copying them by hand into the proper places is fraught with peril.

I ran into this the other day and beat my head against the wall trying to figure out how to properly install the gem from the local sources that I'd fixed up. First, I tried typing rake, which looks like it should process the Rakefile much like a Makefile. Unfortunately, this spit out an error.

rake aborted!
No such file or directory - git ls-files
somegem.gemspec:19:in ``'
somegem.gemspec:19:in `block in <top (required)>'
somegem.gemspec:5:in `new'
somegem.gemspec:5:in `<top (required)>'
(See full trace by running task with --trace)

Wat. Who is trying to use git? Unpopular though I may be, I don't use git currently, and don't have it installed. So I peek into somegem.gemspec and what do I see?

  s.files = `git ls-files`.split("\n")

Uh, well, okay. A bit weird for it to try to pull in files from git. What weirdo would do this? As it turns out, almost every gem I have (each of which, successfully installed, mind you) has this line in there. Somehow gem is doing some magic with this and interpreting it, itself.

At this point I was able to kind of plot out a potential solution: I would "gem unpack" the somegem.gem file, modify the sources, repack it with "gem build", and do "gem install somegem.gem" to install from the file rather than pulling the source from the server.

This was all going so well until repacking the gem failed with the same git error. Turns out it tries to read the gemspec file, sees the git command, and tries to run it. UGH. But this blog has a happy conclusion, so read on.

I was able to make the gem commands do my bidding, actually. I used the command "gem spec --ruby somegem.gem > somegem.gemspec" to parse the gem file and rewrite it with the file list expanded. Something in there handled the git command; I don't know what, and I don't really care. This puts an expanded file list in the gemspec, so I can now repack it with "gem build" and finally install.



The Curious Case of the 9-Letter Bug

I recently fixed a very interesting bug that had been stumping me for a long time. It was a pain to debug--if you could really call it that--so I thought I'd share about it.

The bug repro steps go something like this:

  1. Start up my MungeTLS test EXE
  2. Start up my openssl test client, which does: openssl s_client -connect localhost:8879 -debug -state -pause -msg -legacy_renegotiation -tlsextdebug -tls1
  3. After the handshake completes, type any 9 characters and hit enter
  4. The server immediately terminates the connection

Whaaaaaa...? The number of typing shall be 9. It shall not be 8 (for then the bug won't repro); it shall not be 10 (for the bug surely won't manifest). It will be 9 only, no more, no less. Okay, that's not quite true... but we'll come to that in a little bit.

Finding the bug

Get ready for a rainbow explosion. Are you ready for this? You aren't even ready. Let's take a look at the various logs in play. First, let's see the client logs, where I type 'a' 9 times:

New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
    Protocol  : TLSv1
    Cipher    : AES128-SHA
    Master-Key: 2A76F3897A72CC25A19E78D0DFC7D289F3E1917CB00C764104ADFF9315EB1D
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1364315335
    Timeout   : 7200 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
write to 0x1cfe360 [0x1db6b46] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 ed 4b 27-e3 a8 0a 8f 79 fc da ea   .... .K'....y...
0010 - 28 85 bc e6 b0 c8 cb 4a-68 d7 b5 1d f8 59 12 46   (......Jh....Y.F
0020 - 3f 26 1a 37 b3 17 03 01-00 20 ef 1b 53 67 4a 0c   ?&.7..... ..SgJ.
0030 - a8 0e e9 15 fc 16 5e bf-b2 ff f1 16 30 53 b8 5a   ......^.....0S.Z
0040 - ef f0 aa 3c f1 50 83 33-f4 e0                     ...<.P.3..

SSL3 alert write:warning:close notify

Important information we see from this is that we're using TLS 1.0 with AES128-SHA. This tells us the format of the TLSCiphertext, as it relates to the encrypted body, IV, and MAC.

Great. Let's see the other side of the story--the server logs. Why oh why did you bail out like that, dear server?

successfully parsed TLSCiphertext. CT=23

decrypting. ciphertext (32 bytes):
ED 4B 27 E3 A8 0A 8F 79 FC DA EA 28 85 BC E6 B0 C8 CB 4A 68 D7 B5 1D F8 59 12 46 3F 26 1A 37 B3

setting IV to:
96 05 06 F0 35 34 26 7E 10 B7 39 90 8B F6 BA 6F

done initial decrypt: cb=21, size=32
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B

looking for 11 padding bytes

MAC text is 13 bytes
sequence number: 1
MAC hash text:
00 00 00 00 00 00 00 01 17 03 01 00 00 

received MAC:
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 

computed MAC:
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 

// there were actually two messages

successfully parsed TLSCiphertext. CT=23
decrypting. ciphertext (32 bytes):
EF 1B 53 67 4A 0C A8 0E E9 15 FC 16 5E BF B2 FF F1 16 30 53 B8 5A EF F0 AA 3C F1 50 83 33 F4 E0

setting IV to:
C8 CB 4A 68 D7 B5 1D F8 59 12 46 3F 26 1A 37 B3

FALSE WindowsCrypto.cpp:380 - (CryptDecrypt( hKeyNew, 0, fFinal, 0, &pvbDecrypted->front(), &cb)) == 80090005

That error at the end is NTE_BAD_DATA, which basically means, "haha something smells funny in this input data." It's about as useful as ear hair.

For a comparison, how about we take a look at a working case and see if we can spot anything different? Here is the MungeTLS log for the last message (the one that failed before), but this time I'm sending the letter 'a' ten times rather than nine.

successfully parsed TLSCiphertext. CT=23

decrypting. ciphertext (48 bytes):
C9 EF 6F FD CF 03 A3 0F 4D CD 85 9C 37 77 27 1C 09 4A 6D 35 58 A4 B6 50 37 6E 70 6F 5C 28 01 DA E1 9D 33 BF 6B 4D DA 84 D9 37 8C 5C 6E 94 2C A0 

setting IV to:
79 C4 E7 86 77 CC 9E C8 03 67 CC 76 75 FC 1F BA 

done initial decrypt: cb=33, size=48
61 61 61 61 61 61 61 61 61 61 0D 0A 66 17 E3 19 62 93 5D FD EC 33 C7 D7 CA FB 50 B4 A6 98 57 40 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F

looking for 15 padding bytes

decrypted fragment:
61 61 61 61 61 61 61 61 61 61 0D 0A

There are several interesting things here, all of which are actually kind of the same. First and most importantly, when I said that I was sending nine characters before in the failing case, that's wrong--I didn't realize the client was tacking on a CRLF, making it actually eleven characters. Eleven characters of payload plus a 20 byte SHA1 MAC is 31 bytes, which leaves exactly zero bytes of padding. That is, only the 1-byte padding length field would be present at the end of the block, having a value of 0x00, and there would be no actual padding bytes.

Likewise, with the addition of that extra character, we've pushed the payload length into the next ciphertext block, bringing it up to a total of 48 bytes, with the last block containing entirely padding (15 padding bytes + 1 padding length byte).

I'm certain at this point that CryptoAPI is not handling 0 padding bytes well, even though OpenSSL clearly expects it to. Sure enough, the bug reproes with 27 characters (27 + 20 byte MAC + 1 byte padding length = 3 16-byte ciphertext blocks) and other multiples.

The Fix

So how do we fix this monstrosity? It's silly, but the fix is actually very simple. This is a CryptoAPI quirk/bug, and the fix is luckily at that layer, too. Thank god, because I would hate have to handle this in another layer and introduce a leak into the "decrypt" abstraction.

CryptDecrypt takes a parameter called Final which tells whether the data that's been given is the last block of a chunk of data. It does a bunch of extra things when it sees this flag, among them is verifying the padding within the function and failing if it doesn't look right.

Previously, we were passing TRUE for Final. What happens if we pass FALSE instead? Well, CryptDecrypt will treat the incoming data like any block--it decrypts it and passes back the data without looking for padding or anything. After all, the "final" block isn't magic; it's still the same length as any other ciphertext block, and the padding bytes just happen to be interpreted that way because we arbitrarily decide they should be.

So, uh, yeah... that's the fix. Change a TRUE to a FALSE. The gnarliest bugs sometimes have the simplest fixes, don't they? Well, I'm a little disappointed I didn't get to dive into some more debugging this time, but perhaps it's for the best, huh?


MungeTLS... it's Out Now

Hello everybody. It's been quite a while, hasn't it? I have been mostly idle but not completely idle. The big news of today (okay, last week) is that I finally released MungeTLS! This represents a bit over a year of wall-clock time that I've spent on it, and probably the single longest project I've actually brought to any state resembling completion.

Of course, it's far from actually complete. What I shipped with it is the core TLS portion of the engine, of course, and the sample HTTPS server that calls into it and implements (at least superficially) all the callbacks. It's a neat proof of concept, but otherwise not terribly useful. I'm now presented with the even more difficult challenge of what to do next. I have a few ideas on the docket.

I think a TLS fuzzer would be a useful thing. However, it seems a company called Codenomicon has already built a TLS fuzzer. But on the other hand, it costs money, so maybe I have something to offer here, after all?

Along similar lines, a TLS conformance test suite would be useful. Both of these tools would be actual tools, not libraries for testing. The part that probably requires the most design thought is how to author the tests. Putting all of the tests directly into the EXE seems like a lot of code and cumbersome. I'll have to think about this some more.

And for both of these, I'd have to think about how I want to automate that in some standard way so that the client just repeatedly connects to walk through the list of tests. That's probably not too bad.

Among other, lighter ideas, I could make a simple library that wraps a TLS "pipe". You provide the certs, and it picks all the correct default behaviors to be a proper TLS channel. Of course, this functionality is already in most platforms, so it's not super compelling.

I had a thought to make a tool to TLS-ify a HTTP server--an HTTPS frontend for a HTTP server. Just a silly idea, but kind of fun.

So yeah, progress may slow down a while as I try to design some of these ideas in my head before putting virtual pen to virtual paper, but it is on my mind! Speaking of things slowing down, most of the code was actually done around, say, September 2012, but I was trying to get some people to code review it. Turns out it's hard to get people to give up their free time to look at some dude's boring code--and I don't blame them! I was finally able to wrangle some reading out of some guys on Twitter and from work (who are credited on the project page).

Welp, see yall next time I do something interesting!


A Quick Reminder About shared_ptr constness

Short blog here, just pointing out a weird problem I ran into the other day in the handling of shared_ptrs. I have some classes that keep shared_ptr members, and sometimes I need to hand out more references to callers... you know, share them. That getter might look a bit like this:

class Klass
    shared_ptr<int> GetX() { return x; }
    const shared_ptr<int> GetX() const { return x; }

    shared_ptr<int> x;

Right? Looks good? That's the normal way I write my pointers, anyway. But... there's something wrong here, and maybe you've already spotted it. Here's the code that triggers it, anyway.

int __cdecl wmain(int argc, wchar_t* argv[])
    Klass k;
    k.x = shared_ptr<int>(new int(3));

    const Klass* pk = &k;
    shared_ptr<int> spX(pk->GetX()); // invokes const GetX()

    *spX = 13;
    wprintf(L"x: %d\n", *spX); // x: 13

    return 0;

Hey, wait a second! This invokes the const getter, but then the caller is able to modify the value. How did that circumvent the const? Well, the problem is that you returned a const shared_ptr<int> value, pointing to a non-const int, so the shared_ptr object can't be modified, but the thing it points to can. Moreover, since it's returning a value (rather than a pointer), the const doesn't even matter.

The fix is simple. Here ya go:

shared_ptr<const int> GetX() const { return x; }

// and later in wmain()

shared_ptr<const int> spX(pk->GetX()); // invokes const GetX()

Now we get a compiler error, as we want:

main.cpp(35) : error C3892: 'spX' : you cannot assign to a variable that is const

This difference is analogous to the difference in constness on raw pointer types: int* const vs const int* (same as int const*). These rules don't seem very inuitive or nice, but you should know them if you're trying to make your code correct.


Sandwich Mechanics: An Honest and Unironic Chat About Sandwiches

Well, it's 2013, and it's looking like it'll be another great year for sandwiches... in that sandwiches still exist, and I'm still going to be eating them. Life is good.

Sometime back in December, my college friend Kirill hit me up on AIM, and we proceeded to have a totally unironic, legitimate conversation about sandwiches. It's got a smattering of stupid college-time inside jokes, but there's real magic here, you guys.

Kirill: OK
Kirill: your latest post about sandwiches made me realize something
Kirill: I toast the shit out of everything
knutaf: uh oh
Kirill: which means I might be MISSING OUT
Kirill: on soft texturezzz
Kirill: and whatnot
knutaf: that's possible
knutaf: i like toasting, too, but not on regular lunchmeat
knutaf: somehow it never comes out right for me
knutaf: kind of this weird mix of hot and cold
Kirill: OK here's something bufu
Kirill: for me, cheese works as a bridge between the hot and the cold
Kirill: I don't know how to describe it further
knutaf: i guess I kind of buy it
knutaf: but meat is the core for me, and that ends up unsatisfying in the end
Kirill: oh wow, TWO levels of qualifiers
Kirill: a provisional purchase
Kirill: AND a Keith-grade yig
knutaf: yeah, i've got some pretty strong reservations about that
knutaf: oh man, haven't thought about keith in like seven years
knutaf: "keith"
Kirill: no, I completely buy your reservations
Kirill: and even reserve another batch of your reservations
Kirill: in a meat-centric sandwich, things either serve the meat
knutaf: oh man, a twofold reservation
... idiotic nonsense ensues...
Kirill: a comment I'd meant to make earlier is that, for me, there are many sandwiches where meat is merely a participant, not the star of the show
Kirill: I can think of no better example than the corn beef sandwich
knutaf: really? corned beef is pretty strong in taste, if i remember right. it's been a while
Kirill: right, so check it
Kirill: you also add something equally strong -- sauerkraut -- and use cheese to bind them together
Kirill: not to mention that the rye bread has a very strong character of its own
knutaf: so it's more like a saurkreef sandwich than a corned beef with saurkraut
Kirill: sure
knutaf: i dig it. that sounds pretty good
Kirill: in the USSR, there's also a nontrivial fondness for open-faced sandwiches
knutaf: i have to be wary of saurkraut though. too often it ends up too wet, and sogs up the sandwich
Kirill: yeah, this is very important
knutaf: maybe not "too often", but it happens
Kirill: of all the ingredients in a corn beef sandwich, the kraut demands the most scrutiny quality-wise
knutaf: open faced sandwiches just seem inconvenient. two slices of bread is a totally fine ratio of bread to filling
Kirill: well, I'll describe to you the various pros and you can decide whether you buy them
Kirill: a beloved children's cartoon has a cat that urges kids to eat their open-faced sandwich face-down so that your tongue can directly contact the top parts
Kirill: it's completely cheeky and it's obvious that the cat is just making a joke
Kirill: but I was one of the kids that took it seriously
Kirill: and there are many meats and spreads alike where the cat's advice made a difference with respect to how much mileage you got out of the top stuff
Kirill: which invokes the further point that, yes, 2:1 bread to filling ratio is reasonable for many -- but not all -- things
knutaf: so you attempted to eat your open faced sandwich upside down? how'd that work out for ya?
Kirill: it's a subcultural joke at this point
Kirill: you'll see people eating it face-down and you KNOW where they got the concept from
knutaf: that's awesome. but doesn't your sandwich just fall all over the place?
Kirill: not if you're careful, which might be a terrible answer
knutaf: what the heck. i can't even picture this
Kirill: for me, it was a take-it-or-leave-it concept until my mom gave sandwiches with these truffle-infused cheese spreads and whatnot
Kirill: I didn't always enjoy them to the extent that my mom did
Kirill: but I did know that I wouldn't even be able to ACCESS the nuances to evaluate them had I added an extra portion of bread
knutaf: hmm
knutaf: this may not have been a luxury for back there, but sounds like the panini might be a good compromise here
Kirill: panini are a good counterexample
Kirill: they rely so much on the structure (i.e. the things being properly joined) that an open-faced rendition will be harder
Kirill: but to me, that's just part of a spectrum
Kirill: if I want to try out a new kind of salami for the first time, after I've had it without any bread, I'll have it on an open-faced sandwich first
knutaf: right, an open faced panini is probably not going to work, but the compressed bread reduces the ratio while still giving structure
Kirill: yeah, I was about to mention bread compression
Kirill: because slicing the bread thin Doesn't Cut It (HUUHUHHUHUHHH)
knutaf: hahahaa noooooo
knutaf: noooooo eeeeeeyes
knutaf: nooooo ryyyyyeeees
Kirill: less bread, however, is something I completely buy
Kirill: i.e. a small sandwich not too far from serving something on a ritz cracker
Kirill: incidentally
Kirill: caviar is something I'd NEVER serve close-faced
knutaf: yeah, I buy that. caviar is ultar expensive, though, so you really want as minimal a vehicle as possible
Kirill: I'd argue that it's not really about cost, but texture
Kirill: how familiar are you with caviar?
knutaf: oh. i've barely ever had caviar. maybe just once
Kirill: I smell an experiment
knutaf: it didn't make much of an impression on me. but i have a pretty pedestrian palate sometimes
Kirill: I can mail you a jar of caviar for you to perform sandwich experiments on
knutaf: like, are you suggesting using it as a spread?
Kirill: good caviar is not easy to find and many crooks sell shitty ones
Kirill: well, I can describe it how most Russians serve it
Kirill: there are two kinds of well-known caviar
Kirill: black and red
Kirill: black might be banned or something
Kirill: but it's rare to the point that I would have to do a LOT of homework to trust it
Kirill: red is easier to find
Kirill: black is very small granules and more subtle, red is larger granules with a stronger flavor
Kirill: so I'll be talking about red
Kirill: most of the time, you serve it as an appetizer on tiny slices of a white baguette
Kirill: butter or another spread optional
Kirill: I personally think that dark breads, like rye, bring to it another dimension
Kirill: but I can also respect the view that chooses the white bread as a blank slate to let the caviar shine
knutaf: I could see either way working, yeah
knutaf: I think I had red, the one time I had it
Kirill: I think this is what I'll do
Kirill: I'll find out from relatives who sells the most trustworthy caviar and mail you a can
Kirill: and you should just treat it like an alien mailed you some alien bizsauce
Kirill: and see where it works and doesn't
knutaf: i'm intrigued. sounds great
knutaf: gonna get foodz in a sec. i'll probably end up uploading this chat to the blog (edited some)
knutaf: it's too sandwich related not to
Kirill: word
Kirill: spread the word