So Your Ruby Gem Won't Install, Huh?

I'm one of like three people in the world that does anything with Ruby on Windows, so I run into all kinds of fun things, because everything was written and tested on Linux. Fun, fun. For instance, a pretty common problem I've hit is not being able to install some gem.

Usually it will be some configuration issue or compiler error in code that wasn't quite cross-platform, so I'll need to go in and edit a Makefile and recompile an extension. In some cases, the gem comes with both a native extension and a Ruby wrapper, so all parts need to be installed. Copying them by hand into the proper places is fraught with peril.

I ran into this the other day and beat my head against the wall trying to figure out how to properly install the gem from the local sources that I'd fixed up. First, I tried typing rake, which looks like it should process the Rakefile much like a Makefile. Unfortunately, this spit out an error.

>rake
rake aborted!
No such file or directory - git ls-files
somegem.gemspec:19:in ``'
somegem.gemspec:19:in `block in <top (required)>'
somegem.gemspec:5:in `new'
somegem.gemspec:5:in `<top (required)>'
(See full trace by running task with --trace)

Wat. Who is trying to use git? Unpopular though I may be, I don't use git currently, and don't have it installed. So I peek into somegem.gemspec and what do I see?

  s.files = `git ls-files`.split("\n")

Uh, well, okay. A bit weird for it to try to pull in files from git. What weirdo would do this? As it turns out, almost every gem I have (each of which, successfully installed, mind you) has this line in there. Somehow gem is doing some magic with this and interpreting it, itself.

At this point I was able to kind of plot out a potential solution: I would "gem unpack" the somegem.gem file, modify the sources, repack it with "gem build", and do "gem install somegem.gem" to install from the file rather than pulling the source from the server.

This was all going so well until repacking the gem failed with the same git error. Turns out it tries to read the gemspec file, sees the git command, and tries to run it. UGH. But this blog has a happy conclusion, so read on.

I was able to make the gem commands do my bidding, actually. I used the command "gem spec --ruby somegem.gem > somegem.gemspec" to parse the gem file and rewrite it with the file list expanded. Something in there handled the git command; I don't know what, and I don't really care. This puts an expanded file list in the gemspec, so I can now repack it with "gem build" and finally install.

WHEW. THE END.

The Curious Case of the 9-Letter Bug

I recently fixed a very interesting bug that had been stumping me for a long time. It was a pain to debug--if you could really call it that--so I thought I'd share about it.

The bug repro steps go something like this:

1. Start up my MungeTLS test EXE
2. Start up my openssl test client, which does: openssl s_client -connect localhost:8879 -debug -state -pause -msg -legacy_renegotiation -tlsextdebug -tls1
3. After the handshake completes, type any 9 characters and hit enter
4. The server immediately terminates the connection

Whaaaaaa...? The number of typing shall be 9. It shall not be 8 (for then the bug won't repro); it shall not be 10 (for the bug surely won't manifest). It will be 9 only, no more, no less. Okay, that's not quite true... but we'll come to that in a little bit.

Finding the bug

Get ready for a rainbow explosion. Are you ready for this? You aren't even ready. Let's take a look at the various logs in play. First, let's see the client logs, where I type 'a' 9 times:

New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES128-SHA
    Session-ID:
    Session-ID-ctx:
    Master-Key: 2A76F3897A72CC25A19E78D0DFC7D289F3E1917CB00C764104ADFF9315EB1D
75580AEC9FDA9A351E4FBD7DD5F84084
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1364315335
    Timeout   : 7200 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
aaaaaaaaa
write to 0x1cfe360 [0x1db6b46] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 ed 4b 27-e3 a8 0a 8f 79 fc da ea   .... .K'....y...
0010 - 28 85 bc e6 b0 c8 cb 4a-68 d7 b5 1d f8 59 12 46   (......Jh....Y.F
0020 - 3f 26 1a 37 b3 17 03 01-00 20 ef 1b 53 67 4a 0c   ?&.7..... ..SgJ.
0030 - a8 0e e9 15 fc 16 5e bf-b2 ff f1 16 30 53 b8 5a   ......^.....0S.Z
0040 - ef f0 aa 3c f1 50 83 33-f4 e0                     ...<.P.3..

SSL3 alert write:warning:close notify

Important information we see from this is that we're using TLS 1.0 with AES128-SHA. This tells us the format of the TLSCiphertext, as it relates to the encrypted body, IV, and MAC.

Great. Let's see the other side of the story--the server logs. Why oh why did you bail out like that, dear server?

successfully parsed TLSCiphertext. CT=23

decrypting. ciphertext (32 bytes):
ED 4B 27 E3 A8 0A 8F 79 FC DA EA 28 85 BC E6 B0 C8 CB 4A 68 D7 B5 1D F8 59 12 46 3F 26 1A 37 B3

setting IV to:
96 05 06 F0 35 34 26 7E 10 B7 39 90 8B F6 BA 6F

done initial decrypt: cb=21, size=32
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B

looking for 11 padding bytes

MAC text is 13 bytes
sequence number: 1
MAC hash text:
00 00 00 00 00 00 00 01 17 03 01 00 00 

received MAC:
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 

computed MAC:
74 CB AB C8 2D 62 76 84 4A 80 F5 C6 8D 14 90 8E D7 F4 59 03 


// there were actually two messages


successfully parsed TLSCiphertext. CT=23
decrypting. ciphertext (32 bytes):
EF 1B 53 67 4A 0C A8 0E E9 15 FC 16 5E BF B2 FF F1 16 30 53 B8 5A EF F0 AA 3C F1 50 83 33 F4 E0

setting IV to:
C8 CB 4A 68 D7 B5 1D F8 59 12 46 3F 26 1A 37 B3

FALSE WindowsCrypto.cpp:380 - (CryptDecrypt( hKeyNew, 0, fFinal, 0, &pvbDecrypted->front(), &cb)) == 80090005

That error at the end is NTE_BAD_DATA, which basically means, "haha something smells funny in this input data." It's about as useful as ear hair.

For a comparison, how about we take a look at a working case and see if we can spot anything different? Here is the MungeTLS log for the last message (the one that failed before), but this time I'm sending the letter 'a' ten times rather than nine.

successfully parsed TLSCiphertext. CT=23

decrypting. ciphertext (48 bytes):
C9 EF 6F FD CF 03 A3 0F 4D CD 85 9C 37 77 27 1C 09 4A 6D 35 58 A4 B6 50 37 6E 70 6F 5C 28 01 DA E1 9D 33 BF 6B 4D DA 84 D9 37 8C 5C 6E 94 2C A0 

setting IV to:
79 C4 E7 86 77 CC 9E C8 03 67 CC 76 75 FC 1F BA 

done initial decrypt: cb=33, size=48
61 61 61 61 61 61 61 61 61 61 0D 0A 66 17 E3 19 62 93 5D FD EC 33 C7 D7 CA FB 50 B4 A6 98 57 40 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F

looking for 15 padding bytes

decrypted fragment:
61 61 61 61 61 61 61 61 61 61 0D 0A

There are several interesting things here, all of which are actually kind of the same. First and most importantly, when I said that I was sending nine characters before in the failing case, that's wrong--I didn't realize the client was tacking on a CRLF, making it actually eleven characters. Eleven characters of payload plus a 20 byte SHA1 MAC is 31 bytes, which leaves exactly zero bytes of padding. That is, only the 1-byte padding length field would be present at the end of the block, having a value of 0x00, and there would be no actual padding bytes.

Likewise, with the addition of that extra character, we've pushed the payload length into the next ciphertext block, bringing it up to a total of 48 bytes, with the last block containing entirely padding (15 padding bytes + 1 padding length byte).

I'm certain at this point that CryptoAPI is not handling 0 padding bytes well, even though OpenSSL clearly expects it to. Sure enough, the bug reproes with 27 characters (27 + 20 byte MAC + 1 byte padding length = 3 16-byte ciphertext blocks) and other multiples.

The Fix

So how do we fix this monstrosity? It's silly, but the fix is actually very simple. This is a CryptoAPI quirk/bug, and the fix is luckily at that layer, too. Thank god, because I would hate have to handle this in another layer and introduce a leak into the "decrypt" abstraction.

CryptDecrypt takes a parameter called Final which tells whether the data that's been given is the last block of a chunk of data. It does a bunch of extra things when it sees this flag, among them is verifying the padding within the function and failing if it doesn't look right.

Previously, we were passing TRUE for Final. What happens if we pass FALSE instead? Well, CryptDecrypt will treat the incoming data like any block--it decrypts it and passes back the data without looking for padding or anything. After all, the "final" block isn't magic; it's still the same length as any other ciphertext block, and the padding bytes just happen to be interpreted that way because we arbitrarily decide they should be.

So, uh, yeah... that's the fix. Change a TRUE to a FALSE. The gnarliest bugs sometimes have the simplest fixes, don't they? Well, I'm a little disappointed I didn't get to dive into some more debugging this time, but perhaps it's for the best, huh?

MungeTLS... it's Out Now

Hello everybody. It's been quite a while, hasn't it? I have been mostly idle but not completely idle. The big news of today (okay, last week) is that I finally released MungeTLS! This represents a bit over a year of wall-clock time that I've spent on it, and probably the single longest project I've actually brought to any state resembling completion.

Of course, it's far from actually complete. What I shipped with it is the core TLS portion of the engine, of course, and the sample HTTPS server that calls into it and implements (at least superficially) all the callbacks. It's a neat proof of concept, but otherwise not terribly useful. I'm now presented with the even more difficult challenge of what to do next. I have a few ideas on the docket.

I think a TLS fuzzer would be a useful thing. However, it seems a company called Codenomicon has already built a TLS fuzzer. But on the other hand, it costs money, so maybe I have something to offer here, after all?

Along similar lines, a TLS conformance test suite would be useful. Both of these tools would be actual tools, not libraries for testing. The part that probably requires the most design thought is how to author the tests. Putting all of the tests directly into the EXE seems like a lot of code and cumbersome. I'll have to think about this some more.

And for both of these, I'd have to think about how I want to automate that in some standard way so that the client just repeatedly connects to walk through the list of tests. That's probably not too bad.

Among other, lighter ideas, I could make a simple library that wraps a TLS "pipe". You provide the certs, and it picks all the correct default behaviors to be a proper TLS channel. Of course, this functionality is already in most platforms, so it's not super compelling.

I had a thought to make a tool to TLS-ify a HTTP server--an HTTPS frontend for a HTTP server. Just a silly idea, but kind of fun.

So yeah, progress may slow down a while as I try to design some of these ideas in my head before putting virtual pen to virtual paper, but it is on my mind! Speaking of things slowing down, most of the code was actually done around, say, September 2012, but I was trying to get some people to code review it. Turns out it's hard to get people to give up their free time to look at some dude's boring code--and I don't blame them! I was finally able to wrangle some reading out of some guys on Twitter and from work (who are credited on the project page).

Welp, see yall next time I do something interesting!

A Quick Reminder About shared_ptr constness

Short blog here, just pointing out a weird problem I ran into the other day in the handling of shared_ptrs. I have some classes that keep shared_ptr members, and sometimes I need to hand out more references to callers... you know, share them. That getter might look a bit like this:

class Klass
{
    public:
    shared_ptr<int> GetX() { return x; }
    const shared_ptr<int> GetX() const { return x; }

    shared_ptr<int> x;
};

Right? Looks good? That's the normal way I write my pointers, anyway. But... there's something wrong here, and maybe you've already spotted it. Here's the code that triggers it, anyway.

int __cdecl wmain(int argc, wchar_t* argv[])
{
    Klass k;
    k.x = shared_ptr<int>(new int(3));

    const Klass* pk = &k;
    shared_ptr<int> spX(pk->GetX()); // invokes const GetX()

    *spX = 13;
    wprintf(L"x: %d\n", *spX); // x: 13

    return 0;
}

Hey, wait a second! This invokes the const getter, but then the caller is able to modify the value. How did that circumvent the const? Well, the problem is that you returned a const shared_ptr<int> value, pointing to a non-const int, so the shared_ptr object can't be modified, but the thing it points to can. Moreover, since it's returning a value (rather than a pointer), the const doesn't even matter.

The fix is simple. Here ya go:

shared_ptr<const int> GetX() const { return x; }

// and later in wmain()

shared_ptr<const int> spX(pk->GetX()); // invokes const GetX()

Now we get a compiler error, as we want:

main.cpp(35) : error C3892: 'spX' : you cannot assign to a variable that is const

This difference is analogous to the difference in constness on raw pointer types: int* const vs const int* (same as int const*). These rules don't seem very inuitive or nice, but you should know them if you're trying to make your code correct.

Sandwich Mechanics: An Honest and Unironic Chat About Sandwiches

Well, it's 2013, and it's looking like it'll be another great year for sandwiches... in that sandwiches still exist, and I'm still going to be eating them. Life is good.

Sometime back in December, my college friend Kirill hit me up on AIM, and we proceeded to have a totally unironic, legitimate conversation about sandwiches. It's got a smattering of stupid college-time inside jokes, but there's real magic here, you guys.

Kirill: OK

Kirill: your latest post about sandwiches made me realize something

Kirill: I toast the shit out of everything

knutaf: uh oh

Kirill: which means I might be MISSING OUT

Kirill: on soft texturezzz

Kirill: and whatnot

knutaf: that's possible

knutaf: i like toasting, too, but not on regular lunchmeat

knutaf: somehow it never comes out right for me

knutaf: kind of this weird mix of hot and cold

Kirill: OK here's something bufu

Kirill: for me, cheese works as a bridge between the hot and the cold

Kirill: I don't know how to describe it further

knutaf: i guess I kind of buy it

knutaf: but meat is the core for me, and that ends up unsatisfying in the end

Kirill: oh wow, TWO levels of qualifiers

Kirill: a provisional purchase

Kirill: AND a Keith-grade yig

knutaf: yeah, i've got some pretty strong reservations about that

knutaf: oh man, haven't thought about keith in like seven years

knutaf: "keith"

Kirill: no, I completely buy your reservations

Kirill: and even reserve another batch of your reservations

Kirill: in a meat-centric sandwich, things either serve the meat

Kirill: or GET THA FUCC OUTT

knutaf: oh man, a twofold reservation

... idiotic nonsense ensues...

Kirill: a comment I'd meant to make earlier is that, for me, there are many sandwiches where meat is merely a participant, not the star of the show

Kirill: I can think of no better example than the corn beef sandwich

knutaf: really? corned beef is pretty strong in taste, if i remember right. it's been a while

Kirill: right, so check it

Kirill: you also add something equally strong -- sauerkraut -- and use cheese to bind them together

Kirill: not to mention that the rye bread has a very strong character of its own

knutaf: so it's more like a saurkreef sandwich than a corned beef with saurkraut

Kirill: sure

knutaf: i dig it. that sounds pretty good

Kirill: in the USSR, there's also a nontrivial fondness for open-faced sandwiches

knutaf: i have to be wary of saurkraut though. too often it ends up too wet, and sogs up the sandwich

Kirill: yeah, this is very important

knutaf: maybe not "too often", but it happens

Kirill: of all the ingredients in a corn beef sandwich, the kraut demands the most scrutiny quality-wise

knutaf: open faced sandwiches just seem inconvenient. two slices of bread is a totally fine ratio of bread to filling

Kirill: well, I'll describe to you the various pros and you can decide whether you buy them

Kirill: a beloved children's cartoon has a cat that urges kids to eat their open-faced sandwich face-down so that your tongue can directly contact the top parts

Kirill: it's completely cheeky and it's obvious that the cat is just making a joke

Kirill: but I was one of the kids that took it seriously

Kirill: and there are many meats and spreads alike where the cat's advice made a difference with respect to how much mileage you got out of the top stuff

Kirill: which invokes the further point that, yes, 2:1 bread to filling ratio is reasonable for many -- but not all -- things

knutaf: so you attempted to eat your open faced sandwich upside down? how'd that work out for ya?

Kirill: it's a subcultural joke at this point

Kirill: you'll see people eating it face-down and you KNOW where they got the concept from

knutaf: that's awesome. but doesn't your sandwich just fall all over the place?

Kirill: not if you're careful, which might be a terrible answer

knutaf: what the heck. i can't even picture this

Kirill: for me, it was a take-it-or-leave-it concept until my mom gave sandwiches with these truffle-infused cheese spreads and whatnot

Kirill: I didn't always enjoy them to the extent that my mom did

Kirill: but I did know that I wouldn't even be able to ACCESS the nuances to evaluate them had I added an extra portion of bread

knutaf: hmm

knutaf: this may not have been a luxury for back there, but sounds like the panini might be a good compromise here

Kirill: panini are a good counterexample

Kirill: they rely so much on the structure (i.e. the things being properly joined) that an open-faced rendition will be harder

Kirill: but to me, that's just part of a spectrum

Kirill: if I want to try out a new kind of salami for the first time, after I've had it without any bread, I'll have it on an open-faced sandwich first

knutaf: right, an open faced panini is probably not going to work, but the compressed bread reduces the ratio while still giving structure

Kirill: yeah, I was about to mention bread compression

Kirill: because slicing the bread thin Doesn't Cut It (HUUHUHHUHUHHH)

knutaf: hahahaa noooooo

knutaf: noooooo eeeeeeyes

knutaf: nooooo ryyyyyeeees

Kirill: AAAAAAAAAAAAAAA

Kirill: less bread, however, is something I completely buy

Kirill: i.e. a small sandwich not too far from serving something on a ritz cracker

Kirill: incidentally

Kirill: caviar is something I'd NEVER serve close-faced

knutaf: yeah, I buy that. caviar is ultar expensive, though, so you really want as minimal a vehicle as possible

Kirill: I'd argue that it's not really about cost, but texture

Kirill: how familiar are you with caviar?

knutaf: oh. i've barely ever had caviar. maybe just once

Kirill: I smell an experiment

knutaf: it didn't make much of an impression on me. but i have a pretty pedestrian palate sometimes

Kirill: I can mail you a jar of caviar for you to perform sandwich experiments on

knutaf: like, are you suggesting using it as a spread?

Kirill: good caviar is not easy to find and many crooks sell shitty ones

Kirill: well, I can describe it how most Russians serve it

Kirill: there are two kinds of well-known caviar

Kirill: black and red

Kirill: black might be banned or something

Kirill: but it's rare to the point that I would have to do a LOT of homework to trust it

Kirill: red is easier to find

Kirill: black is very small granules and more subtle, red is larger granules with a stronger flavor

Kirill: so I'll be talking about red

Kirill: most of the time, you serve it as an appetizer on tiny slices of a white baguette

Kirill: butter or another spread optional

Kirill: I personally think that dark breads, like rye, bring to it another dimension

Kirill: but I can also respect the view that chooses the white bread as a blank slate to let the caviar shine

knutaf: I could see either way working, yeah

knutaf: I think I had red, the one time I had it

Kirill: I think this is what I'll do

Kirill: I'll find out from relatives who sells the most trustworthy caviar and mail you a can

Kirill: and you should just treat it like an alien mailed you some alien bizsauce

Kirill: and see where it works and doesn't

knutaf: i'm intrigued. sounds great

knutaf: gonna get foodz in a sec. i'll probably end up uploading this chat to the blog (edited some)

knutaf: it's too sandwich related not to

Kirill: word

Kirill: spread the word

Kirill: BIIIIIIIIIIIIIIH