20120319

Public Key Encryption in Windows

One of the things I try to do on this blog is talk about problems I encounter and solutions I arrive at, with the slim hope that someone else may stumble upon it and say, "Oh, thank god, someone figured this out!

In the project I've been working on in my spare time (this mysterious, shadowy project that will not be revealed yet), I have a need to do some public/private key RSA encryption. The project is targeting Windows, so I needed to learn how CryptAPI on Windows works. The short answer is, "in a very complicated way." I have no idea about OpenSSL as an alternative. I didn't want to introduce a weird dependency at this time. Ultimately, I think those crypto-library-specific considerations could be abstracted, anyway.

So my seemingly simple task before me at this time is to take a certificate--having both the public and private key--and encrypt a blob of data using the private key and decrypt it with the public key (and vice versa).

The actual two functions used for encrypting and decrypting an arbitraty blob of data are CryptEncrypt and CryptDecrypt, respectively. These at least are pretty concise: they take the blob of data, the length, and a handle to the key to use to do the encrypting or decrypting (a HCRYPTKEY). It's this last part that's a pain to get.

In my code I need to start from a certificate. I'll assume that the certificate is already installed in the Windows certificate stores. Finding this certificate is not too difficult. I call CertOpenStore and indicate the "current user" store location (as opposed to the local machine store location) and the "my" (a.k.a. personal) store within it. That gives you a HCERTSTORE.

With that handle, you can call CertFindCertificateInStore and pass the subject name the certificate is issued to as part of the search. From this, you get a PCERT_CONTEXT, or rather, a PCCERT_CONTEXT (just a const one). This represents a single certificate and can be used to look up things related to it.

Now we need to get a hold of both keys from the certificate. The way to get keys from a certificate requires first acquiring a "context", or an instance of a cryptographic security provider, or something. The terminology is weird and abstract, and the docs don't define them well. Regardless, it is a value of type HCRYPTPROV that you're looking for.

One function that gets this is CryptAcquireContext, but you can only use if you know the "container name" where the key you need lives. I generated this certificate using makecert.exe. I don't even know what a container is in this context. Argh, I used the word "context" again! Well, it seems any cryptographic key in Windows lives in a "container", but I'm not sure why. But it does, and you'll have to work with it.

There's luckily a second function you can call that doesn't require knowledge of the key container name: CryptAcquireCertificatePrivateKey. Of course, this only gets you the private key. We'll have to meander down the path of cryptic functions to find the public one later.

As an aside, once you have the private key context, you can now retroactively find out the key container name by calling CryptGetProvParam with PP_CONTAINER. The answer is unexciting: just a GUID that is probably generated by makecert.exe.

This function can return one of two types of key--indicated in the pdwKeySpec out-variable--a signature key or an exchange key (AT_SIGNATURE or AT_EXCHANGE, respectively). I'm not really clear at this point on why this distinction exists, but in my usage, it is governed by the -sky command line flag I passed to makecert.exe when I created the certificate. For encrypting traffic to be sent to another machine, you want an exchange key. I guess a signature key is used for signing stuff? I don't really know.

You're almost there! Call CryptGetUserKey to get a coveted HCRYPTKEY that you can use for encrypting. This is, of course, the private key, since you got it from a private key context.

I need to point out something that confused me greatly at first. I ran my code to do just the encryption repeatedly on the same data. I expected to get the same output every time; after all, I'm using the same RSA key and blob of data. Instead, I got vastly different output every time: I couldn't find any common pattern of bytes within. I was really confused.

cleartext:
61 00 62 00 63 00 64 00

encrypted:
47 A8 4C AF 10 0B 24 42 DD D1 80 44 5C AB 37 E6 F9 0D 3D FA 43 60 CD 04 CD 22 01
6E 8C FB CF 51 AE 8A 06 C0 1D 39 62 EB D4 FB 9A 30 13 E3 AB CC 3F DA 26 C0 84 3F
97 9E 32 2B 2C 17 81 68 76 1F 82 0A 78 0D 8C D4 94 4D 10 F0 51 BC 6F D4 0B DF 0D
6C A1 72 AB 28 17 61 63 9C 67 29 C8 CC 9C 8A 54 67 24 96 10 5A 0C ED E5 91 12 3B
F4 82 88 5B 18 19 DE 63 07 D1 14 16 7E A7 B1 C5 E0 8D 36 A1


cleartext:
61 00 62 00 63 00 64 00

encrypted:
A1 BE 08 97 B0 C6 CC F6 81 00 93 82 AF 4E 38 55 06 61 DA B7 D2 13 6B 54 8E 2A 88
E0 21 84 D4 5D EC 56 71 6A E8 3A E8 42 62 11 4D B4 A0 90 17 86 C5 8E C6 46 9A 60
B4 B7 F0 1E 55 DD 47 A2 2A B8 44 B5 A2 B7 23 D8 2B E2 FF CF 0A 4D 79 A4 BD 41 57
B0 43 86 AA 44 B4 F9 D2 0E 34 19 AF 70 A7 6A 6B 39 C9 4D 2A 78 60 97 12 B1 D6 60
07 21 8F 8F 84 8E 43 01 2C A5 3A D0 04 AE 12 26 BE 61 C3 13

I thought that my code must be wrong. There must be some flag I need to set. Maybe it's automatically including some salt in the encryption. Yet another thing about cryptography I don't know about. Actually, I was on the right track there: the reason it looks completely different is that the cleartext is first augmented with PKCS#1 padding (section 8.1 of RFC 2313), which consists of random bytes of data plus a padding length field so it can be removed after decryption. Of course any change in the cleartext, including this, will yield a vastly different ciphertext.

Now that we've successfully encrypted with the private key, let's decrypt with the public key! But getting the HCRYPTKEY for the public key is weird and unintuitive. Maybe I missed some handy function that does it for you, but basically the only way I found is to call CryptImportPublicKeyInfo.

Let's do that thing where we try and match up inputs and outputs, the same way you do when plugging in your audio/video equipment when hooking up your TV. CryptImportPublicKeyInfo needs a HCRYPTPROV. What's something that gives an HCRYPTPROV? No, CryptAcquireCertificatePrivateKey won't work! We need the public key! How about just plain old CryptAcquireContext? There are flags you can pass to it to create a new key container. Maybe we'll create a temporary key container into which we'll import the public key info. It sounds hacky, I know, but it works.

if (!CryptAcquireContextW(
         &hPubProv,
         L"pub_key",
         MS_ENHANCED_PROV,
         PROV_RSA_FULL,
         0))
{
    if (GetLastError() == NTE_BAD_KEYSET)
    {
        if (!CryptAcquireContextW(
                 &hPubProv,
                 L"pub_key",
                 MS_ENHANCED_PROV,
                 PROV_RSA_FULL,
                 CRYPT_NEWKEYSET))
        {
            hr = HRESULT_FROM_WIN32(GetLastError());
            goto error;
        }
    }
    else
    {
        hr = HRESULT_FROM_WIN32(GetLastError());
        goto error;
    }
}

Still, we need a CERT_PUBLIC_KEY_INFO to supply to CryptImportPublicKeyInfo. Well, that function is called "import", so is there an "export", too? Yup, there's CryptExportPublicKeyInfoEx. This function also needs a HCRYPTPROV, presumably different than the "empty" one we're about to import into. If you read the docs, it says, "The CryptExportPublicKeyInfoEx function exports the public key information associated with the provider's corresponding private key." Hmm, I wonder if we can give the HCRYPTPROV from the private key. Spoilers: yes, we can.

But now we have everything we need to try to decrypt with this public key! Let's do it!

Umm, why is CryptDecrypt returning NTE_NO_KEY? "No key"? There's definitely a key! I just imported it! So here's the deal. For some reason on Windows you can't use this method to decrypt with a public key. I don't quite know why. I found some forum post saying I need an AT_SIGNATURE key to decrypt with the public key, but I wasn't able to get that to work either. I don't really know. Luckily, in my app, I don't need to, so I am leaving this unsolved.

Instead, if I encrypt with the public key, I'm readily able to decrypt with the private key, and that's good enough for me!

encrypt with public, decrypt with private:
cleartext:
61 00 62 00 63 00 64 00 

encrypting
found we need 128 bytes for ciphertext

encrypted:
3F 02 BF AD 87 96 26 E8 CB 3D E9 94 50 27 46 28 4F 16 E6 24 EE C5 6A A3 3C 7D 01
51 BB D4 6D 95 C8 14 08 42 B8 04 B6 4A D8 DB 63 8B 98 5E 33 56 0F 7B DD 91 C0 1A
C2 EF 1C 68 D6 D3 94 48 B3 60 B6 86 94 5A 90 B7 03 85 9A DE 45 BC 9C 21 4F 03 54
D1 48 1E 7A 09 61 AE AD 35 B2 33 FC 95 77 94 6C 6E C6 18 9F D7 39 EB 03 6C 22 FA
D6 CF 97 B5 9F A9 0D 63 4E C5 FD D7 A9 3D 98 FE 3A 4E F5 4D 

decrypted:
61 00 62 00 63 00 64 00 

cleartext and decrypted are the same


encrypt with private, decrypt with public:
cleartext:
61 00 62 00 63 00 64 00 

encrypting
found we need 128 bytes for ciphertext

encrypted:
E5 A6 D1 99 C2 E0 1A E3 F6 C2 D3 47 17 51 8A 22 8F 78 2C F3 C4 01 83 FD 1A 67 D5
EE 38 C9 8F 5B DF 6E C1 D9 ED 40 04 CE 31 15 2E 46 BD 77 9D 2A 14 59 8A 0B 58 07
BD EF 71 F3 A9 2A 0D F1 A2 E2 84 22 46 05 63 38 74 E8 61 C7 8F 25 C2 BC A0 5F 80
D2 07 02 7E 15 9A 64 66 55 89 14 23 68 57 D7 F8 6F 8B 86 EE 35 E5 96 A0 11 F4 3A
90 BF D2 39 22 02 F7 A5 5F 63 0A 8F 15 6A 19 25 FA 47 B4 22 

decrypting
error? 8009000D

I'd be a real jerk not to share the code at this point, since it's such a pain to figure all this stuff out. Here is a zip file containing the whole project:

  • The source code, main.cpp
  • The Makefile to build it with
  • create_cert.cmd, which creates and installs the certificate hard-coded into the app
  • makecert.exe, which create_cert.cmd uses

Adventures in crypto! Just the beginning for me, I fear. Yowza.

0 comments:

Post a Comment