Lesser Adventures of Symmetric Encryption in Windows

I'm continuing my adventures with encryption in Windows. Last time, we looked at public/private key encryption using RSA. It was kind of a pain. Next I got symmetric key encryption working. Spoiler alert: it was much eaiser.

Most of the reason it was probably easier is from the fact that we're generating a symmetric key from scratch, rather than getting a pre-generated key from a certificate. In fact, if I had generated the RSA public/private key pair at runtime in the previous exercise, it probably would have been tantamount in its simplicitly.

Much of the code from the RSA encryption can be carried over wholesale: the calls to CryptEncrypt and CryptDecrypt, for instance, are identical. The new calls that are introduced are for CryptGenKey and CryptDeriveKey. The former is used to create a new, random key; the latter is used to deterministically create a new key based on a seed value.

The only real hiccup that you may choke up from your programming throat here is the selection of the algorithm. Windows supports a large number of cryptographic hash and key algorithms, which are all outlined on the page for ALG_ID. The trick is that they require different "cryptographic service providers". I'm not really sure why this concept even exists, but that's how it is.

So if you want to make an RC4 key, you need to use the Base Cryptographic Provider, but if you want an AES key, or, oddly, a SHA1 hash, you'll need an instance of the AES and RSA provider. Maybe it's not that odd; maybe I just don't know enough crypto for it to make sense.

The way you specify the provider to use is to pass it into your call to CryptAcquireContextW (hey, remember this little guy from last time?). And you'll need to go through the same dance of creating a new "key set" for the generated key to be stored in, too.

When generating or deriving the key, make sure to flag it with CRYPT_EXPORTABLE, so that you can export the key to a blob later. Well, maybe you don't need to, but if you're going to be sending it to the other end of a secure channel, you'll need to.

When it comes to deriving the key, the way to specify the base data is by using a hash. You have some piece of raw data that is your "seed", and the key is derived from this hash value. Creating a hash is a pretty straightforward two-step process: call CryptCreateHash to create an empty hash object and CryptHashData to actually hash data in the object.

It took a few tries to get the right parameters to CryptExportKey, specifically the dwBlobType parameter, which indicates the format in which to export the key. I thought SIMPLEBLOB (export a session key) sounded pretty good, but that actally requires a public key that automatically encrypts the session key for safety. It turns out this actually would suit my purposes here, but I wanted to do all of this "the hard way", so I wanted the raw key value. For that, you pass PLAINTEXTKEYBLOB. Not too bad.

That's it! Here's the (elided) output from the program. I wrote a few basic tests to validate my assumptions about the randomness or determinism of the types of keys I was generating and their encryption/decryption results. In my code, I'm using AES128 for the symmetric keys and SHA256 for the hashes.

>symenc.exe abc

------------- Testing Random Key Generation ------------
random keys:
D9 99 9D A9 6B 54 37 D1 C1 51 26 F0 10 63 A9 81 
E5 90 F3 E6 D6 BB D1 35 6A 65 D5 91 22 4B D6 C6 
PASS: random keys are different


------------- Testing Same Fixed Key Generation ------------
hash value:
BA 5E C5 1D 07 A4 AC 0E 95 16 08 70 44 31 D5 9A 02 B2 1A 4E 95 1A CC 10 50 5A 8D C4 07 C5 01 EE 

hash value:
BA 5E C5 1D 07 A4 AC 0E 95 16 08 70 44 31 D5 9A 02 B2 1A 4E 95 1A CC 10 50 5A 8D C4 07 C5 01 EE 

fixed keys:
BA 5E C5 1D 07 A4 AC 0E 95 16 08 70 44 31 D5 9A 

BA 5E C5 1D 07 A4 AC 0E 95 16 08 70 44 31 D5 9A 
PASS: fixed keys are the same

------------- Testing Different Fixed Key Generation ------------
hash value:
9D 1E 0E 2D 94 59 D0 65 23 AD 13 E2 8A 40 93 C2 31 6B AA FE 7A EC 5B 25 F3 0E BA 2E 11 35 99 C4 

hash value:
4D 7B 3E F7 30 0A CF 70 C8 92 D8 32 7D B8 27 2F 54 43 4A DB C6 1A 4E 13 0A 56 3C B5 9A 0D 0F 47 

fixed keys:
9D 1E 0E 2D 94 59 D0 65 23 AD 13 E2 8A 40 93 C2 

4D 7B 3E F7 30 0A CF 70 C8 92 D8 32 7D B8 27 2F 
PASS: fixed keys are different


--------- random key - encrypt and decrypt ---------------
exported random key:
63 30 4B 6B CF 93 97 A1 53 47 C4 FE 8F 1F 97 2D 

cleartext:
61 00 62 00 63 00 64 00 

encrypted:
A0 86 DF 42 23 DC 3B 71 90 97 57 1F 75 CC FD 56 

decrypted:
61 00 62 00 63 00 64 00 
PASS: cleartext and decrypted are the same


--------- fixed key - encrypt and decrypt ---------------

hash value:
CA 97 81 12 CA 1B BD CA FA C2 31 B3 9A 23 DC 4D A7 86 EF F8 14 7C 4E 72 B9 80 77 85 AF EE 48 BB 

exported fixed key:
CA 97 81 12 CA 1B BD CA FA C2 31 B3 9A 23 DC 4D 

cleartext:
61 00 62 00 63 00 64 00 

encrypted:
A0 86 DF 42 23 DC 3B 71 90 97 57 1F 75 CC FD 56 

decrypted:
61 00 62 00 63 00 64 00 
PASS: cleartext and decrypted are the same


--------- mismatched keys - encrypt and decrypt ---------------
cleartext:
61 00 62 00 63 00 64 00 

encrypted:
A0 86 DF 42 23 DC 3B 71 90 97 57 1F 75 CC FD 56 

decrypting
PASS: got expected hr 80090005 for bad decryption

Woah, I didn't notice this until reading this output in a web browser, but the AES128 key derived from a SHA256 hash is just a truncation of it. What the...? Is that normal? It could explain why these SHA hashes are provided by the same cryptographic service provider as AES keys.

And I musn't be remiss in providing the code for the project. This time only the Makefile and cpp file are needed! Weeeee!

Public Key Encryption in Windows

One of the things I try to do on this blog is talk about problems I encounter and solutions I arrive at, with the slim hope that someone else may stumble upon it and say, "Oh, thank god, someone figured this out!

In the project I've been working on in my spare time (this mysterious, shadowy project that will not be revealed yet), I have a need to do some public/private key RSA encryption. The project is targeting Windows, so I needed to learn how CryptAPI on Windows works. The short answer is, "in a very complicated way." I have no idea about OpenSSL as an alternative. I didn't want to introduce a weird dependency at this time. Ultimately, I think those crypto-library-specific considerations could be abstracted, anyway.

So my seemingly simple task before me at this time is to take a certificate--having both the public and private key--and encrypt a blob of data using the private key and decrypt it with the public key (and vice versa).

The actual two functions used for encrypting and decrypting an arbitraty blob of data are CryptEncrypt and CryptDecrypt, respectively. These at least are pretty concise: they take the blob of data, the length, and a handle to the key to use to do the encrypting or decrypting (a HCRYPTKEY). It's this last part that's a pain to get.

In my code I need to start from a certificate. I'll assume that the certificate is already installed in the Windows certificate stores. Finding this certificate is not too difficult. I call CertOpenStore and indicate the "current user" store location (as opposed to the local machine store location) and the "my" (a.k.a. personal) store within it. That gives you a HCERTSTORE.

With that handle, you can call CertFindCertificateInStore and pass the subject name the certificate is issued to as part of the search. From this, you get a PCERT_CONTEXT, or rather, a PCCERT_CONTEXT (just a const one). This represents a single certificate and can be used to look up things related to it.

Now we need to get a hold of both keys from the certificate. The way to get keys from a certificate requires first acquiring a "context", or an instance of a cryptographic security provider, or something. The terminology is weird and abstract, and the docs don't define them well. Regardless, it is a value of type HCRYPTPROV that you're looking for.

One function that gets this is CryptAcquireContext, but you can only use if you know the "container name" where the key you need lives. I generated this certificate using makecert.exe. I don't even know what a container is in this context. Argh, I used the word "context" again! Well, it seems any cryptographic key in Windows lives in a "container", but I'm not sure why. But it does, and you'll have to work with it.

There's luckily a second function you can call that doesn't require knowledge of the key container name: CryptAcquireCertificatePrivateKey. Of course, this only gets you the private key. We'll have to meander down the path of cryptic functions to find the public one later.

As an aside, once you have the private key context, you can now retroactively find out the key container name by calling CryptGetProvParam with PP_CONTAINER. The answer is unexciting: just a GUID that is probably generated by makecert.exe.

This function can return one of two types of key--indicated in the pdwKeySpec out-variable--a signature key or an exchange key (AT_SIGNATURE or AT_EXCHANGE, respectively). I'm not really clear at this point on why this distinction exists, but in my usage, it is governed by the -sky command line flag I passed to makecert.exe when I created the certificate. For encrypting traffic to be sent to another machine, you want an exchange key. I guess a signature key is used for signing stuff? I don't really know.

You're almost there! Call CryptGetUserKey to get a coveted HCRYPTKEY that you can use for encrypting. This is, of course, the private key, since you got it from a private key context.

I need to point out something that confused me greatly at first. I ran my code to do just the encryption repeatedly on the same data. I expected to get the same output every time; after all, I'm using the same RSA key and blob of data. Instead, I got vastly different output every time: I couldn't find any common pattern of bytes within. I was really confused.

cleartext:
61 00 62 00 63 00 64 00

encrypted:
47 A8 4C AF 10 0B 24 42 DD D1 80 44 5C AB 37 E6 F9 0D 3D FA 43 60 CD 04 CD 22 01
6E 8C FB CF 51 AE 8A 06 C0 1D 39 62 EB D4 FB 9A 30 13 E3 AB CC 3F DA 26 C0 84 3F
97 9E 32 2B 2C 17 81 68 76 1F 82 0A 78 0D 8C D4 94 4D 10 F0 51 BC 6F D4 0B DF 0D
6C A1 72 AB 28 17 61 63 9C 67 29 C8 CC 9C 8A 54 67 24 96 10 5A 0C ED E5 91 12 3B
F4 82 88 5B 18 19 DE 63 07 D1 14 16 7E A7 B1 C5 E0 8D 36 A1


cleartext:
61 00 62 00 63 00 64 00

encrypted:
A1 BE 08 97 B0 C6 CC F6 81 00 93 82 AF 4E 38 55 06 61 DA B7 D2 13 6B 54 8E 2A 88
E0 21 84 D4 5D EC 56 71 6A E8 3A E8 42 62 11 4D B4 A0 90 17 86 C5 8E C6 46 9A 60
B4 B7 F0 1E 55 DD 47 A2 2A B8 44 B5 A2 B7 23 D8 2B E2 FF CF 0A 4D 79 A4 BD 41 57
B0 43 86 AA 44 B4 F9 D2 0E 34 19 AF 70 A7 6A 6B 39 C9 4D 2A 78 60 97 12 B1 D6 60
07 21 8F 8F 84 8E 43 01 2C A5 3A D0 04 AE 12 26 BE 61 C3 13

I thought that my code must be wrong. There must be some flag I need to set. Maybe it's automatically including some salt in the encryption. Yet another thing about cryptography I don't know about. Actually, I was on the right track there: the reason it looks completely different is that the cleartext is first augmented with PKCS#1 padding (section 8.1 of RFC 2313), which consists of random bytes of data plus a padding length field so it can be removed after decryption. Of course any change in the cleartext, including this, will yield a vastly different ciphertext.

Now that we've successfully encrypted with the private key, let's decrypt with the public key! But getting the HCRYPTKEY for the public key is weird and unintuitive. Maybe I missed some handy function that does it for you, but basically the only way I found is to call CryptImportPublicKeyInfo.

Let's do that thing where we try and match up inputs and outputs, the same way you do when plugging in your audio/video equipment when hooking up your TV. CryptImportPublicKeyInfo needs a HCRYPTPROV. What's something that gives an HCRYPTPROV? No, CryptAcquireCertificatePrivateKey won't work! We need the public key! How about just plain old CryptAcquireContext? There are flags you can pass to it to create a new key container. Maybe we'll create a temporary key container into which we'll import the public key info. It sounds hacky, I know, but it works.

if (!CryptAcquireContextW(
         &hPubProv,
         L"pub_key",
         MS_ENHANCED_PROV,
         PROV_RSA_FULL,
         0))
{
    if (GetLastError() == NTE_BAD_KEYSET)
    {
        if (!CryptAcquireContextW(
                 &hPubProv,
                 L"pub_key",
                 MS_ENHANCED_PROV,
                 PROV_RSA_FULL,
                 CRYPT_NEWKEYSET))
        {
            hr = HRESULT_FROM_WIN32(GetLastError());
            goto error;
        }
    }
    else
    {
        hr = HRESULT_FROM_WIN32(GetLastError());
        goto error;
    }
}

Still, we need a CERT_PUBLIC_KEY_INFO to supply to CryptImportPublicKeyInfo. Well, that function is called "import", so is there an "export", too? Yup, there's CryptExportPublicKeyInfoEx. This function also needs a HCRYPTPROV, presumably different than the "empty" one we're about to import into. If you read the docs, it says, "The CryptExportPublicKeyInfoEx function exports the public key information associated with the provider's corresponding private key." Hmm, I wonder if we can give the HCRYPTPROV from the private key. Spoilers: yes, we can.

But now we have everything we need to try to decrypt with this public key! Let's do it!

Umm, why is CryptDecrypt returning NTE_NO_KEY? "No key"? There's definitely a key! I just imported it! So here's the deal. For some reason on Windows you can't use this method to decrypt with a public key. I don't quite know why. I found some forum post saying I need an AT_SIGNATURE key to decrypt with the public key, but I wasn't able to get that to work either. I don't really know. Luckily, in my app, I don't need to, so I am leaving this unsolved.

Instead, if I encrypt with the public key, I'm readily able to decrypt with the private key, and that's good enough for me!

encrypt with public, decrypt with private:
cleartext:
61 00 62 00 63 00 64 00 

encrypting
found we need 128 bytes for ciphertext

encrypted:
3F 02 BF AD 87 96 26 E8 CB 3D E9 94 50 27 46 28 4F 16 E6 24 EE C5 6A A3 3C 7D 01
51 BB D4 6D 95 C8 14 08 42 B8 04 B6 4A D8 DB 63 8B 98 5E 33 56 0F 7B DD 91 C0 1A
C2 EF 1C 68 D6 D3 94 48 B3 60 B6 86 94 5A 90 B7 03 85 9A DE 45 BC 9C 21 4F 03 54
D1 48 1E 7A 09 61 AE AD 35 B2 33 FC 95 77 94 6C 6E C6 18 9F D7 39 EB 03 6C 22 FA
D6 CF 97 B5 9F A9 0D 63 4E C5 FD D7 A9 3D 98 FE 3A 4E F5 4D 

decrypted:
61 00 62 00 63 00 64 00 

cleartext and decrypted are the same


encrypt with private, decrypt with public:
cleartext:
61 00 62 00 63 00 64 00 

encrypting
found we need 128 bytes for ciphertext

encrypted:
E5 A6 D1 99 C2 E0 1A E3 F6 C2 D3 47 17 51 8A 22 8F 78 2C F3 C4 01 83 FD 1A 67 D5
EE 38 C9 8F 5B DF 6E C1 D9 ED 40 04 CE 31 15 2E 46 BD 77 9D 2A 14 59 8A 0B 58 07
BD EF 71 F3 A9 2A 0D F1 A2 E2 84 22 46 05 63 38 74 E8 61 C7 8F 25 C2 BC A0 5F 80
D2 07 02 7E 15 9A 64 66 55 89 14 23 68 57 D7 F8 6F 8B 86 EE 35 E5 96 A0 11 F4 3A
90 BF D2 39 22 02 F7 A5 5F 63 0A 8F 15 6A 19 25 FA 47 B4 22 

decrypting
error? 8009000D

I'd be a real jerk not to share the code at this point, since it's such a pain to figure all this stuff out. Here is a zip file containing the whole project:

• The source code, main.cpp
• The Makefile to build it with
• create_cert.cmd, which creates and installs the certificate hard-coded into the app
• makecert.exe, which create_cert.cmd uses

Adventures in crypto! Just the beginning for me, I fear. Yowza.