It came to my intention that this sample code – which shows how to plug-in a new symmetric algorithm (e.g. Twofish) via CNG (Crypto API: Next Generation) and CMS (Cryptographic Message Syntax) – has a usage error in the CMS portion.
Specifically, the BCryptSetProperty BCRYPT_CHAINING_MODE call is supposed to be made against the provider handle, and not the key handle. Also, the property data input length should correspond the length of the whole property string, and not just the size of the pointer. In other words, this code:
should actually be this:
sizeof(WCHAR) * (1 + wcslen(BCRYPT_CHAIN_MODE_CBC)), 0));
Now, to be clear, I haven’t tested the latter. Furthermore, I was told that the former, while technically incorrect, is allowed (at least in the Vista RTM version of CNG).