Mdthbs

I have implemented a simple encryption/decryption program based on AES-256 in CBC mode¹.

Actually, it is more precise to describe it as a compression+encryption / decryption+decompression program.

If one provides the wrong key to the decryption+decompression function, the decompression stage will explicitly fail, as expected, since the decrypted content (being based on the wrong key) will be just noise.

I would like modify the decryption phase of the scheme so that it detects on its own that the wrong key has been used, before it proceeds to the decompression phase. I am looking for a scheme to support this functionality that does not weaken the overall cryptographic strength of the framework.

A naive approach would be to generate the encrypted content as²

AES256_CBC(key, iv, SENTINEL_STRING + plaintext)

where SENTINEL_STRING is a string that the decryption phase can know in advance. While I am being naive about it, I could make the SENTINEL_STRING be equal to the key, for example.

I imagine there are pretty standard ways to solve this problem. In fact, for all I know, the design of AES-256-CBC already has built into it a way to check that decryption succeeded.

I hope someone can enlighten me on these matters.

Also, if this problem is common enough to have a generally accepted name (suitable as search engine fodder) please let me know.

^{1 For what it's worth, my current implementation of this uses Python's pycrypto module, but an earlier implementation used Perl's Crypto::CBC package. Both versions can reciprocally decrypt+decompress files compressed+encrypted by the other. I bother mentioning all this to stress the fact that this question is primarily about AES-256-CBC in general, and not about any specific implementation of it.}

^{2 I hope my ad-hoc notation here is not too stupid. I mean it as a shorthand for "encrypt the string SENTINEL_STRING + plaintext with AES-256 in CBC mode, using the key key and initialization vector iv".}

You should use an authenticated encryption mode. There are several reasons for that, but one (relatively minor) one is that it automatically gives you the ability to detect incorrect keys, since the authentication will fail.

If you insist on using a traditional non-authenticated encryption mode, or if you'd like to have some way of distinguishing "incorrect key" from "corrupted ciphertext", you can include a key check value together with the ciphertext, and verify it before attempting decryption. There are several possible ways to implement one.

A traditional method is to encrypt an all-zero block using the raw block cipher (i.e. "ECB mode") and use the resulting ciphertext as the key check value, but note that this makes it possible for an attacker to tell whether two messages have been encrypted using the same key by comparing the key check values. Alternatively, assuming that you're using an authenticated encryption mode anyway, you could just generate the key check value by encrypting an empty message using the same mode. Assuming that you're using a unique nonce / IV for each encryption, as you should be, this should eliminate the information leak by making every key check value also unique.

(BTW, note that compressing data before encryption can leak (extra) information about the plaintext. Basically, this happens because all general-purpose encryption schemes necessarily leak information about the length of the plaintext, and compression makes the plaintext length depend on its content. Padding reduces this leak slightly, but does not eliminate it. This has been used in actual attacks.)

Ilmari Karonen already mentioned using an authenticated encryption mode, which would solve the concerns, but should you not go that way, please note the flaw in your premise:

If one provides the wrong key to the decryption+decompression function, the decompression stage will explicitly fail, as expected, since the decrypted content (being based on the wrong key) will be just noise.

Pretty much any wrong key or change in the ciphertext will decrypt to something which is not valid for the decompression algorithm you used, and that it will detect (and hopefully your program will detect that error). But that isn't guaranteed.

A trivial solution would be to include an HMAC/hash of the plaintext/compressed content, which would allow you to validate that you decrypted the right content.

This is even more important taking into account that your use case is long-term storage. I wouldn't be surprised if it was possible to truncate one such file at the right boundary that it was unnoticed not only by the AES-CBC but also by the underlying compression function. Just adding a key-check value won't detect that issue.

Since you are aiming for storage, it probably won't matter for you, but note that if this was used in eg. an online protocol, detecting decompression errors would provide an oracle to an attacker that sent your decrypter multiple corrupted blocks, based on the difference of time it needed for processing it.

There is no way to detect the wrong key before decrypting: I can create a message M, a totally valid and legitimate key K, and then I encrypt the message with a different key K': Any information that I give you about the key K will be totally legitimate and fine, it will just not be about the key I used to encrypt the message.

You can try to include additional information, but whatever you ask for, I can supply you with information that would be correct with the key K, and then use the key K'.

I can even start encrypting using K, encrypt 90% of the message that way, and use K' for the remaining 10%. So there is no way to detect an incorrect key other than decrypting the complete message, and checking that the decrypted message is valid.