Caveats in ASP.NET Core Data Protection

ASP.NET Core provides a built-in Data Protection mechanism to let us encrypt or decrypt sensitive data. There are some limitations that can cause problems while bringing convenience. I met some problems these days.

My Scenario

My blog system has a feature to send email notifications, so you need to configure an email account to let the program use that account to send mail to an administrator or users. This involves the question of how to securely store your account password. As programmers with morals, we certainly can't store plaintext passwords to databases like many platforms in China. In this scenario, we also can't store passwords with HASH, because sending messages is done by the system in the background, and the user is not required to enter a password to compare the HASH with the database stored values. Therefore, my first thought is to use the symmetric encryption algorithm such as AES, store the encrypted ciphertext in the database, the program decrypts the values according to key, and then use the account to send mail.

Don't Reinvent the Wheel

Before designing a feature, I usually check the internet first to see if there is already a framework feature that comes with the functionality to complete the requirements. As a result, ASP.NET Core's own Data Protection APIs is brought to my attention.

1. The benefits of using Data Protection APIs are:
2. Replacement for the classic MachineKey
3. No need to design an encryption algorithm on your own, just use the most secured framework provided algorithms.
4. Without having to manage the key yourself, the framework will automatically store and manage the key for you.
5. The key is refreshed every 90 days by default
6. API is easy to use without requiring to know deep about encryptions
7. Retain flexibility and extensibility, allowing steps such as custom algorithms, key storage, and more

You can check the detailed introduction on Microsoft Docs here.

The algorithm used by Data Protection is by default AES, which can meet my needs.

Encryption and Decryption Flow

After the framework helps us hide the complex algorithm process, we can complete the encryption and decryption with just 3 simple API calls:

A common practice is: Add DataProtection in Startup

public void ConfigureServices(IServiceCollection services)
{
    services.AddDataProtection();
    // ...
}

Create a service like this to use in other parts of the system

public class EncryptionService
{
    private readonly IDataProtectionProvider _dataProtectionProvider;
    private const string Key = "cxz92k13md8f981hu6y7alkc";

    public EncryptionService(IDataProtectionProvider dataProtectionProvider)
    {
        _dataProtectionProvider = dataProtectionProvider;
    }

    public string Encrypt(string input)
    {
        var protector = _dataProtectionProvider.CreateProtector(Key);
        return protector.Protect(input);
    }

    public string Decrypt(string cipherText)
    {
        var protector = _dataProtectionProvider.CreateProtector(Key);
        return protector.Unprotect(cipherText);
    }
}

I used this method to encrypt the email account password and store it in the database. Then change the code to successfully decrypt the data, run on my own machine and complete the function of sending mail without a problem. So I deployed it to production.

Production Blow Up!

An exception occurred in the production environment while decrypted the ciphertext in the database:

System.Security.Cryptography.CryptographicException: The key {bd424a84-5faa-4b97-8cd9-6bea01f052cd} was not found in the key ring.

After research, this is because ASP.NET Core generates different keys to encrypt data when it is running on different machines, and the ciphertext in my database is encrypted with the key of the developer machine, not the key of the server. So when you try to decrypt, you can't find the key for encryption, and this exception blows up the server sky high.

ASP.NET Core can store Key to Windows Registry, IIS User Profile, Azure KeyVault, Azure Storage Account or File System, etc. In Azure App Service, the Key is stored in %HOME%\ASP.NET\DataProtection-Keys folder, it will automatically sync into other instances in your App Service. You can see the folder with Kudu:

Therefore, to solve the problem of key inconsistency in different environments, only need to find a consistent storage location. But that's not going to solve my problem. Because by default, a new key is regenerated every 90 days, so that the ciphertext in the database will fail to decrypt again if it is not updated.

In addition, the AntiForgeryToken in ASP.NET Core is also using the same Data Protection APIs, which in case when you deploy multiple web servers on your own (not using App Service to scale out), it will result in different keys in every server, and the user will fail on submitting form data.

The Solution

Although we can do this by saving the key in a unified location, we can also specify an automatic refresh cycle, but I do not recommend it. Because this mechanism applies only to encrypted, short-aging data, it is not designed for data that is persisted into the database. So in this scenario, we still have to write our own service of encryption and decryption.

First (very faceless) copy a pair of AES encryption and decryption functions from Microsoft's official documents:

Encryption

private static byte[] EncryptStringToBytes_Aes(string plainText, byte[] key, byte[] iv)
{
    if (plainText == null || plainText.Length <= 0)
        throw new ArgumentNullException(nameof(plainText));
    if (key == null || key.Length <= 0)
        throw new ArgumentNullException(nameof(key));
    if (iv == null || iv.Length <= 0)
        throw new ArgumentNullException(nameof(iv));
    byte[] encrypted;

    using (var aesAlg = Aes.Create())
    {
        aesAlg.Key = key;
        aesAlg.IV = iv;

        var encryptor = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV);
        using (var msEncrypt = new MemoryStream())
        {
            using (var csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
            {
                using (var swEncrypt = new StreamWriter(csEncrypt))
                {
                    swEncrypt.Write(plainText);
                }
                encrypted = msEncrypt.ToArray();
            }
        }
    }

    return encrypted;
}

Decryption

private static string DecryptStringFromBytes_Aes(byte[] cipherText, byte[] key, byte[] iv)
{
    if (cipherText == null || cipherText.Length <= 0)
        throw new ArgumentNullException(nameof(cipherText));
    if (key == null || key.Length <= 0)
        throw new ArgumentNullException(nameof(key));
    if (iv == null || iv.Length <= 0)
        throw new ArgumentNullException(nameof(iv));

    string plaintext;
    using (var aesAlg = Aes.Create())
    {
        aesAlg.Key = key;
        aesAlg.IV = iv;

        var decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);
        using (var msDecrypt = new MemoryStream(cipherText))
        {
            using (var csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
            {
                using (var srDecrypt = new StreamReader(csDecrypt))
                {
                    plaintext = srDecrypt.ReadToEnd();
                }
            }
        }

    }

    return plaintext;
}

Create an EncryptionService

public class EncryptionService
{
    private readonly KeyInfo _keyInfo;

    public EncryptionService(KeyInfo keyInfo = null)
    {
        _keyInfo = keyInfo;
    }

    public string Encrypt(string input)
    {
        var enc = EncryptStringToBytes_Aes(input, _keyInfo.Key, _keyInfo.Iv);
        return Convert.ToBase64String(enc);
    }

    public string Decrypt(string cipherText)
    {
        var cipherBytes = Convert.FromBase64String(cipherText);
        return DecryptStringFromBytes_Aes(cipherBytes, _keyInfo.Key, _keyInfo.Iv);
    }
	
    // EncryptStringToBytes_Aes and DecryptStringFromBytes_Aes above ...
}

KeyInfo is designed as a separate class to flexibly allow the user to choose to assign byte[] array or a string type Key as well as an initial vector (IV)

public class KeyInfo
{
    public byte[] Key { get; }
    public byte[] Iv { get; }

    public string KeyString => Convert.ToBase64String(Key);
    public string IVString => Convert.ToBase64String(Iv);

    public KeyInfo()
    {
        using (var myAes = Aes.Create())
        {
            Key = myAes.Key;
            Iv = myAes.IV;
        }
    }

    public KeyInfo(string key, string iv)
    {
        Key = Convert.FromBase64String(key);
        Iv = Convert.FromBase64String(iv);
    }

    public KeyInfo(byte[] key, byte[] iv)
    {
        Key = key;
        Iv = iv;
    }
}

Register into DI container

services.AddTransient(ec => new EncryptionService(new KeyInfo("45BLO2yoJkvBwz99kBEMlNkxvL40vUSGaqr/WBu3+Vg=", "Ou3fn+I9SVicGWMLkFEgZQ==")));

The key and IV can be obtained by KeyInfo's parameterless constructor. After saving these values, you can always use this pair of key, to ensure that the subsequent encryption and decryption data are consistent.

Usage

private readonly EncryptionService _encryptionService;

public HomeController(EncryptionService encryptionService)
{
    _encryptionService = encryptionService;
}

public IActionResult Index()
{
    var str = "Hello";
    var enc = _encryptionService.Encrypt(str);
    var dec = _encryptionService.Decrypt(enc);
    return Content($"str: {str}, enc: {enc}, dec: {dec}");
}

Conclusion

ASP.NET Core comes with a Data Encryption API that is secure, easy to use, and flexible. However, that key storage and timed refresh apply only to short-aging encryption. For long-time saved static ciphertext, you can implement an encryption and decryption service on our own.

The complete sample code is on my GitHub: https://github.com/EdiWang/DotNet-Samples/tree/master/AspNet-AES-Non-DPAPI