Transparent Database Encryption
Databases like Oracle, Microsoft SQL Server, Postgres, along with modern document-level databases and clustered big data platforms such as MongoDB, Cassandra, and others, all store vast amounts of data. As such, the likelihood of these data stores containing sensitive data is high. To meet the need to encrypt this sensitive data, many database vendors have introduced Transparent Database Encryption (TDE).
TDE is a mechanism for “transparently” encrypting data that is processed by the database, such that when the data is stored at rest, it remains encrypted. This way, it is secured from prying eyes of a system administrator. Likewise, if the database files were to be copied away or exfiltrated, they are encrypted and thus are unrecoverable.
With the general concept behind Transparent Database Encryption, let’s take a step back and break these concepts down.
Data and Databases
(this section is non-technical for those who need a primer on how databases store and retrieve data. If you’re familiar with databases, skip to the next section)
Often times, in networking diagrams and other software architecture, databases are depicted as a cylinder, like this:
But this is too simple, and while it’s an easy icon to use, doesn’t really show us how databases work. In order to understand TDE, we need to understand slightly more about how databases function.
Databases generally come in two flavors: relational and document databaes. First, how they are similar.
Imagine a file cabinet, with drawers filled with files, folders, and paper.
Those are our database files, and quite literally, all databases are fundamentally files on disk. Some databases are a terrifically huge collection of files, and other databases are one big file (like this drawer) with tons of data inside it (folders and paper), but regardless of its structure on disk, databaes — all databases — are fundamentally files on disk.
Now let’s say your boss asks you to go retrieve the file for client XYZ. You open the drawer with clients T – Z and start flipping through folders until you find the folder for client XYZ. You pull the folder, and hand it over.
Databases do exactly this same procedure when retrieving data. SAy you give the database a command, like:
SELECT client FROM orders WHERE order_date = “2018-12-09”;
Essentially, it’s searching through its files to retrieve all the orders that occurred on December 9, 2018.
What’s the difference between a relational and a document database? Besides the vast technical details, conceptually it’s very simple.
Relational databases create “relationships” between files and folders that enable administrators to shortcut through the data. For example, if you put a sticky-note on Client XYZ’s folder that said “Orders placed between December 1, 2018 and December 26, 2018” you just gave yourself a shortcut because you know that when you look in the orders files for Client XYZ, you would only need to look between those dates.
Relational databases setup these relationships between tables (i.e. folders) and within them, columns (files inside the folder).
Document databases are essentially the equivalent of having a Gmail inbox you never clean. You have thousands and thousands of emails in your inbox, no relationship whatsoever, no grouping, filing, or tagging. When you want to find something, you simply use Google’s all-powerful search to look through all of them and find what you want. Document databases essentially operate like this.
What about Encryption?
As you can see, databases can be thought of as a filing cabinet filled with data, some sensitive, that anybody who had access to your office would be able to look through. Not very secure.
Imagine for a moment that you had a particularly high-profile client and you certainly did not want anybody peeking into their files. You could just lock up their file, like so:
That way, anytime you needed anybody else’s data, you simply pull it from the cabinet. But if you needed the VIP client data, you’d have to go get a key and unlock it.
And, you may think, while we’re at it, why not just lock up the whole cabinet? Maybe not all the information is equally sensitive, but why not?
And this is exactly how Transparent Database Encryption for databases like Microsoft SQL Server and Oracle work. They enable you to “lock up”, or encrypt, various types of data. You can encrypt a column (folder), or you can encrypt the entire database (cabinet). Every time data is retrieved from either, the requestor must provide a key to unlock the data or they simply don’t get access.
The Technical Details
We’ve taken our analogies far enough.
Here’s how TDE generally works:
Let’s break it down:
- A key is created that will encrypt the database data called the Data Encryption Key (DEK). This key, however, must be protected by something. So it’s wrapped with another key.
- The wrapper key is the Master Encryption Key (MEK), and is stored somewhere – the database itself (not safe), or a key vault, in an HSM, or in a cloud-based KMS like Azure.
- When a DBA runs a query on encrypted data, they must first authenticate to the master key in order for the DEK to be unlocked. Depending on the configuration, this authentication can happen via certificate or password. Once authenticated, the master key unwraps the DEK which is then available for use to encrypt / decrypt data.
- This encrypt / decrypt by the DEK is completely transparent to the DBA. Once it is activated, all encryption is handled by the database engine.
That is why it’s called Transparent Database Encryption.
The advantages of TDE are primarily around convenience and ease of use.
- TDE is relatively easy to setup and configure. With less than 10 SQL statements, entire databases can be encrypted.
- This is important because remember, Database Administrators (DBA’s) aren’t usually security experts. Database vendors know this and strive to make it as natural as possible for a DBA to implement this level of security.
- Because databases run autonomously, there are even mechanisms to unlock the DEK automatically when the database is started.
- TDE provides a quick and easy way to fulfill regulatory requirements to encrypt certain types of regulated data: PII, credit card information, etc.
In summary, TDE gives DBA’s the ability to implement their own security using products they are trained in without having to rely on third-party security products that cost additional budget and require training and maintenance.
The disadvantages of TDE are primarily around its limited capabilities and “convenience” to the DBA.
- Only data at rest is encrypted, which means data red by the database (i.e. presented to the DBA or application) does not remain encrypted.
- TDE can have varying performance impact (encryption operations are not “free”, they require compute and processing cycles, however small).
- All data is encrypted, not just sensitive data. This means that performance hit is taken even when reading or writing non-sensitive data.
- Extra complexity for the DBA when restoring databases, particularly on new servers that aren’t configured with the keys.
- Key management burden, and without an external KMS, questionable key security.
- Requires a more expensive edition of the database (for Oracle and SQL Server).
- No separation of duties: DBA’s responsible for their own security of the database.
- Security teams have no good way of auditing how well TDE is implemented, if it’s implemented according to security standards, or whether it’s even turned on. Auditing which databases are encrypted is a manual effort across each and every one.
- TDE is a point solution. While it may encrypt some databases, others still need another approach to encryption. It won’t work on anything other than the databases it’s bundled with. Security policy that is broadly requiring encryption would look better at a standardized, consistent approach across all enterprise assets.
Transaprent Database Encryption is a useful tool for teams that need an “easy button” to implement encryption within their databases. Is it the most powerful solution to encryption? Of course not. It is a base level of encryption, with moderate security capabilities, and should be seen as a “good enough” solution for teams facing an encryption of data-at-rest checkbox on their next audit. Likewise, TDE should be seen as a point solution – a solution that does one thing: encrypt some databases. It rarely fits well in a larger enterprise encryption strategy, offers no centralization, auditing, or even verification that it’s being used effectively.