We all have something to hide, I think everyone has to admit that. Being a developer you need to store lots of passwords, tokens, addresses and other pieces of information that you should not reveal to anyone. You could keep them all in a text file, but a part of you knows that you really should not. What are the alternatives? Let's look at one of them, Vault, which is an awesome project by Hashicorp.

What is Vault?

Like with many tools, it's easier to say what it does than what it is exactly. If I had to, I'd say that it is a storage for your secret data, which has two levels of security one must pass in order to get the information. While the way it works is always the same, it can be backed up with different means of actual storage: it can keep data in memory, in a file, and more. Except for those two levels, which I will describe in a moment, a special characteristic of Vault is that it acts as a server providing an HTTP API to the data it guards. It can, obviously, be configured to restrict access to some data, make some keys read-only for everyone, or for a particular group of users. The configuration is done using HCL files (HashiCorp Configuration Language, more on them here).

Since Vault is a storage for secrets, its main use case is pretty straightforward. You can use it as a place you keep all private information important for projects you are working on (and everything else you want to keep private, really), but it can also be used to share that knowledge across members of your team. It allows you to give and revoke access to the users identified by personal tokens, making sure that only particular pieces of your know-how are visible to authorized personnel only.

Recently I've been playing around with Vault as a local secret storage (for MySQL credentials, tokens to various tools, etc.) and I found that it made my work much faster when I need to log in somewhere. Probably I saved the most time because I now access information from within my console so that I don't force myself to switch windows and click it out, but it was possible thanks to Vault after all. I might have found some other, ready tools to do that, but nothing beats a fun side project with helps you on the daily basis, right?

A bit of theory

Before we start, we must understand what are those two levels of security that Vault provides. The first defines if you are able to event attempt any read or write operation on the storage, by defining if Vault is sealed or unsealed. It all starts when you initialize your instance and define two properties important for unsealing:

  • key shares indicates how many keys should Vault generate, that may later be used to change its state,
  • key treshold indicates how many of those generated keys are required to unseal your Vault.

If you want to access sealed Vault, you will get an error regardless of the permissions you possess (even root cannot do anything!):

Vault - sealed access

In order to get access to the Vault, a given (key treshold) number of keys must be used with unseal command first, so that the first lock is open and the storage is accessible:

Vault - unsealing

Note that after the first unseal, the Vault is still sealed. You cannot unseal it with the same key twice, which is reasonable - in the real life, you want at least two people to decide that the storage is safe and can be accessed by the users. Now you can actually read/write data:

Vault - unsealed access

What if you discover (or suspect) that your Vault has been attacked, you should immediately seal it. That immediately is crucial here, as unlike unsealing, you don't need any keys to seal the Vault. Again, sounds reasonable since you need to do that as soon as you realize something is wrong, right?

Vault - sealing

Getting started

How does this look using the command line? We might get started by using Vault in development mode but it is very limited. While we might be fine with playing around with storage kept in memory, the dev creates Vault which is already initialized and doesn't require keys to seal or unseal (you do that without passing the key at all). That's why we'll create a simple configuration file and pass it to the vault server command.

First, let's create a configuration file called config.hcl with the following content:

# /path/to/config.hcl
storage "inmem" {}

listener "tcp" {
    address = "127.0.0.1:8888"
    tls_disable = 1
}

Later on you might want to understand the HCL more, but for now, it would be enough to know that you need to define storage type, HTTP API address and disable TLS (we're not using HTTPS for now) to get it started. Then we can start the server by passing the configuration file with -config argument:

$ vault server -config /tmp/config.hcl 
...
==> Vault server configuration:
...
            Listener 1: tcp (addr: "127.0.0.1:8888", cluster address: "127.0.0.1:8889", tls: "disabled")
...
==> Vault server started! Log data will stream in below:

Now that we have Vault instance working, we need to initialize it:

$ VAULT_ADDR='http://127.0.0.1:8888' vault operator init -key-shares=3 -key-threshold=2
Unseal Key 1: 74KweSo0P8fbqbUiZgfMc6qKSz/fk59S173IUCY+vel8
Unseal Key 2: X1xFdOcOptU/gOWHd9QxgPqGoCcelkVIAf59sSs5qNho
Unseal Key 3: L1gEXFgcfxgpKUBjIPjt7l/Ycpv/I4IYmOC4EgNgPM2K

Initial Root Token: 253372b7-41ce-8a65-2eb6-d8ef16408b32
...

Now we can try to write something to the store since we got the root (access) token, but we already know what's going to happen:

$ VAULT_ADDR='http://127.0.0.1:8888' vault login       
Token (will be hidden): 
Error authenticating: Error looking up token: Error making API request.

URL: GET http://127.0.0.1:8888/v1/auth/token/lookup-self
Code: 503. Errors:

* Vault is sealed

Yep, we cannot do anything because the Vault is sealed. What if we unseal it with one of the tokens?

$ VAULT_ADDR='http://127.0.0.1:8888' vault operator unseal 74KweSo0P8fbqbUiZgfMc6qKSz/fk59S173IUCY+vel8
Key                Value
---                -----
...
Sealed             true
Total Shares       3
Threshold          2
Unseal Progress    1/2
...

You can see that after calling operator unseal with one of the keys, we are halfway through - we still need another one to open the vault:

$ VAULT_ADDR='http://127.0.0.1:8888' vault login       
Token (will be hidden): 
...
Code: 503. Errors:

* Vault is sealed

Still no success. Let's try with the second seal key:

$ VAULT_ADDR='http://127.0.0.1:8888' vault operator unseal
Unseal Key (will be hidden): 
Key             Value
---             -----
...
Sealed          false
...

Now, what if we want to login?

$ VAULT_ADDR='http://127.0.0.1:8888' vault login                                     
Token (will be hidden): 
Success! You are now authenticated. ...
...

Yeah! We're in! Now we can write data to the storage...

$ VAULT_ADDR='http://127.0.0.1:8888' vault write secret/foo value=bar
Success! Data written to: secret/foo

...and reading from it:

$ VAULT_ADDR='http://127.0.0.1:8888' vault read secret/foo           
Key                 Value
---                 -----
refresh_interval    768h
value               bar

At any point, the Vault can sealed ...

$ VAULT_ADDR='http://127.0.0.1:8888' vault operator seal       
Success! Vault is sealed.

...and we cannot do anything

$ VAULT_ADDR='http://127.0.0.1:8888' vault read secret/foo
Error reading secret/foo: Error making API request.

URL: GET http://127.0.0.1:8888/v1/secret/foo
Code: 503. Errors:

* Vault is sealed

Summary

This was just a basic introduction to the Vault and what it is capable of. Next posts will describe some more advanced topics including managing access, using Vault via Go client, etc. Stay tuned!