I've been meaning to play with BoltDB for a very long time, recently I finally had some time to do so. Why? I often made a mistake of setting up a large database when starting some private project only to see most of my effort going there instead of the app itself. For starters, the simplest solutions are the best, and Bolt is an example of such. Let's check out how can it be used in a small project.

What is BoltDB

BoltDB is a file-based, easy-to-use key-value database written in Go. It's very fast for reading data, but its main downside is that reading operation is blocking the file. If you can live with that (or you're okay with opening the connection for your operations) than you should definitely check out Bolt for your next side project.

The database is structured into named buckets, which allow you to group your data the way you like. If you are familiar with Redis, you'll quickly catch up with its API, since you can use find keys with Seek(..) function, you can also safely increment values since everything is run in transactions.

Without further ado, let's get into some real business.

My small application will have to do four things. We will have a piece of clothing that has some ID, description and a counter of how many times it was worn. I would like to do the following things:

  • add a piece of clothing (save item),
  • list all pieces with their wear-number (read items),
  • increment a wear-number (update item),
  • clear wear-number for a piece (update item).

Saving data

In order to save data to Bolt database file, you need to open the connection:

db, err := bolt.Open("diwia.db", 0600, nil)
...
defer db.Close()

Then you need to run Update function which takes a transaction as an argument and returns an error. It is executed by Bolt itself, so you don't have to worry about passing the tx. Since everything is kept in buckets, you need to access it (for inserts it's best to create if it doesn't exist) and Put a new key-value pair there:

return db.Update(func(tx *bolt.Tx) error {
    bk, err := tx.CreateBucketIfNotExists(bucketClothes)
    if err != nil {
        return fmt.Errorf("failed to create bucket: %v", err)
    }

    if err := bk.Put([]byte(id), []byte(desc)); err != nil {
        return fmt.Errorf("failed to insert '%s': %v", desc, err)
    }
    return nil
})

Retrieving data

Reading data looks very similar, but instead of Update you should use View function, and for that you can open the database in a read-only mode:

db, err := bolt.Open("diwia.db", 0600, &bolt.Options{ReadOnly: true})
...
defer db.Close()

Apart from that, the flow is very similar. Instead of putting we Get values from the bucket:

var count int
err := db.View(func(tx *bolt.Tx) error {
    bk := tx.Bucket([]byte(bucketWears))
    if bk == nil {
        return errors.Wrapf(ErrNoBucket, "failed to get 'wears' bucket")
    }

    bs := bk.Get([]byte(id))
    if bs == nil {
        return errors.Wrapf(ErrNotFound, "failed to find wears for 's'", id)
    }

    var err error
    count, err = strconv.Atoi(string(bs))
    if err != nil {
        return errors.Wrapf(ErrWearNaN, "invalid wear value for '%s'", id)
    }

    return nil
})
return count, err

For reading multiple key-value pairs you should use Cursor() and iterate over it using First() and Next() functions:

bk := tx.Bucket([]byte(bucketClothes))
if bk == nil {
    return errors.Wrapf(ErrNoBucket, "failed to get 'clothes' bucket")
}

c := bk.Cursor()
for k, v := c.First(); k != nil; k, v = c.Next() {
    pieces = append(pieces, Piece{
        ID:          string(k),
        Description: string(v),
    })
}

Batching

Since writing operations are blocking, you may find making multiple data modifications a bit annoying since you should repeatedly open and close connections. To solve that, there is a Batch(..) function which groups operations in a single transaction and executes them together:

...
var wg sync.WaitGroup
for _, id := range ids {
    wg.Add(1)
    go func(itemID string) {
        errs <- db.Batch(func(tx *bolt.Tx) error {
            return tx.Bucket(bucketWears).Delete([]byte(itemID))
        })
        wg.Done()
    }(id)
}

wg.Wait()
...

Working example

I've created a small application to show the basic usage of Bolt and help me with doing the laundry. The app is called diwia (Dit I Wear It Already?) and its full source code is available on Github.