Badger_ Fast Key-Value DB in Go

Badger is an embedded key-value database, written in Go.Licensed under Apache 2.0.

1.Badger: Fast Key-Value DB in Go Manish R Jain, Dgraph Labs Apr 14, 2018 Gopher China, Shanghai

2.Dgraph Labs Fast, Distributed graph database. Sparse data sets. Lots of relationships.

3.What is Badger? Badger is an embedded key-value database, written in Go. Licensed under Apache 2.0. go get

4.Current Status Closing v2.0. Close to 3500 Github starts. 42 contributors. Used by Dgraph, Go-IPFS, 0-stor, Sandglass.

5.Serving 300TB (and growing) at Usenet Express

6.Basic Operations

7.Set a key-value func set() error { fmt.Println("\nRunning SET") return db.Update(func(txn *badger.Txn) error { if err := txn.Set([]byte("foo"), []byte("bar")); err != nil { return err } fmt.Println("Set foo to bar") return nil }) }

8.Get a key-value func get() error { fmt.Println("\nRunning GET") return db.View(func(txn *badger.Txn) error { item, err := txn.Get([]byte("foo")) // handle err if err != nil { return err } val, err := item.Value() // handle err if err != nil { return err } fmt.Printf("The value is: %s\n", val) return nil }) }

9.Iterate key-values func iterate() error { fmt.Println("\nRunning ITERATE") return db.View(func(txn *badger.Txn) error { opts := badger.DefaultIteratorOptions it := txn.NewIterator(opts) defer it.Close() for it.Rewind(); it.Valid(); it.Next() { k := it.Item().Key() v, err := it.Item().Value() // handle err if err != nil { return err } fmt.Printf("key=%s, value=%s\n", k, v) } return nil }) }

10.Run the code func main() { opt := badger.DefaultOptions opt.Dir = "/tmp/db" opt.ValueDir = opt.Dir var err error db, err = badger.Open(opt) if err != nil { panic(err) } defer db.Close() fmt.Println("DB opened") set() get() iterate() fmt.Println("DB done") } Run

11.Badger != replacement for Go map

12.Motivation and Outcome

13.Cgo is not Go Some people, when confronted with a problem, think “I know, I’ll use cgo.” Now they have two problems. ->Cgo is not Go, Dave Cheney

14.RocksDB Great write throughput. Okay read throughput. Cons: Required Cgo.

15.BoltDB Pure Go. Great read throughput. Cons: Bad write throughput.

16.Why build it? Go native key-value DB for Dgraph. No compromise in read-write performance. Avoid Cgo.

17.What did we spend? Spent a few months. Built with <1 full-time gopher. Aka, the power of Go!

18.What did we gain? A faster key-value DB for Go. Ability to run Go pro lers all the way down to disk. Clean Go code (no C).

19.Launch Reception Within 12 hours of blog post release First page of HN for a day. 355 points, 96 comments. 1250 Github stars in 4 days.


21.Recruiters loved it! Got 3 di erent emails from 3 di erent recruiters...

22.Recruiters loved it! For jobs in the same company.


24.Two common Trees LSM trees B+ trees

25.LSM Trees More levels High write throughput High read latency Example: RocksDB

26.B+ Trees Fewer levels Low write throughput Low read latency Example: BoltDB

27.Badger is based on LSM trees.

28.LSM Trees

29.Writes in LSM trees: Memtable to L0