In my last raft post, we discussed the raft paper and some intricacies of the algorithm. Next, I wanted to post my implementation in raft. But before really diving into implementation details, let's see a practical use-case of raft. In this post, we'll implement a distributed key-value store using the etcd's raft library. So we do not have to worry about the raft implementation yet, and we'll see what services we can build over raft.
We'll implement the kv store in golang all in a single file.
Table of contents
Need for replication
Alright, before we start implementing our kv, let's understand the need for replication. In various critical production environments, a very important requirement is to have the database engine be highly available i.e. in case of intermittent failures of nodes or network the database service stays available. To solve for that, we need a mechanism to replicate the state of the database to other nodes, so that, if a node fails, there are some other nodes where the client requests can be routed. Often the requirements are that the state between the nodes should be strongly consistent. Raft helps us provide such guarantees and can tolerate (n-1)/2 failures.
Key-Value Store Implementation
We'll be using a raft library from etcd to create raft nodes which will help handle the replication and leader election in case of failures.
Implementing the State Machine
Let's add structs and methods for the key value data structures and operations.