Here is the app if you just want to shorten some links SRTL. It was the shortest domain I could grab it stands for Short Link or at least thats what I’m telling myself.

I recently saw a post on Medium about RocksDB that sparked my curiosity. It’s a high performance key value store using a LSM tree. The DB has very fast read and write performance designed for NVME flash storage. I created this simple project just to test out linxGnu’s grocksdb implementation of a go wrapper, as RocksDB is developed in C++.

Contents

Building using Docker

Initially I had some issues with glibc and other libraries conflicting with what I had on my local machine vs what was installed in my cloud node. To simplify things I used Docker to build the libraries and binaries so that they would be consistent with that environment.

Everything is on my github if you want to check out the full source code. The install script was yoinked from this gist.

RocksDB can take a while to compile so order is important in your Dockerfile, you don’t want it to have to recompile with each change of your code.

FROM ubuntu:20.04

COPY install.sh /app/install.sh

RUN apt update && apt install -y \
 build-essential \
 git \
 curl \
 wget \
 gcc

RUN /app/install.sh

RUN wget -P \
    /tmp \
    https://dl.google.com/go/go1.19.linux-amd64.tar.gz

RUN tar -C /usr/local -xzf \
    /tmp/go1.19.linux-amd64.tar.gz
RUN rm /tmp/go1.19.linux-amd64.tar.gz

RUN apt install -y \
    libgflags-dev \
    libsnappy-dev \
    zlib1g-dev \
    libbz2-dev \
    liblz4-dev \
    libzstd-dev

ENV GOPATH /go
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH

COPY . /app

WORKDIR "/app" 

RUN go mod download

RUN go build -tags builtin_static \
    -o srtl ./cmd/http

Testing It Out

RocksDB has sensible defaults if you’re just testing it out. It even has functionality for a TTL, although this is more of a minimum TTL, items only get dropped during the compaction stage so you can’t depend on it as a timer. But in the case of a link shortener with short lived urls, the url functioning until the next compaction is not an issue.

bbto := grocksdb.NewDefaultBlockBasedTableOptions()
bbto.SetBlockCache(grocksdb.NewLRUCache(3 << 30))

opts := grocksdb.NewDefaultOptions()
opts.SetBlockBasedTableFactory(bbto)
opts.SetCreateIfMissing(true)

db, err := grocksdb.OpenDbWithTTL(opts, "./db", 86400)

The benefits of using a key value store is once initialized the DB is incredibly easy to use. With simple Put and Get functions. Keys and values are pure byte streams, theres no limit to their size but in my case the keys are only 5ish characters long.

err = db.Put(grocksdb.NewDefaultWriteOptions(), key, value)
val, err = db.Get(grocksdb.NewDefaultReadOptions(), key)

Creating a systemd Service

Now that I had a working binary I wanted to use systemd to keep it running. I used the following config: /etc/systemd/system/srtl.service.

[Unit]
Description=SRTL Go Service
ConditionPathExists=/Path/To/Binary/srtl
After=network.target

[Service]
Type=simple
User=james
Group=james

WorkingDirectory=/Path/To/Binary/
ExecStart=/Path/To/Binary/srtl
Restart=on-failure
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=srtlgoservice

[Install]
WantedBy=multi-user.target

Then I ran the below commands to register the service.

sudo systemctl daemon-reload
sudo service srtl start
sudo service srtl status
sudo systemctl enable srtl
sudo systemctl start srtl

In my case I had some environment variables that I wanted to be set, in that case you need to create a file in /etc/systemd/system/srtl.service.d/myenv.conf with the following variables set.

[Service]
Environment=HOST=srtl.ie
Environment=PORT=8080
Environment=LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib

Rocksdb vs Pebble?

During my research about RocksDb I found a post by CockroachDB who are moving away from RocksDb and replacing it with pebble which was inspired by RocksDb/LevelDb written in go. It has a subset of features but at least from my testing has improved read and write speeds. If you’re comfortable with the different feature set its worth having a look. It is also far easier to manage as you don’t have to deal with compiling libraries.

            RocksDb          Pebble
Set_Key:   4.18µs ± 5%    0.93µs ± 6%  -77.77%  (p=0.008 n=5+5)
Get_Key:   1.86µs ± 4%    1.59µs ± 7%  -14.82%  (p=0.008 n=5+5)