Hi. It’s been a while. This post is covering go-redis, which is a popular Redis client for Go. There’s other Redis clients as well as other key-value store backends that you could opt for, but this post is about go-redis. Importantly, this post is not a primer on Redis itself (maybe I’ll have one of those in the future, but this ain’t it). I have a couple use cases in mind that will be the focus of this post:

  • Key-value store. Sometimes you want a key value store for your application. This is typically the case when your application runs as a horizontally scalable collection of instances, so an in-memory store won’t do. Redis provides a distributed (i.e., horizontally scalable) key-value store. Also your data is persistent if you service happens to crash/restart.
  • Pub-Sub. Redis Streams make it very easy to set up a basic pub-sub service. This opens up a whole world of functionality for your applications.

There’s a bunch of other features go-redis has to offer, but for the sake of brevity I’m going to showcase these two.

Ok, what about documentation? There’s a GitHub page and an official website. I’ve found these to be sufficient to get up and running. You’ll also want to be familiar with working with Redis itself; the Redis documentation is quite good so be sure to give it a read. In general, if you know the Redis command you’re interested in, it’s quite easy to map that to the relevant go-redis command.

Setup and Connection⌗

First you need to get your Redis service up and running and then connect to it. You can do this in a variety of ways. For demo purposes, let’s just assume you’re running a Redis container on your local machine (on a user defined bridge network for service discovery):

docker network create demo-net
docker run --rm --network demo-net -p 6379:6379 --name redis redis:7

Lets assume you’ve built and tagged your Go application image as app. You can run that container with:

docker run --rm --network demo-net --name app app

Now, in order to connect to your Redis service, you’ll want to pass in some env var(s) denoting the connection parameters. You can use redis.NewClient and redis.Options to configure the connection manually, but I prefer just passing the whole connection string in as a single env and using redis.ParseURL. The full Go program looks like:

package main

import (
    "context"
    "fmt"
    "os"

    "github.com/redis/go-redis/v9"
)

func main() {
    ctx := context.Background()
    opt, err := redis.ParseURL(os.Getenv("REDIS_CONN"))
    if err != nil {
        panic(err)
    }
    rdb := redis.NewClient(opt)
    fmt.Printf("%s\n", rdb.Ping(ctx).String())
}

And the command to run it is:

docker run --rm --network demo-net --name app -e REDIS_CONN=redis://:@redis:6379/0 app

At this point you should be up and running; you’ll see output like ping: PONG, which means the client successfully connected to the Redis service and issue a command. Neat.

To see a list of built in commands, check out the redis.Cmdable interface. There’s far too many to list here, but it’s just about all the common Redis commands you know and love. On the off chance you want to run an unsupported command, you can do so with the Do method on the Redis client.

Use Case: Key-Value Store⌗

The first thing to cover is the basic key-value functionality. The most basic command is Get:

val, err := rdb.Get(ctx, "key").Result()

Redis commands (i.e., Get in the above example) return these <Foo>Cmd concrete types where <Foo> is typically some type like String, Int, Duration, etc. For instance, Get returns *StringCmd. The returned value usually has an underlying val field that has the same type indicated by <Foo>. These returned types have a number of helper methods that perform type conversions for you; they’ll typically return a value of the indicated type and an error. The Result method typically returns the underlying val and an error. If you call Get and your key doesn’t exist, go-redis will return redis.Nil, which is a sentinel error you can explicitly check for and handle.

You can set values with Set:

status := rdb.Set(ctx, "key", "value", 1*time.Second)

Some commands return a *StatusCmd, which typically indicate success/failure. The other noteworthy aspect of Set is that the last argument is a time.Duration indicating how long the key should live in the cache before expiring.

Use Case: Pub-Sub with Redis Streams⌗

This is incredibly easy to implement with go-redis. I’m going to reiterate here that it’s worth your time to look over the Redis docs themselves to become more acquainted with Redis Streams (there’s even a fantastic tutorial; it’s genuinely a great read).

To show how to leverage Streams with go-redis, I’m just going to yoink the code from the examples and add some inline comments and some tweaks:

// subscribe to a stream by providing a context and a stream name
pubsub := rdb.Subscribe(ctx, "some_topic")
// make sure to defer the close
defer pubsub.Close()

// range over the pubsub channel to get messages
ch := pubsub.Channel()
for msg := range ch {
    fmt.Println(msg.Channel, msg.Payload)
}

You can publish to the stream similarly easily:

err := rdb.Publish(ctx, "some_topic", "hello!").Err()
if err != nil {
    panic(err)
}

Now, it’s also worth noting that you have access to the lower level Redis functions themselves, like XAdd, XRead, and so on, but this pubsub abstraction really makes your life easier. Keep in mind that message receiving is not safe for concurrent use by multiple goroutines.

Use-Case: Blocking Streams⌗

I was going to stop here, but I want to provide a more concrete streams example that shows the Streams API and how to iterate over messages. Suppose we want a Stream that’s just a series of random values. Specifically, lets suppose we want to push a map onto the stream. The map will have a key "x" denoting some time, and a key "y" denoting some value. The below snippet shows what that might look like:

// assume some redisClient is available and connected
var redisClient redis.Client
// ...
func (s *service) runStreamSource(ctx context.Context) {
    // Note this `rand.New()` call. This is a **bonus** snippet because the default
    // random number generator is deterministic; this shows one way to generate
    // unique sequences of random numbers. Also note that this is not safe to use
    // for random numbers you intend to be secret; use `crypto/rand` for those.
	r1 := rand.New(rand.NewSource(time.Now().UnixNano()))
	data := map[string]interface{}{}
    // Setup the stream args. MaxLen sets the max length of the stream,
    // and Approx means the length restriction is only approximate; this
    // can be much more performant due to how Redis works under the hood.
	args := &redis.XAddArgs{}
	args.Stream = "rand"
	args.MaxLen = 1000
	args.Approx = true

	// run long lived loop in a goroutine
	go func() {
		// create the ticker
		ticker := time.NewTicker(100 * time.Millisecond)
		defer ticker.Stop()
		// send to client until cancelled
		for loop := true; loop; {
			select {
			case t := <-ticker.C:
				data["x"] = t.Format("2006-01-02T15:04:05.000")
				data["y"] = r1.Intn(100)
				// send data to redis
				args.Values = data
				_, err := redisClient.XAdd(ctx, args).Result()
				if err != nil {
					fmt.Println("Error sending to redis client: ", err)
				}
			case <-ctx.Done():
				loop = false
				break
			}
		}
	}()
}

Now, how do you read from this Stream? Here’s the consumer code below:


func runStreamRead(ctx context.Context) {
    type streamPayload struct {
        X string `json:"x"`
        Y int    `json:"y"`
    }

    // Read indefinitely from the stream and send data through this channel.
	data := make(chan streamPayload, 1)
	go func() {
		lastID := "0"
		args := &redis.XReadArgs{} // blocking read
		for {
			// do a blocking read on the stream
			args.Streams = []string{"rand", lastID}
            // note that the `Block` field is unset. This field is a
            // `time.Duration` denoting how long to block. If it is
            // unset, then the read should block indefinitely since the
            // `go-redis` implementation appears to set the block value
            // to 0 in that case. If you don't want to block, you may
            // set this to a negative number.
			xs, err := redisClient.XRead(ctx, args).Result()
			if err != nil {
				fmt.Println("Error reading redis rand stream: ", err)
			}
			if len(xs) > 0 {
				for _, xMsg := range xs[0].Messages {
					x := xMsg.Values["x"].(string)
					y := xMsg.Values["y"].(int)
					pr := streamPayload{
						X: x,
						Y: y,
					}
					data <- pr
					lastID = xMsg.ID
				}
			}
        }
	}()
    // here you may do whatever you want with the `data` chan above;
    // you probably want to read from it and send that data somewhere
    go func() {
        for loop := true; loop; {
            select {
            // send message to client
            case msg := <-data:
                someEgressChan <- msg
            // handle shutodwn
            case <-ctx.Done():
                loop = false
                break
            }
        }
    }()
    return nil
}

Ok, that’s it for now. I might turn this into a series of posts showing off some more detailed use cases, but I’m more keen to get building with this stuff than just writing about it :)