Opaque Cursors with DynamoDB

I’m pretty sure I found this somewhere, but decided to document here for myself and others who find it useful.

In an attempt to keep the implementation of DynamoDB pagination transparent to an API, I wanted to obfuscate the usage of item keys and use just an opaque string cursor that can be passed around. This is what that looks like:

import { marshall, unmarshall } from "@aws-sdk/util-dynamodb";
import type { QueryCommandInput } from "@aws-sdk/client-dynamodb";
import { QueryCommand } from "@aws-sdk/client-dynamodb";

export const getEvents = async (storeId: string, cursor?: string) => {
  const startKey = cursor
    ? marshall(JSON.parse(Buffer.from(cursor, "base64").toString("ascii")))
    : undefined;

  const queryCommandInput: QueryCommandInput = {
    TableName: "events",
    KeyConditionExpression: "#pk = :pk",
    ExpressionAttributeNames: {
      "#pk": "pk",
    },
    ExpressionAttributeValues: marshall({
      ":pk": `STORE#${storeId}`,
    }),
    ExclusiveStartKey: startKey,
  };
  const queryCommand = new QueryCommand(queryCommandInput);
  const queryResult = await dynamodb.send(queryCommand);
  const resultItems = queryResult.Items
    ? queryResult.Items.map(result => (unmarshall(result) as EventRecord).data)
    : [];

  const lastKey = queryResult.LastEvaluatedKey
    ? Buffer.from(
        JSON.stringify(unmarshall(queryResult.LastEvaluatedKey))
      ).toString("base64")
    : null;

  return {
    cursor: lastKey,
    events: resultItems,
  };
};

The crucial components are the first bit and the last bit of this function. To get an opaque cursor, we unmarshal the last evaluated key in the response from Dynamo, JSON.stringify it, and encode it into Base64.

On the flip side, when given a cursor to us as the start key for another query, we Base64 decode it, JSON.parse it, and marshal that key.

Simple as that.