NAV Navbar
cURL JavaScript
  • Introduction
  • API Clients
  • Authentication
  • Authorization
  • Rate Limits
  • Buckets
  • Keys
  • Scripting
  • Errors

    • Introduction

    Welcome to the KVdb API! You can use our simple REST API to store arbitrary key-value pairs, which can contain any type of data, such as text, numbers, counters, and binary data. Depending on the data type, the API offers additional features to make collecting and accessing data easier.

    While all the examples are given in Shell script (using curl on the command line), it's possible to request JSON and URL-encoded output.

    API Clients

    // from HTML
<script src="https://unpkg.com/kvdb.io@v1.0"></script>

// from node.js
const KVdb = require('kvdb.io')

    While this document covers the HTTP API, you may also use the kvdb.io-commonjs JavaScript module to interact with the API from a web browser, node.js, or any CommonJS-compatible runtime. Over time, we will add official API clients for other languages. If you wish to contribute one, please let us know!

    Authentication

    To authenticate with a bucket access key or access token:

    // 2nd argument is optional, can be a secret key or access token
const bucket = KVdb.bucket('MY_BUCKET_ID', 'MY_ACCESS_TOKEN')

await flavor = bucket.get('users:1:flavor')

    # Provide the key as the username using HTTP Basic authentication
curl https://kvdb.io/BUCKET/KEY
  -u 'mykey:'

# Or in the query string
curl https://kvdb.io/BUCKET/KEY?access_token=mykey

# Or in the Authorization header
curl -H "Authorization: Bearer mykey" https://kvdb.io/BUCKET/KEY

    While not all API endpoints require the use of an API key, should you choose to use one, use the key in the username field when sending HTTP Basic authentication credentials. The key can also e specified in the key query string parameter.

    In any case, use of the bucket secret key provides full access to the bucket, where as the bucket write key provides write access to any keys in the bucket. You SHOULD NOT use either of these in client-side applications.

    Authorization

    To generate an access token valid for 1 hour with read/write permission for the prefix user:123:

    const tokenOpts = {
  prefix: 'user:123:',
  permissions: ['read', 'write'],
  ttl: 3600,
}

await token = bucket.accessToken(tokenOpts)

    curl https://kvdb.io/BUCKET/tokens/
  -d 'prefix=user:123:&permissions=read,write&ttl=3600'
  -u mykey:

    To authorize a request with an access token:

    // pass the access token as the 2nd argument to the bucket function
const bucket = KVdb.bucket('MY_BUCKET_ID', 'MY_ACCESS_TOKEN')

    # Provide the access token in the Authorization header
curl https://kvdb.io/BUCKET/KEY
  -H "Authorization: Bearer MY_TOKEN"

# Or in the query string
curl https://kvdb.io/BUCKET/KEY?access_token=MY_TOKEN

    For most applications, you do not want to give out the bucket's secret or write keys, since it gives clients access to all keys in the bucket. Instead, you can use the KVdb API to generate access tokens, which grant the holder access to a specific prefixed portion of the key space with an exact set of permissions. By default, access tokens are stateless and have a fixed lifetime.

    Key naming schemes are often prefixed with some path containing the application user ID, for example, user:123:email, user:123:profile, etc. A secure way to grant a user of your application read/write access to only their own set of sub-keys is to generate an access token for the key with their user ID prefix, user:123: for instance. Furthermore, the access token can embed only the minimum set of permissions required.

    Access tokens offer the ability to securely provide external access to your bucket resources without an intermediate authentication or authorization layer, while still maintining application-level control.

    Rate Limits

    The rate limit is 1,000 requests per IP address per hour. Refer to the X-Ratelimit-Limit, X-Ratelimit-Remaining, and X-Ratelimit-Reset response headers to know when you are approaching the limit. Once the rate limit is reached, the server will return a HTTP 429 status code until the rate limit resets.

    Paid plans have a much more generous rate limit allowance.

    Buckets

    A bucket is a collection of keys. The keys in a bucket share the same access keys, expiration (TTL) setting, and other bucket policy. Depending on your application, you may use one bucket for all your keys, or segregate keys based on users, groups, or other categorization in your application.

    You may create bucket-specific access keys:

    Key Default Set this to...
    secret_key None Manage bucket policy and other keys
    read_key None Prevent public reads from the bucket
    write_key None Prevent public writes to the bucket
    signing_key None Enable access token generation

    Set the signing_key to a random, yet secure value, as it will be used to generate cryptographically signed access tokens. Changing the signing_key will immediately invalidate all previously issued access tokens.

    In addition, you may configure the following bucket policies:

    Setting Default Meaning
    default_ttl 604800 (1 week) Keys not updated expire after this time (seconds)

    Create a Bucket

    When creating a bucket, ensure that you set a valid email address in order to be able to manage the bucket in the future and avoid loss of data.

    // Create a new bucket that lets anyone read, write, and list keys
const bucket_id = await KVdb.createBucket({email: 'user@example.com'})

    # Create a new bucket that lets anyone read, write, and list keys
curl -d 'email=user@example.com' https://kvdb.io

    Create a more restricted private bucket:

    const bucket_id = await KVdb.createBucket({
  email: 'user@example.com',
  secret_key: 'supersecret',
  write_key: 'knock',
  default_ttl: 3600
})

    # Create a new bucket with a secret_key, write_key,
# and set all keys to expire after 1 hour
curl https://kvdb.io \
     -d 'email=user@example.com' \
     -d 'secret_key=supersecret' \
     -d 'write_key=knock' \
     -d 'default_ttl=3600'

    Our API will generate a bucket id for you to use. Make sure to set a secret_key to protect the bucket. It is not possible to add a secret_key to a bucket after creation, however, it is possible to change it.

    There is no API to list publicly-accessible buckets, so using a bucket without an access key is safe as long as the bucket id isn't made public. We only recommend these kinds of buckets for testing purposes.

    Update Bucket Policy

    # Make bucket read-only
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF \
     -d 'write_key=knock' \
     -u 'supersecret:' \
     -XPATCH

    If a bucket has a secret_key set, you may update its policy (and the secret_key itself).

    List Keys

    # List keys
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/ \
     -u 'supersecret:'

# List keys with the prefix hits_2018
curl 'https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/?prefix=hits_2018' \
     -u 'supersecret:'

# List keys and their values as JSON
curl 'https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/?values=true&format=json' \
     -u 'supersecret:'

    This would return an array of JSON arrays with key-value pairs:

    [
  ["visitors", 101],
  ["flavor-today", "coconut"]
]

    Listing keys in a bucket is simple. If the bucket is protected by a secret_key, be sure to provide it.

    The following query string paramateres can be used to influence the output:

    Parameter Default Meaning
    limit 10000 Maximum number of keys to return
    skip 0 Number of keys to skip before returning first result
    prefix none Only return keys matching the given prefix
    reverse false If true, reverse lexicographic sort order
    values false If true, return values in addition to keys
    format Accept header Change the output format (see below)

    The HTTP Accept header is used to determine the output if the format parameter is not specified. If no specific header is sent, text is assumed.

    The following formats are supported:

    Format Without Values With Values Value Escaping
    text key key=value Newline: \n
    json ["key"] [["key", value]] None
    jsonl "key" ["key", value] None

    jsonl is newline-delimited JSON, a format easy to process with command-line and other tools expecting one JSON object per line. It must be specifically requested with a ?format=jsonl query string.

    If you have stored any JSON items in your bucket (by setting Content-Type header to application/json when setting a key-value), then those values are returned as JSON instead of escaped strings when requesting json or jsonl output format.

    Delete Bucket

    // Use the HTTP API directly to delete a bucket

    # Delete bucket
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/ \
     -XDELETE
     -u 'supersecret:'

    Use this to delete a bucket and all of its keys. For large buckets, this operation may not complete immediately, even though a successful response is returned by the API.

    Keys

    Key names can be of arbitrary composition (even binary data), but are limited to 128 bytes in length. Values can be of any data type but are limited to 16 KB in size. Numeric values are detected automatically and are enforced by the API.

    Set a Key Value

    // Set key
await bucket.set('hello', 'world')

// Set key with a 30 second expiry
await bucket.set('hello', 'world', {ttl: 30})

    # Set key
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/hello \
     -d 'world'

# Set key with a 30 second expiry
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/hello?ttl=30 \
     -d 'world'

    The maximum length of a key is 128 bytes.

    To override the bucket's default key expiration, use the ttl query parameter to specify the TTL for this key.

    Execute Transaction

    # Execute transaction with 1 set and 1 delete
curl -H "Content-Type: application/json" \
     https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF \
     -d '
{
  "txn": [
    {"set": "users:email:new@example.com", "value": "user 1"},
    {"delete": "users:email:old@example.com"}
  ]
}'

    To set multiple keys in one atomic operation, use the transaction API. Construct a JSON object with a txn array consisting of one or more of the following objects:

    Set

    Field Value
    set Key to set
    value Any supported value: string, number, JSON
    ttl Key expiration, defaults to bucket TTL if not specified

    Delete

    Field Value
    delete Key to delete

    Get a Key Value

    // Get value
await bucket.get('hello')

    # Get value
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/hello

    The response is the value:

    world

    Update a Key (Counter)

    await bucket.incr('visits', 1)

    # Increment counter by 1
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/visits
     -d '+1'
     -XPATCH

    The API responds with the new value of the counter:

    1

    To store counters, the server will detect integer (64-bit signed) and floating point (64-bit) values and store them efficiently, to allow incrementing and decrementing them. Use the PATCH HTTP method to operate on a value this way. If the key does not exist, it is created and assumed to be zero before the operation.

    To increment the value, use +.

    To decrement the value, use -.

    You can also use the ttl query parameter to set the TTL for this key.

    Delete a Key

    await bucket.delete('hello')

    # Delete key
curl https://kvdb.io/N7cmQg1DwZbADh2Hu3NncF/hello \
     -XDELETE

    If you need to delete a key before it expires, use this method.

    Scripting

    KVdb provides a powerful Lua-based scripting API that lets you build server-side functionality with fast access to keys and values. Each bucket can have one or more scripts, which execute in the context of that bucket and an HTTP request. Furthermore, the keys a script can access can be restricted by the permissions granted by the access token in the HTTP request. With these features, scripts can be used to implement any imaginable level of server-side business logic.

    Refer to the Scripting Reference for the full Lua API.

    To view and create your scripts, visit https://kvdb.io/BUCKET/scripts/

    Errors

    The API uses the following error codes:

    Error Code Meaning
    400 Bad Request -- Your request is invalid.
    401 Unauthorized -- Your API key is wrong.
    403 Forbidden -- Bucket policy blocked the request.
    404 Not Found -- The specified key could not be found.
    405 Method Not Allowed -- You tried to access a bucket or key with an invalid method.
    406 Not Acceptable -- You requested an unsupported output format.
    410 Gone -- The requested resource has been removed from our servers.
    418 I'm a teapot.
    429 Too Many Requests -- You're making too many requests! Slow down!
    500 Internal Server Error -- We had a problem with our server. Try again later.
    503 Service Unavailable -- We're temporarily offline for maintenance. Please try again later.
    cURL JavaScript