OpenSSL Heartbleed Bug – RedisToGo Password Reset Instructions

On Monday April 7th, a vulnerability in OpenSSL was disclosed known as Heartbleed. This is a serious vulnerability as OpenSSL is widely used to secure HTTPS traffic, and so it affects a large part of the internet.

RedisToGo has patched all of our systems and applied new SSL Certificates. As a precautionary measure, we also recommend that customers update their passwords. We have no evidence that our systems were compromised, but given the malicious nature of the bug, and the amount of time it existed, it is better to err on the side of caution. Instructions on how to do this follows below.

Heroku users are also recommended to change their Heroku Account Passwords

Password Reset Instructions

Step 1 – Login to your RedisToGo account

Step 2 – Click ‘Account’

Redis_To_Go_1

Step 3 – Click ‘Edit’

Redis_To_Go_2

Step 4 – Input a new password, confirm the new password and click ‘Update’

Redis_To_Go_3

Step 5 – Logout of your account and login with your new password

As always, if you have any questions or comments please contact us at support@redistogo.com.

Thanks,

The RedisToGo Team

Konnichiwa – RedisToGo available in the AWS Asia Pacific (Tokyo) Region.

We’re excited to announce that RedisToGo now provides Redis Hosting and Support for AWS AP-Northeast-1 in the Asia Pacific region, or Tokyo to be more specific. The same plans are available as we offer in AWS-US-East, AWS-US-West and AWS-EU-West.

How to create an AWS-AP-Northeast RedisToGo Server

Creating a RedisToGo instance in AWS-AP-Northeast-1 is quick and easy. First, create a new account or login to your existing account and select “New Instance”.

RedisToGo New Instance

Next, a drop-down list on the https://redistogo.com/instances/new page lets you select your availability zone.

RedisToGo Select Asia

Unfortunately, you can’t yet provision to AWS-AP-Northeast via Heroku as Heroku currently only supports US and EU regions.

We love to hear your feedback, so if you have any questions or comments please contact us at support@redistogo.com.

The Evolution of Dispatch at Uber.

The Evolution of Dispatch from benarent on Vimeo.

Building a Distributed State Machine with Node.js and Redis by Amos Barreto.

Filmed at RedisConfNano 2013.

Filmed by Matthew Barker, Edited by Ben Arent.

Made possible with support from Rackspace.

A short-term plan for Redis

A short-term plan for Redis by @antirez from benarent on Vimeo.

Keynote from Salvatore Sanfilippo Aka @antirez from RedisConfNano 2013.

Filmed by Matthew Barker, Edited by Ben Arent.

Made possible with support from Rackspace.

RedisConfNano – A Look Back

RedisConfNano – A Look Back

Last month, more than 50 users and developers got together at RedisConfNano in Portland, Ore. to see Salvatore Sanfilippo (@antirez) talk about future development paths for Redis, and to discover more about how the Redis community is pushing usage forward.

RedisToGo and Rackspace hosted the one-day event.

Here are some highlights of the day (and over the next week or so we will post videos of some of the presentations).

The Morning – Voodoo Doughnuts and Developer Time

It wouldn’t be Portland without Voodoo Doughnuts. We started the day with stacks of pink doughnut boxes and spent time in groups dishing on Redis use cases and problems. The RedisToGo team had some great conversations with companies such as moot.it, which is using Redis as its primary datastore and Regenology, a UK company with a time lapse photography service that uses Redis on a camera-attached Raspberry Pi for queuing and messaging.

The Afternoon – The Main Event – Antirez & Uber

Uber Takes The Stage

After an obligatory Portland Food Truck lunch – again, when in Portland! –  we returned to see Uber’s Amos Baretto present “Building a Distributed State Machine with Node.js and Redis.”

Amos Uber

Uber uses Redis for its driver dispatch service. Uber investigated Redis after experiencing problems scaling its datastore architecture. With the holiday season fast approaching, it had some decisions to make. So why Redis? From Amos’ presentation:

“Mongo couldn’t keep up with GPS points, node-mongodb-native was buggy with regard to replica sets… and we heard it was fast, so why not? [And we were] already using as a cache.”

Today, Uber’s architecture serves roughly 200 dispatch workers and still growing. Uber serves more than 40 cities and Redis is now its datastore, not a cache.

A Short Term Plan From Antirez

Next up, Antirez spoke about “A short term plan for Redis.”

antirez

Salvatore hit on a number of development themes for Redis including Redis Cluster. Here are three potential development areas he mentioned for future versions of Redis during his presentation:

1. Persistence – combine AOF and RDB formats; gain faster AOF rewrites and reloads; one format is better than two.

2. Redis Doctor – an evolution of INFO; check the latency of many operations; store metrics as a time series; tell the user if there are problems

3. Sentinel – here to stay but needs changes; use Redis Cluster algorithms (versioned changes); use persistent state like Redis Cluster

Or, he suggested, just use Redis Cluster itself, only enabling monitoring and failover.

Overall, it was a packed day with enough Redis insight to make whet your appetite for Redis’ bright future.

If you want to see more from the presentations, stay tuned as we will post videos of the day over the next week or so.

Finally, thanks to Ben Arent for organizing and turning around a great event at the last minute.

RedisConfNano Update – October 17th, 2013, Portland OR

RedisConfNano is just under two weeks away and we’re working hard to finalize details for the event.

We’ll spend the day at Union/Pine, starting at 0930 with a hack morning, followed by presentations from 1300. We have some amazing speakers lined up:

  • Salvatore Sanfilippo aka @antirez will give a keynote in person and talk about Redis in 2014.
  • Amos Barreto from Uber will talk about “Build a Distributed State Machine with Redis”.
  • Ezra Zygmuntowicz aka @ezmobius will also be presenting.

Can’t make it? Buy a community pass. Get a RedisConf Tee and early access to the recorded talks.

Buy your tickets for RedisConfNano here.

RedisConfNano

10 Days ago we decided to pull the plug on a full blown Redis Conf production. We’ve decided to run a more intimate, informal and smaller event for the Redis and RealTime community.

redis-conf-nano

The Day – October 17th

We’re splitting the day in half. The first half will be informal coffee and open hack day. A chance to network, share code, problems and anything else your working on. Bring laptops.

The second half of the day we’ll be hosting four speakers. A keynote from @antirez (in person) and three other speakers that are to be confirmed.

The Speakers.

We’re excited to announce the Salvatore Sanfilippo aka antirez will give a keynote in person. Expect to hear updates about Redis 3.0, Redis Cluster and whole lot more.

I’ll be announcing the other three speakers next week.

We will be recording the sessions, with early access to any RedisConfNano community ticket holder supporters.

Tickets.

We’ve selling three tickets for the event.  A ticket for a hackday morning and meetup, a ticket for just the afternoon meetup and a third ticket to help support the event, with the bonus of early access to the Redis Conf videos.

Purchase your tickets today!

If you have already purchased a ticket for RealTimeWeek or Redis Conf, these tickets will be refunded. You will still need to purchase a ticket for RedisConfNano.

Accommodation.
We recommend booking a room at the Jupiter Hotel, it’s the official RealTimeConf hotel. The location of the event is TBC depending on interest of this event.

Flights.
Flights are a great price now, as little as $120 from SFO and $450 from NYC. I would recommend booking now.

Stay for RealTimeConf;  I highly recommend everyone stay around for RealTimeConf

Next year.

We really missed a beat with this years Redis Conf planning, we’re going to rethink what it means for RedisConf14 and will likely roll it into a larger open source conference.

<3 Rackspace

All of this couldn’t be possible without the awesome sponsorship of Rackspace and RedisToGo . They have been a great support in both helping give time and money to get this event off the ground.

If your company would like to help out too, we’re still open to sponsorship for the after party. Please email me for more details.

Lastly a big thanks to @anteriz continued support and help in making something happen.

Looking forward to seeing you all in Portland  ~ Ben

Redis Roundup: What Companies Use Redis?

As a technology, Redis continues to grow in popularity. More and more companies are starting to use Redis, while RedisToGo supports more than 30,000 instances and is growing quickly. Let’s take a look at a few major companies and see how they use Redis.

Instagram – Instagram uses Redis heavily to run their main feed, activity feed and session store. You can check out some of the articles about their infrastructure here and here, see how they scaled in a great presentation by one of the founders, and read about their switch from Cassandra to Redis. Lastly, check out Redis-Faina, a query analyzer that the engineering team at Instagram built.

Github – Github is using Redis for exception handling and queue management.  They also use Redis for configuration management, and as a persistent key/value store for routing all kinds of data.

Stack Overflow – Stack Overflow uses Redis as a caching layer for their entire network.  They praise the speed with which Redis is able to perform: “In our (admittedly limited) experience, Redis is so fast that the slowest part of a cache lookup is the time spent reading and writing bytes to the network.” They report that about 1,300,000 keys are being stored by Redis at any given time, most of which expire within minutes.  At most, several hundred read/write operations per second occur within Redis.  They use around 6GB of memory and have extremely low CPU usage (1%).

Pinterest - Pinterest is a heavy user of Redis. They use it for their follower model, which is their graph of who is following whom (grammar FTW!). In fact, given the tens of millions of users accessing Pinterest every day, this usage is one of the more robust uses of Redis today. For more on their architecture you can see this article, or go through this fantastic transcript about Pinterest’s architecture and Redis usage.

Twitter - Twitter makes heavy use of Redis, and has open-sourced some of the projects they built internally to take advantage of Redis. Twemproxy is a fast proxy for Redis that reduces the connection count on backend caching servers. Manju, the creator of Twemproxy, talked about it during the SF Redis Meetup. Check out the slides!

Tumblr - Tumblr uses Redis to power dashboard notifications for their tens of millions of users. To do so, they built Staircar, a tool that gave vastly better performance than the MySQL setup they were previously using for notifications. Redis a key part of their scalable architecture, as their high scalability interview demonstrated.

Other companies like Twilio, Fullscreen, Craigslist, YouPorn and EngineYard also use Redis. In short, it’s an awesome technology that’s gaining rapid adoption by the tech community – especially those companies with lots of traffic.

If this stuff is interesting to you, also check out this excellent list of Redis usecases. Or, start playing around with Redis with a free instance from RedisToGo.

Redis, PHP and Dragon City

Sometimes someone in the Redisphere does something really cool. In such times of jubilation, we at RedisToGo like to draw attention to others in the Redis community doing cool things. One thing that came across my radar this week was an excellent slide deck by Ricard Clau, a software engineer at Hailo.

If you’re at all interested in using Redis and PHP together this Slideshare will be of interest. Enjoy!

 

Adding Interval Sets to Redis

This is a blog post from Kenny Hoxworth, who writes at Starkiller.net

Redis bills itself as a data structure server directly on its homepage; indeed, the project provides a myriad of data structures accessible in a key-value interface. In fact, Redis provides many key components that one might desire in a remote data structure server: a fast asynchronous networking layer, a reference-counting based memory management layer, multiple built-in persistence strategies, and immediate client support for new commands (for some clients, anyway).

However, what Redis lacks is an easy way to add new data structures to the core. Salvatore has explained why he is against a plugin system, and it seems as if such a system will not make its way to Redis. This means we’ll have to add a new data structure directly to the Redis core.

There are five areas that we’ll address as we add a custom datatype:

  • Initial declarations
  • Memory Management Functions
  • Defining the Custom Datatype
  • Disk Persistence
  • Tying into the Build System

For this exercise, I am working off of the 2.6.13 codeline. A working example of the code in this exercise can be found on GitHub.

Interval Trees

A quick interlude about the datatype we are going to be adding…

A coworker (Bill Mill) and I decided to extend Redis with a custom datatype last year when attempting to determine the set of globally announced CIDR addresses a given IPv4 address falls in. This type of problem is a traditional stabbing problem, one that is commonly solved by interval trees.

To implement interval trees, I decided to build an augmented AVL tree as described in the wonderful CLRS. In such a tree, the nodes are sorted by the minimum value of the range the node represents. Each node carries an additional attribute that represents the largest maximum range value of all of the node’s subtrees. With this information, stabbing queries can easily be written that perform at O(log(n)) complexity.

We represented interval trees in Redis as a “Interval Set” datatype; in such a set, a value is added to the set along with a range. The operations added to Redis act as a somewhat inverse to sorted sets; whereas with sorted sets you can ask for all members and their respective scores that fall within a range, with interval sets, you can ask for all members that have ranges that include a given score.

We can model intervals using standard Redis types and achieve functionality similar to what we will be adding here, but at the cost of two to three orders of magnitude in performance.

Setting the Stage: Declaring the Custom Type

So let’s get started!

We can first add a couple of constants that will be used throughout the process to identify our interval set and its internal encoding. We can add the following to the object types defined starting at line 125 of src/redis.h:


And we can add the following to the object encoding definitions at line 132 of src/redis.h:

#define REDIS_ENCODING_AVLTREE 8

We also need to add a declaration/definition for the dictType used for our custom datatype. This structure is used to properly insert, retrieve, and destroy our custom datatype from Redis’ internal dict type. We add the following to the extern declarations at line 761 of src/redis.h:

extern dictType isetDictType;

And we add the definition where the other dictType definitions are made at line 480 of src/redis.c:

dictType isetDictType = {
    dictEncObjHash,            /* hash function */
    NULL,                      /* key dup */
    NULL,                      /* val dup */
    dictEncObjKeyCompare,      /* key compare */
    dictRedisObjectDestructor, /* key destructor */
    NULL                       /* val destructor */
};

The last Redis-specific declaration we need to make is the declaration of a function responsible for allocating memory for our new datatype. The function responsible for deallocation does not need a declaration, as it will not be accessible outside of the memory management module we’ll edit shortly. We’ll add the creation function to the object implementation section at line 864 of src/redis.h:

robj *createIsetObject(void);

With the Redis-specific declarations done, we should now declare/define the structs needed to represent our interval trees as well as declare the functions required to manage the structures. We can add the structs after the sorted set (“zset”) declarations around line 465 of src/redis.h:

/* ISET specialized AVL structures */
typedef struct avlNode {
       robj *obj;
       double scores[2];
       double subLeftMax, subRightMax;
       char balance;
       struct avlNode *left, *right, *parent, *next;
} avlNode;

typedef struct avl {
       struct avlNode *root;
    dict *dict;
       unsigned long size;
} avl;

typedef struct iset {
       avl *avltree;
       dict *dict;
} iset;

The management functions can be placed after the object implementation section, around line 931 of src/redis.h:

avl *avlCreate(void);
avlNode *avlCreateNode(double lscore, double rscore, robj *obj);
void avlFreeNode(avlNode *node, int removeList);
void avlFree(avl *tree);
int avlNodeCmp(avlNode *a, avlNode *b);
void avlLeftRotation(avl * tree, avlNode *locNode);
void avlRightRotation(avl * tree, avlNode *locNode);
void avlResetBalance(avlNode *locNode);
int avlInsertNode(avl * tree, avlNode *locNode, avlNode *insertNode);
avlNode *avlInsert(avl *tree, double lscore, double rscore, robj *obj);

Finally, we should declare the functions that will handle the external Redis commands exposed for our custom datatype. We’ll add the following declarations at the end of the commands prototypes section at line 1063 of src/redis.h:

void iaddCommand(redisClient *c);
void iremCommand(redisClient *c);
void irembystabCommand(redisClient *c);
void istabCommand(redisClient *c);

As you can see, we’ll be defining four separate operations for managing our custom datatype.

Defining the Memory Management Functions

With most of the declarations out of the way, we can flesh out the object allocation and deallocation routines. Redis represents internal objects as robj types, which include a pointer to the actual data, the type of the data, the encoding of the data, and a reference count for automated deallocation.

We will define our previously declared createIsetObject() function in the src/object.c file, which is responsible for the memory management of objects in Redis. We’ll also define the freeIsetObject() for deallocating the structure.

robj *createIsetObject(void) {
    avl *a = avlCreate();
    robj *o = createObject(REDIS_ISET,a);
    o->encoding = REDIS_ENCODING_AVLTREE;
    return o;
}

void freeIsetObject(robj *o) {
    avlFree(o->ptr);
}

Creating the object is pretty straightforward; we create the augmented AVL structure we will define later, create a new Redis object based off of the tree, and set the encoding.

All we have left to do is tie the deallocation to the reference counter. We’ll modify the decrRefCount() at line 196 of src/object.c and add the following as a case to the switch statement:

case REDIS_ISET: freeIsetObject(o); break;

When the object is no longer needed, it will be automatically freed.

Defining the Interval Set

With the majority of the supporting tie-in code written, we can turn our focus on the actual data structure and the functions that respond to the external commands. First, we should add our command functions to the command table at line 115 of src/redis.c.

{"iadd",iaddCommand,-5,"wm",0,NULL,1,1,1,0,0},
{"irem",iremCommand,-3,"w",0,NULL,1,1,1,0,0},
{"irembystab",irembystabCommand,3,"w",0,NULL,1,1,1,0,0},
{"istab",istabCommand,-3,"r",0,NULL,1,1,1,0,0},

We’ll define that AVL management functions as well as the four above functions in src/t_iset.c, a new module we create to hold just our custom datatype.

I won’t be going through every aspect of the datatype, as the implementation is almost 1,000 lines long (view the full source). However, I do want to touch on a couple of points.

  • The avlCreate() function creates a structure for holding an interval tree as well as an internal Redis dict using dictCreate(). We use this dict to maintain the set of values that have already been added to the interval sets, guaranteeing constant member lookup and deletion.
  • The avlFree() function calls decrRefCount() with the robj at the node being freed. This is required to ensure Redis properly removes the object when no longer referenced (it may still be referenced in the dict mentioned above).
  • The iaddCommand() function is variadic, and thus can be used to add multiple items to the set with a single call. This is a good example method to reference for implementing similar functionality.

Persistence

Redis provides two forms of disk persistence: RDB and AOF. RDB is a binary storage format that represents snapshots of the entire Redis database at any point in time. AOF (append-only file) is an append-only log of all Redis commands that have been run to build the database in its current format.

For our custom datatype, we will build in support for both types of persistence. If you can be sure that you will only use one type of persistence, you can implement support for only that format. Be forewarned, however, that the Redis server will throw a panic if you attempt to use a persistence strategy that is not supported by your custom type.

RDB

We’ll start off by adding RDB support for interval sets. First, we need to define a constant that represents our interval type for RDB at the datatype definitions starting at line 72 of src/rdb.h.

#define REDIS_RDB_TYPE_ISET   5

Next, we need to add the code responsible for saving the interval set. First we need to add the interval set case to the switch statement inrdbSaveObjectType() at line 430 of src/rdb.c.

case REDIS_ISET:
    return rdbSaveType(rdb,REDIS_RDB_TYPE_ISET);

Second, we will add the actual save logic to rdbSaveObject() at line 478 of src/rdb.c.

else if (o->type == REDIS_ISET) {
    avl * tree = o->ptr;
    dictIterator *di = dictGetIterator(tree->dict);
    dictEntry *de;

    if ((n = rdbSaveLen(rdb,dictSize(tree->dict))) == -1) return -1;
    nwritten += n;

    while((de = dictNext(di)) != NULL) {
        robj *eleobj = dictGetKey(de);
        double *scores = dictGetVal(de);

        if ((n = rdbSaveStringObject(rdb,eleobj)) == -1) return -1;
        nwritten += n;
        if ((n = rdbSaveDoubleValue(rdb,scores[0])) == -1) return -1;
        nwritten += n;
        if ((n = rdbSaveDoubleValue(rdb,scores[1])) == -1) return -1;
        nwritten += n;
    }
    dictReleaseIterator(di);
}

This code creates an iterator on the dict internal to the interval set and proceeds to walk the dict. For each element encountered, the element’s value is saved along with the minimum and maximum scores associated with the interval.

After finishing with the save logic, we need to implement load logic in the rdbLoadObject() function at line 771 of src/rdb.c.

else if (rdbtype == REDIS_RDB_TYPE_ISET) {
    size_t isetlen;
    avl * tree;
    size_t maxelelen = 0;

    if ((isetlen = rdbLoadLen(rdb,NULL)) == REDIS_RDB_LENERR) return NULL;
    o = createIsetObject();
    tree = o->ptr;

    while(isetlen--) {
        robj *ele;
        double score1;
        double score2;

        avlNode *inode;

        if ((ele = rdbLoadEncodedStringObject(rdb)) == NULL) return NULL;
        ele = tryObjectEncoding(ele);
        if (rdbLoadDoubleValue(rdb,&score1) == -1) return NULL;
        if (rdbLoadDoubleValue(rdb,&score2) == -1) return NULL;

        /* Don't care about integer-encoded strings. */
        if (ele->encoding == REDIS_ENCODING_RAW &&
            sdslen(ele->ptr) > maxelelen)
            maxelelen = sdslen(ele->ptr);

        inode = avlInsert(tree, score1, score2, ele);
        dictAdd(tree->dict,ele,&inode->scores);
        incrRefCount(ele); /* Added to dictionary. */
    }
}

We start the load code by reading the number of elements to expect during the load; this was automatically written by rdbSaveObject(). We then simply create a new interval set with createIsetObject() and then read in each node one by one, adding each to the tree withavlInsert() and dictAdd.

With that, the RDB code is complete.

Append-Only File

AOF support is much simpler; by default, each command sent to the Redis server is appended to the log, and thus support is already baked in. However, Redis does intelligently rewrite the append-only file when the file gets too big to eliminate redundancy. For instance, if a specific element has been added and deleted 500 times, we shouldn’t have 500 copies of the same add and delete commands. Thus we need to add a method that will iterate the interval set and write out the IADD commands to regenerate the set.

We can add the function after all of the other datatype output functions at line 773 of src/aof.c.

int rewriteIntervalSetObject(rio *r, robj *key, robj *o) {
    long long count = 0, items = isetLength(o);

    if (o->encoding == REDIS_ENCODING_AVLTREE) {
        dictIterator *di = dictGetIterator(((avl *) o->ptr)->dict);
        dictEntry *de;

        while((de = dictNext(di)) != NULL) {
            robj *eleobj = dictGetKey(de);
            double **scores = dictGetVal(de);

            if (count == 0) {
                int cmd_items = (items > REDIS_AOF_REWRITE_ITEMS_PER_CMD) ?
                    REDIS_AOF_REWRITE_ITEMS_PER_CMD : items;

                if (rioWriteBulkCount(r,'*',2+cmd_items*3) == 0) return 0;
                if (rioWriteBulkString(r,"IADD",4) == 0) return 0;
                if (rioWriteBulkObject(r,key) == 0) return 0;
            }
            if (rioWriteBulkDouble(r,(*scores)[0]) == 0) return 0;
            if (rioWriteBulkDouble(r,(*scores)[1]) == 0) return 0;
            if (rioWriteBulkObject(r,eleobj) == 0) return 0;
            if (++count == REDIS_AOF_REWRITE_ITEMS_PER_CMD) count = 0;
            items--;
        }
        dictReleaseIterator(di);
    } else {
        redisPanic("Unknown interval set encoding");
    }
    return 1;
}

This code is quite similar to the RDB save logic; we create an interval associated with the interval set, step through each node, and write the node to the persistence log.

We tie this method into the AOF process by adding a call to the method in rewriteAppendOnlyFile() at line 906 of src/aof.c.

else if (o->type == REDIS_ISET) {
    if (rewriteIntervalSetObject(&aof,&key,o) == 0) goto werr;
}

That should do it for disk persistence.

Tie into the Build System and Try it Out

With the code implemented, all we need to do is add t_iset.o to line 102 of src/Makefile.

With that done, we can compile and run the Redis server:

> make
> src/redis-server

Now we can run a basic test:

redis 127.0.0.1:6379> iadd test 0 10 test1
(integer) 1
redis 127.0.0.1:6379> iadd test 0 20 test2
(integer) 1
redis 127.0.0.1:6379> iadd test 10 30 test3
(integer) 1
redis 127.0.0.1:6379> istab test 5
1) "test1"
2) "test2"
redis 127.0.0.1:6379> istab test 15
1) "test3"
2) "test2"
redis 127.0.0.1:6379> istab test 25
1) "test3"

Hooray! We should now have successfully added interval sets to Redis!

While we had to go through a large number of steps to get us here, adding a datatype and supporting commands to Redis is not very difficult. The supporting infrastructure for Redis truly makes it a great system for storing data structures.

 

 

Thanks Kenny for allowing us to repost this! If you want to get started with Redis, sign up for a free Redis instance!