A few days ago, I added search functionality to Phindee so users can quickly find information about a particular happy hour. Search that is well-done often comes with autocomplete, and Phindee is no exception.
Autocomplete in Phindee does a couple of things for the user: 1) it reduces typing, 2) it lets the user quickly know if a specific happy hour is in the database, 3) it allows her to quickly find a particular happy hour that is in the database, and 4) it lets him know if the happy hour is currently happening because it will have a green circle next to it if that’s the case.
What makes this work behind the scenes is an open-source, in-memory, key-value store called Redis. Because it’s in-memory, Redis is fast, which makes it perfect for autocompletion. I’ve known about Redis for a while now, but never had a need to use it, so I’m glad the opportunity finally presented itself. But now that I’ve had a chance to work with it, I’d like to explain how the autocomplete functionality works behind the scenes, and hopefully, teach you a few things for your own projects.
Before we go on, it’s important that you have a basic understanding of Redis. If you’re never used it before, I recommend going through the interactive tutorial on their website; it will help you understand what it’s for, what it can do, and how to use it. Pay special attention to the section on sorted sets because that’s what we’ll be using for autocompletion.
If you’re on a Mac, you can easily install Redis using Homebrew by running the following command:
When it finishes, it’ll give you the command to start the Redis server:
You can then access the Redis command-line by running
redis-cli, which allows you to play around with various Redis commands to see how they work.
Next, you’ll need to hook Redis up with your Rails app, and you can do this by adding the following line to your ‘Gemfile’:
bundle to install it.
Defining a Model for Redis to Work With
First thing we’ll need to do is create an initializer file for setting up our Redis connection. Go ahead and create a file called
redis.rb inside your app’s
/config/initializers directory. Then add the following line into it:
This creates a global variable called
$redis to make it easy for us to access Redis through out our app.
Next, we’ll create a new file called
search_suggestion.rb inside the
/app/models directory. It will contain the code that seeds our Redis database and retrieves a list of suggestions. To start things off, add the following code into it:
1 2 3 4 5 6 7 8 9 10 11 12 13
This creates a class called
SearchSuggestion with a class method called
seed(). Notice that this class doesn’t inherit from
ActiveRecord::Base, which is the base class that the models you create with
rails g model ... inherit from. We don’t need it because we’ll be using Redis instead of ActiveRecord. (By the way, we’re defining a class method instead of an instance method because the logic in this method relates to the class itself, not a specific instance of it.)
All right, now let’s go over the code. Phindee has a model called
Place for storing all the places that have a happy hour, and I’m simply looping over each record stored in it. The reason why I’m doing
Place.find_each instead of
Place.all.each is the
find_each() method works in batches of 1000. This means that if I have thousands of records in my database,
find_each() will load into memory only 1000 at a time, instead of loading them all at once and possibly overwhelming the server, which is the case with
For each place, I’m using the
upto() method to loop over the place’s name n times, where n is the number of characters in the name minus 1 (you’ll see why we’re doing minus 1 later). For example, let’s say the place name is “via delizia”. Our n value would be 10 because the length of the name is 11, but minus 1 brings it down to 10, so we would iterate over the name 10 times.
On the first iteration, n would be 1 and the
prefix variable would be set to the string “v” since we’re extracting the characters from 0 to 1. Then the Redis
ZADD command is used to create a Sorted Set called “search-suggestions:v” since the variable
prefix is set to “v” on the first iteration. (I’m prefixing the set name with “search-suggestions” to keep things organized, but this is not strictly necessary).
Sorted Sets are very similar to Sets because they both store collections of strings, but a Sorted Set also stores an associated score with each string that is then used for sorting. So if we go back to the code, you’ll see that
ZADD initializes the set “search-suggestions:v” with a score of 1 and a value of “via delizia”—the place’s full name.
On the second iteration, a new set will be created called “search-suggestions:vi” since we’re now extracting the characters from 0 to 2, and this initializes the variable
prefix to “vi”. The set itself is then initialized to a score of 1 and a string of “via delizia”, just like the first time.
The same process is then repeated on the subsequent iterations as well. After the 10th iteration, we’ll have 10 different sets initialized to a score of 1 and a string of “via delizia”, like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Note that we don’t create a last set called “search-suggestions:via delizia” because there is no point in returning “via delizia” as a suggested term when a user types “via delizia”. That’s why we added the minus 1 to the length of the name.
By the way, all the scores are identical right now, but they can be incremented later to increase the ranking of popular search terms, although I won’t be covering how to do this here.
Let’s now assume the second place name is “vault martini”. This means that on the very first iteration, with the
prefix variable set to “v” once again, there will be no new set created since we already have a set called “search-suggestions:v”.
ZADD will recognize this and add to the already existing set, instead. This means that the set “search-suggestions:v” will now hold two keys:
And now you can see how autocompletion will work. Whenever a user types “v” in the search bar, we can return a list of search suggestions simply by returning the values in the “search-suggestions:v” set. There is no need for expensive queries that search through the entire database and look for matches. Instead, we find what we’re looking for right away. That’s the beauty of Redis (and other key-value stores).
Extracting Values from a Sorted Set
But how do we extract values from a set? Well, Redis has a command called
ZREVRANGE that does just that. It returns a range of elements sorted by score (with the highest scores listed first). Go ahead and add the following to
1 2 3 4 5 6 7
This function accepts a
prefix variable and uses
ZREVRANGE to return the first 10 elements of a sorted set containing the specified
prefix value. We’ll use it later to return a list of search suggestions to the user.
Creating a Rake Task to Seed Redis
In order to make it easy for us to seed Redis from the command line, we’ll create a Rake task that calls the
seed() method we defined earlier. (If you’re new to Rake, I highly recommend watching the Railscasts episode about it.) Go ahead and create a new file called
search_suggestions.rake inside your app’s
/lib/tasks directory, and add the following into it:
1 2 3 4 5 6 7 8
The code is simple. We’re creating a task called
index and making it dependent on a Rake task provided by Rails called
environment, which loads the Rails environment and gives us access to our
SearchSuggestion class. Then we’re just calling the
seed() method we defined earlier to seed Redis. (We wrap this up inside a namespace called
search_suggestions to keep things neat and organized.)
And now we can
cd into our app’s root directory and call this task from the command line, like so:
You can then go into the Rails console with
rails c and run some Redis commands to see if it worked. For example, if I defined a set called “search-suggestions:v” earlier, I can run the
ZREVRANGE command to return the first 10 elements:
Note that if you want Redis to return the values along with their scores, you need to pass an argument called
with_scores and set it to
true; otherwise, Redis omits the scores.
Setting Up the Front-End
Now that we have the back-end functionality setup, it’s time to set up the front-end. We’ll use the jQueryUI autocomplete widget due to its simplicity and ease of use. We could include it in our app simply by adding the following to our
but this will include the entire library with all the widgets. I don’t like code bloat and prefer to include only the code that I actually need, so we’ll take another route.
Keeping Things Slim
Head over to the jQueryUI download page and under “Components”, deselect the “Toggle All” option, which will deselect all the checkboxes. Then scroll down to the “Widgets” section, select “Autocomplete”, and you’ll see a few other necessary dependencies get selected automatically. Then press “Download”.
If you open the folder it downloaded and go into its
/js directory, you’ll see a file that starts with “jquery-ui-” and ends with a “.custom.js” extension. Open it and copy its code. Then head over to your app, create a new file called
autocomplete.js inside the
Now go back to the folder you just downloaded, go into its
/css directory, find a file with a “.custom.css” extension, open it, and copy its code. Then create another file called
autocomplete.css inside your app’s
/app/assets/stylesheets directory and paste the code into it.
Now we have the code we need and no more.
Hooking It up with HTML
We’re ready to connect the autocomplete code we just added to our app’s HTML. In Phindee, I have a simple form with a search image and an input field that needs the autocomplete functionality:
1 2 3 4 5 6 7 8
In another file, I have the following CoffeeScript code that hooks up the autocomplete widget to the input field I just mentioned above:
1 2 3 4 5 6 7
I’m simply calling the jQueryUI-provided
autocomplete() method on the input field I’m interested in. I then use the
appendTo option to specify that the autocomplete drop-down should be appended to the form itself. Finally, I’m using
source to specify the URL path the autocomplete widget will use to get a list of search suggestions that will be displayed in the drop-down. I chose a path called “/search_suggestions”, but you can choose any path you want.
How It Works
If you look at the documentation for
source, you’ll see that it can accept the search suggestions as an array of strings, a string pointing to a URL that returns an array of strings, or a function with a response callback that also returns an array of strings. We’re using a string pointing to a URL since this fits our needs perfectly.
This is how it will work. The widget will take whatever is typed in the search field and append it to a variable called “term”, which itself will get appended to the URL path we specified in
source. Then it’ll make a GET request to the URL and expect our server to respond with the search suggestions rendered as an array of strings in the JSON format. So for example, if the user types in “v”, the widget will make a GET request to “/search_suggestions?term=v”, and it’ll expect the server to respond with something like
["via delizia","vault martini”].
Our server doesn’t yet know how to respond to a URL like this. Let’s set it up.
Creating a Controller to Handle Requests
First, we’ll create a controller that takes the search phrase the user types in, passes it on to the
terms_for() method we defined in
search_suggestion.rb, and returns the resulting list of suggestions back to the user. We’ll call it
This will create a new file called
search_suggestions_controller.rb. Open it and add the following code inside the
1 2 3 4 5 6 7
We extract the value of the
term variable using
params[:term], pass it on to the
terms_for() method, and tell Rails to render the response in JSON format. Kid stuff.
Then open your app’s
/config/routes.rb file and add the following line into it:
1 2 3 4 5
This maps our
index controller to the path we specified earlier in
source, and our server now knows how to respond to a URL like “/search_suggestions?term=v”.
I think we’re ready for the moment of truth. Restart the rails server, type something in the search field, and if all is well with the world, you should see a drop-down menu with a list of search suggestions. If you don’t, you’ll need to do some debugging to figure out what’s wrong.
Making It Work on a VPS
Installing Redis on a VPS isn’t as easy as running
brew install redis, but it’s not too bad. DigitalOcean has a nice tutorial on the subject. I used it myself to get Redis installed on the server running Phindee, and it worked without a hiccup. I highly recommend it.
Once you have it installed, you’ll need to run the
index task we wrote earlier to seed the database with data. If you’re using Capistrano, you can use the following task to run it from your local computer:
1 2 3 4 5 6 7 8 9 10
If you’re new to Capistrano, feel free to read through an earlier post I wrote, which explains what it is and how you can use it. Or if you’re new to deployment in general, you’re welcome to go through my 6-part series, which covers everything from setting up and securing a server to configuring Nginx, Unicorn, and Capistrano.
All right, that’s all I have. Stay hungry. Stay foolish.