Full text search on Rails: Three easy pieces

01 May 2017 - Seattle

Q: I want to add full-text search to my Rails app and deploy it to Heroku. How do I do that?

This is a well-trodden path, so it’s trivially easy to get something minimal up and running. If you’re deploying to Heroku, there are really only three steps involved:

  1. Install Elasticsearch locally with Homebrew.
  2. Fix inscrutable Java errors by deleting some orphaned indexes deep in the bowels of /var.
  3. Add elasticsearch-rails and elasticsearch-model to your Gemfile and install.
  4. Update your app’s model code so it knows about these.
  5. Create indexes. If you’re feeling saucy, write a rake task to do this.
  6. Add your full-text search functionality to your application code.
  7. Test your app’s full-text search locally.
  8. Make a mental note to fix Travis, which is now broken.
  9. Add an Elasticsearch provider for your Heroku instance.
  10. Update your app’s configuration initializers with Elasticsearch keys for production
  11. Deploy your app to Heroku.
  12. Create indexes on Heroku. Maybe use that rake task. (Not required. That was a stretch goal.)
  13. Fix Travis: Update your .travis.yml file to provision Elasticsearch.
  14. Fix Travis: Explicitly specify database provisioning details in your .travis.yml
  15. Fix Travis: Create a travis-specific copy of your app’s database.yml file
  16. Fix Travis: Add a before_script line to your .travis.yml to use that travis-specific DB config.

What follows is a brief elaboration on this simple procedure in case you need it.

Caveat: The docs are canonical: elasticsearch-rails

The following was done on an open-source demo app running Ruby 2.3.4 and Rails 4.2, on a macOS 10.12.4 machine.

% java -version
# java version "1.8.0_112"
# Java(TM) SE Runtime Environment (build 1.8.0_112-b16)
# Java HotSpot(TM) 64-Bit Server VM (build 25.112-b16, mixed mode)

% elasticsearch -version
# Version: 5.3.2, Build: 3068195/2017-04-24T16:15:59.481Z, JVM: 1.8.0_112

Install Elasticsearch

Use Homebrew. If you don’t have the JDK installed, you can get it using Homebrew Cask:

% brew install elasticsearch
% brew cask install java

Install Elasticsearch libraries

In your Rails app, add the following Elasticsearch gems:

# Gemfile L12-L13 (8fad3e0c)

gem "elasticsearch-model"
gem "elasticsearch-rails"

Gemfile#L12-L13 (8fad3e0c)

Update your app’s model code so it knows about these

See the README for up-to-date usage instructions.

# app/models/squawk.rb L15-L17 (8fad3e0c)

class Squawk < ActiveRecord::Base
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks

app/models/squawk.rb#L15-L17 (8fad3e0c)

# app/models/squawk.rb L71-L78 (8fad3e0c)

# Auto sync with elasticsearch
begin
  Squawk.import
rescue ArgumentError => e
  Rails.logger.info("Forcing creation of ElasticSearch index")
  Rails.logger.info("Exception: #{e}")
  Squawk.import(force: true)
end

app/models/squawk.rb#L71-L78 (8fad3e0c)

Note that the rescue here is most likely rubber band and duct tape. It may not be necessary in your case.

Create indexes

# lib/tasks/elastic_search.rake L3-L8 (8fad3e0c)

namespace :elasticsearch do
  desc "Create elasticsearch indexes"
  task create_indexes: :environment do
    Squawk.__elasticsearch__.create_index!(force: true)
  end
end

lib/tasks/elastic_search.rake#L3-L8 (8fad3e0c)

% bin/rake elasticsearch:create_indexes

Add full-text search to your application code

The heart of this is the #search method, which you’ll most likely want to call in a controller action:

# app/controllers/search_controller.rb L3-L10 (8fad3e0c)

class SearchController < ApplicationController
  def show
    squawks = Squawk.search(params[:q])
                    .paginate(page: params[:page])
                    .records  # or `.results`, if you just want the match text

    render :show, locals: { squawks: squawks,
                            search_term: params[:q] }
  end
end

app/controllers/search_controller.rb#L3-L10 (8fad3e0c)

Note the use of will_paginate‘s #paginate method here. This is not required.

Test your full-text search locally.

Just test it manually for now. Automated testing is left to the reader as an exercise and because I am lazy.

Make a mental note to fix Travis, which is now broken

Whoops!

Add an Elasticsearch provider for your Heroku instance

A decent one is Searchbox.

Again, the docs are canonical: Heroku Dev Center: SearchBox Elasticsearch

% heroku addons:create searchbox

Update your app’s configuration with Elasticsearch keys for production:

# config/initializers/elasticsearch.rb L3-L7 (8fad3e0c)

# Searchbox Elasticsearch configuration
if Rails.env.production?
  Elasticsearch::Model.client =
    Elasticsearch::Client.new(host: ENV["SEARCHBOX_URL"])
end

config/initializers/elasticsearch.rb#L3-L7 (8fad3e0c)

Deploy your app to Heroku

git push heroku master
heroku run rake elasticsearch:create_indexes

Fix Travis

If all the preceding wreaked havoc with your Travis builds, it may be because you need to ensure your runners are provisioned with Elasticsearch. And once you specify one service, you can no longer rely on the sensible defaults.

The docs: Travis-CI: Setting up databases

  1. Update your .travis.yml file to provision Elasticsearch.
  2. Explicitly specify database provisioning details in your .travis.yml
  3. Create a travis-specific copy of your app’s database.yml file
  4. Add a before_script line to your .travis.yml to use that travis-specific DB config.
# .travis.yml L6-L17 (8fad3e0c)

services:
  - elasticsearch
  - postgresql

# for elasticsearch
before_script:
  - sleep 10
before_script:
  - psql -c 'create database travis_ci_test;' -U postgres
  - cp config/database.yml.travis config/database.yml
  - bin/rake db:migrate RAILS_ENV=test
  - bin/rake elasticsearch:create_indexes

.travis.yml#L6-L17 (8fad3e0c)

And that’s it! Three easy pieces.