Boolean Searches with Sphinx

Well, we now have a nice little search system for our entity data. But what if the user want to search for "a" OR "b". What we have so far doesn't handle that. So let's make the modifications we need to make that work.

All I have to say is it's a good thing that the RailsCast on Sphinx addressed this, because I couldn't find this in the thinking_sphinx online docs anywhere. Maybe I just missed it, but I've looked 20 times and just don't see it. (Don't think I'm knocking thinking_sphinx in any way. It's still awesome!)

Anyway, what we'll need to do is modify our search call in our home_controller.rb file. The magic word is :match_mode. So here's the new search line in the controller:

@entities = Entity.search params[:search],
   :page => params[:page],
   :per_page => 10,
   :order => "lastname ASC, firstname ASC",
   :match_mode => :boolean

Notice I've set the match mode to :boolean. This will allow us to do boolean searches. So what is a boolean search? Well, for one thing, it allows us to search for "a" OR "b". However, thinking_sphinx doesn't use the word OR, it uses a pipe (|) symbol. Hmm...I don't think my users will want to do this. I know I'm used to putting the word OR in most of the searches I do. So we'll have to do something about that later. Boolean searches also allow us to use AND, NOT and grouping operators. Those would be &, ! or -, and (). Again, not something most of my users will be able to/like to deal with. (Note that there are other match modes, but I'm not going to be using those. For further information on those, look here.)

So what do we do to allow our end user to use boolean syntax they are familiar with (assuming they're not programmers)? We have to do some string replacement in our code. For simplicity's sake right now, I'm just going to put this in my controller. I'll probably refactor it out later because I don't think that's really the appropriate place for it. Course, I'm a n00b, so what do I know?

Here's what I did. There may be better ways, and maybe I'll stumble on to those later, but for now, this should give us something to work with.

class HomeController < ApplicationController
  def index
    params[:search].gsub!(/ or /i," | ")
    params[:search].gsub!(/ and /i," & ")
    params[:search].gsub!(/\"(.*)\"/,'(\1)')
    params[:search].gsub!(/\'(.*)\'/,'(\1)')
    @entities = Entity.search params[:search],
      :page => params[:page],
      :per_page => 10,
      :order => "lastname ASC, firstname ASC",
      :match_mode => :boolean
  end
end

This makes the necessary replacements in the search string to allow the user to use OR, AND and single- or double-quoted strings in their search. Make sure you include the spaces around the "or" and "and" in your gsub calls! You don't want to be replacing "or" and "and" inside search terms! (Yes I made that mistake.) The /i makes the replacement case-insensitive. This is important because you don't know if the user will type "or" or "OR". Also, notice that we're using gsub! instead of just gsub. This is so that the params[:search] is modified directly. If we had used gsub instead, we'd have to assign each line to a variable, because gsub just returns the value, it doesn't modify the variable it's operating on directly.

So now I'm jazzed! The only thing that bugs me is that when I put a or b in my search box and click the "Search" button, when the search results come back, my search box shows a | b instead of the text I put in. While technically correct, I could see this confusing the tar out of end users. Not a big deal, but attention to detail is important! If I figure this one out, I'll let you know.

Now I'm going to eat lunch before I fall over on my keyboard!

See ya next post!

Chris