Adding search to Hugo with elasticlunr

Well, it finally happened. Shortly after our 50th post, we realized without using google or in current post links it was becoming harder to find articles that had been written in the past. It’s not a secret that we use Hugo, and it is a great platform for static sites, just not for search. After searching on search for Hugo we’ve finally implemented a solution that works for us without the need for outside services.

The hunt

We were hoping that there would be a perfect solution already built that was of the “drop it in” variety. This would be ideal - no code to write or maintain, etc. - sadly it does not exist. Additionally, we didn’t want to have a solution that broke the bank, as there are a lot of those out there too.

Fancy Google URL’s

When you think of search,Google probably comes to mind. This is actually an option - we could have had our search pass a fancy URL to Google. This would have been a quick and easy solution, but a clunky one. No one likes to bounce in and out of a site just to search it.

Write to a database

Another option that was briefly explored was at build time to write all of the posts to a database and then search that. I’m not going to lie - this one really received some consideration. Databases are great at that type of thing and they really shine there. The problem is that writing to a database is yet another build process that would be happening - meaning one more thing to break. That and it just seems wrong on some level for you to build a static site in Hugo, just to dump it into a database for searching.

Third party search

There are a ton of third party search plug-in’s out there with this API or that wiz bang feature. For larger sites this might be the correct solution too, but we’re not at that point yet. Additionally, there is a cost that is associated with them and while we do believe in paying for services, we also believe in only paying for services that meet our needs.

So what did we do?

After all the searching, we ended up going with a Hugo’ish solution, and while not the simple solution that we were looking for, it’s perfect for us right now. Essentially, we are building a JSON file with Hugo that gets read by Elasticlunr to give us a way to search the site without having any external dependency. We’ll cover the code in the next few sections.

You can check it out here, it’s a really neat way to implement the solution.

Hugo JSON Build

Having a JSON file with all the content, titles, and other data is critical to this process. The one we are using is based on this gist - Add JSON seaarch index and adjusted for our needs. Essentially, it’s built into a few parts.

layouts/json/single.html

This is the layout that is needed to build out the JSON. I’ve added and tweaked the fields for our usage, but it’s the general pattern to follow.

{{- $.Scratch.Add "index" slice -}}
{{- range where .Site.Pages "Type" "not in"  (slice "page" "json") -}}
{{- $.Scratch.Add "index" (dict "title" .Title "ref" .Permalink "tags" .Params.tags "content" .RawContent "description" .Description ) -}}
{{- end -}}
{{- $.Scratch.Get "index" | jsonify -}}

content/json.md

This actually calls the layout to be built - not much to look at, but that’s okay!

---
date: 2016-03-05T21:10:52+01:00
type: json
url: index.json
---

Elasticlunr.js

This is an awesome little full-text search engine which runs client side in JavaScript. It’s built on LUNR.js, but has some really cool features such as the ability to boost different parts of the content in a document. Essentially, you pass the generated JSON into it and then it’s able to search it somewhat intelligently.

Loading Elasticlunr

You need to load all the fields into Elasticlunr that you want to use, not just the ones you want to search on. Also, if not already apparent I am using jQuery

$.getJSON("/index.json", function(json){
   index = elasticlunr(function () {
       this.addField('title');
       this.addField('content');
       this.addField('description');
       this.setRef('ref');
   });
   $.each(json, function (key, val) {
     // I filter out a some items here if they are not needed
     if(val.content.length !== 0 && val.description.length !== 0){
       index.addDoc(val);
     }
   })
});

Searching Elasticlunr

The boost option is one of the parts that separated Elasticlunr.js from LUNR.js and stood out the most. It allows you to weight attributes over other attributes in the search.

var results = index.search(value, {
    fields: {
        title: {boost: 3},
        description: {boost: 2},
        content: {boost: 1}
    }
});

Here, all I am doing is passing a value into the Elasticlunr index, where it returns an array of objects that are relevant to the search criteria.

Downsides?

The major downside to this code is the ability to scale indefinitely. Right now the JSON file generated is smaller than some pictures on the site, and it only loads when a reader goes to the search page. As content grows then the file also does, so there will be one point where an action will be needed to address that. Lucky there’s time on our side for that, and we have a few ideas on how to stretch the solution a litter farther. This will really depend on how used the feature is.

Overall and what’s next

This was a really cool little adventure in seeing how far one can go with Hugo and static sites in general. Like I mentioned earlier, the initial plan was for a third party service to take care of this, making this solution even cooler. If you’re a gisty~ user you can probably be assured that you’ll see Elasticlunr.js integration for the search functionality soon!

References

Without the below discussions, libraries, and posts this wouldn’t have been possible.

Always follow the manufacturers instructions, this write up is simply the way that I do it, and it may or may not be the right way. Use your common sense when applying products to or altering your stuff and always wear the appropriate safety gear.