Inbound Thinking

Sometime last year we started running into trouble with the HubSpot dashboard application for all of our customers. The dashboard was beginning to load slowly, very slowly. I investigated the slowness for a while, with help from my co-worker Stephen Huenneke, and eventually figured out that the problem was a collection of hundreds and sometimes thousands of items that were being loaded unnecessarily for the dashboard.

The dashboard’s purpose was to show customers what’s happening with their marketing: if they get new leads, new social media stories, new inbound links, etc. The first version of the dashboard had no paging and after 20 or so items it started to be a long page with much scrolling. Paging was added. But the first version of paging was all done in the freemarker templating language and, although we only displayed 10 items, all of them were still being fetched from the database.

It was obvious that loading all the items was a waste of resources, we only needed the first page of items. This was easy to do with MySQL, just throw offset and limit on the query, right? Well, not quite. Although it was easy to get the first page of items this way, it was not clear that there was a second page of items to be fetched. What if you have exactly 10 items? You get a full page of them and then it is not clear if we should show a link to the next page of items. We didn’t want to run a count query to find out how many total items there were since that is another database call and every call takes time, we wanted the dashboard to be a very fast load since it is the first page our customers see. We also did not care about how many total pages were available, we just wanted to show a single “next page” link.

After a few discussions, Stephen and I came up with what we called the plus-one paging model. To display a page size of 10 items, we fetched 10 + 1 (=11) items from the database, then, in freemarker, we looped over those items and displayed only the first 10. The 11th item was used to determine if we should show the link to the next page. If there were 10 or fewer items we knew there was no need for the next page. Getting a “previous page” button was easy: The “next page” link had a querystring parameter for page number. On the second and greater pages we showed a link to the previous page. The page number parameter was also used to calculate the offset for the other pages: page number * page size = offset (first page number = 0).

I have no idea if others already use the plus-one paging model, but I’ve found it to be pretty handy for situations where you need paging and do not need to know the total number of pages. If you find it helpful please let me know with a comment.