SEO-Friendly Infinite Scroll

As a developer, at some point you may have to decide between the old fashioned pagination and the trendy new infinite scroll, using the latest technologies, inspired by internet giants like Facebook and Pinterest. It depends on your requirements, but it also may come down to your (or your client’s) personal choice.

Infinite can be an efficient way for users to browse content, but it has a lot of disadvantages and can be the wrong choice on certain types of websites.

One of the primary problems with infinite scroll is the fact that it is, in its barest form, not SEO friendly. When you implement infinite scroll, new content is loaded via Ajax. Unless you take specific measures, this renders the content invisible to search engines. So how then do we make infinite scroll SEO friendly? Let’s discuss a few ways we can do this, including one discussed by Google on the Webmaster Central blog.

Include a Sitemap

If the content you’re loading via infinite scroll is separated into various distinct pages, or units, that have their own URLs, search crawlers like Googlebot may not be able to find that content unless it’s linked somewhere in an accessible spot on one of your already-indexed pages.

A sitemap is one way you can ensure a search crawler can find every page, including the content loaded via infinite scroll functionality. A sitemap is a list of pages in a website, usually in XML format. If a part of your website is inaccessible to web crawlers, you can provide a sitemap to inform the crawler of the full list of pages. For more info about sitemaps and how to create one, you can refer to this tutorial on Tuts+.

If you have an account on Google Webmaster Tools, you can use the service to test and submit your sitemap to ensure that a crawler doesn’t miss your site due to errors in your sitemap or other factors.

This is the method, as mentioned earlier, that Google discusses in a recent blog post. Interestingly, it involves providing the pagination alternative to infinite scroll within the same page! Let’s look into the details.

Using HTML5’s History API

When you load new content, Google recommends that you use HTML5’s History API to change the URL of the page to accommodate the page number when the user scrolls to initiate the loading of new new content. For instance, on loading page 3, the current URL would change to http://myexample.com/?page=3. Similarly, on scrolling up to view the previous content, the URL would change to http://myexample.com/?page=2.

In addition to changing the URL as content is scrolled through, Google suggests that you provide your list of pages at the bottom, which are links to the same pages with page numbers in the URL (e.g. http://myexample.com/?page=3). Just like in traditional pagination, when someone clicks on a page, the page refreshes, and the contents of page 3 are shown. At this point, the infinite scroll functionality will still work, along with the HTML5 History API changes outlined above. So if the user scrolls up or down, new content is shown and the page number is changed on the URL.

The Benefits

So what does this mean for search crawlers? Well, when a crawler visits your site, the content of the first page is shown with page numbers at the bottom. Since the crawler cannot trigger infinite scroll, no new content is going to be loaded. However, because the pagination section is still included on the page, the crawler is able to access all the content. This also ensures that while searching, when someone searches for a particular item on page 3, they land directly on the content and not somewhere else.

This method has the added bonus of making deep linking to infinite scroll content easy. So if I want to share a particular item on the 4th page with someone, I can simply copy the URL (which would be on page 4 at the time) and when someone goes to that URL, they would be able to see exactly what I want them to see.

John Mueller’s Demo

A live demo of the implementation of this idea of infinite scroll with pagination is provided by Webmaster Trends Analyst John Mueller.

And as mentioned, you can read a full description of John’s demo on the Google Webmaster Central Blog.

Conclusion

On one of my recent articles discussing implementing infinite scroll with jQuery, someone commented that as more sites start using this technique, the better it will be for those that don’t. This may very well be true.

Google’s own suggestions, along with what I’ve discussed here should provide more incentive for those who want to use the technique but not suffer from SEO limitations.

If you’ve looked at Google’s suggestions or have had to deal with this on one of your own websites or applications, let us know in the comments.