2 Awesome Uses for Screaming Frog

Recently I’ve had to learn a lot about SEO in a very short amount of time. I would say the biggest “gotcha” moment I’ve had is finding out about and using the Screaming Frog SEO Spider.

The Screaming Frog SEO Spider is a website crawler, that allows you to crawl websites’ URLs and fetch key onsite elements to analyse from an SEO perspective. Download for free, or purchase a licence for additional features.

I have been totally blown away by how you can just scan a whole site in a matter of seconds and see immediately all your broken links and image references. I spent a great portion of my web development career believing that SEO was snake oil sales, but I’m realizing its importance now more than ever ¹.

Screaming Frog rules for performing an SEO audit on your site, but I’d like to point out 2 interesting ways to use Screaming Frog.

1. Identifying Slow Loading Pages

Well of course you’re an incredible web developer who is the Michael Jordan of web page performance and you never have any slow loading pages ever, right? Well if for some reason you might have some slow pages Screaming Frog can identify those immediately.

All you need to do is filter by “Response Time” in the main crawl!

2. Finding All Lazy Loaded Images

By default Screaming Frog is really awesome with images. If it finds it in your markup it will automatically make a request for that image, which rules! Now how about if you’re a radical web developer who lazy loads all their images so that their users don’t have to unnecessarily download each one ²? Well you’re in luck because theres a special part of Screaming Frog called “Custom Extractions”. All you need to do is add either an XPath or a regex for your images like so:

Then we can see that we got all the images on the page, even though those were already lazy loaded:

I didn’t really understand XPath (I still don’t), but I find its easiest to get the global images by simply placing //img. Its even easier to get the value through regex, but this will suffice for most cases.

---

Say what you will about Screaming Frog, it’s Windows 98-esque UI, or the fact that it probably should be a web app that has a SaaS model, it is an incredibly powerful application that definitely has changed the way I view SEO. There is so much more that you can do with the application, but its an essential tool in performing a real SEO audit of your webite.

Footnotes:

1. Caveat, there’s still totally snake oil salesmen in that industry.
2. Lol not me on this site.
3. seerinteractive.com/blog/screaming-frog-guide/ is the best link ever and explains 53 cooler things to do with the software.
4. https://www.screamingfrog.co.uk/10-features/ is also a great list of ways to use the software and actually from the company that develops and maintains it.