Discover & Crawl hreflang URLs for Specific Pages with Screaming Frog

Last updated on July 20th, 2022 at 02:42 am

This Screaming Frog configuration will allow you to discover and crawl the hrfelang URLs on a specific page or group of pages. I have used this crawl to help discover and trouble-shoot indexing issues on global websites that have the same pages but are localized for specific languages and country markets.

The output of this crawl will include only the URLs you requested to crawl and the hreflang URLs that those pages point to.

If you are looking for a full tutorial on auditing hreflang using screaming frog then go to the tutorial here: https://www.screamingfrog.co.uk/how-to-audit-hreflang/

Time needed: 5 minutes.


How to configure Screaming Frog to crawl hreflang URLs

  1. Reset to the default configuration

    Go to menu: File > Configuration > Clear default Configuration

  2. Set to list mode

    Go to menu: Mode > List

  3. Open crawl configuration settings

    Go to menu: Configuration > Spider

  4. Update spider configuration “Crawl” tab

    Open Spider Configuration “Crawl” tab and check “Hreflang” and uncheck all “Resource Links” and “Page Links”. Uncheck “Crawl All Subdomains” under “Crawl Behavior” and uncheck “Crawl Linked XML Sitemaps”.hreflang sf spider configuration crawl tab

  5. Update spider configuration “Limits” tab

    Uncheck “Limit Crawl Depth” then click OKhreflang sf spider configuration limits tab

  6. Upload URL(s) and run the crawl

    Menu: Upload > Enter Manually. Put in the URL list you would like to crawl and click “Next”sf upload urls in list mode

IF YOUR HREFLANGS ARE ON DIFFERENT TLDs:

If your hreflang URLs point to different TLDs than the ones you pasted in, you will need to add those additional TLD URLs to the CDN Configuration: Menu > Configuration > CDN

How do I crawl just the Hreflang URLs for a particular page or group of pages using Screaming Frog?

The steps outlined in this article walk you through setting up the Screaming Frog configuration to to extract and crawl the Hreflang URLS.

Where can I find additional resources for Screaming Frog?

Screamingfrog YouTube channel
Screamingfrog User Guide
Screamingfrog Tutorials

Where can I find a tutorial on doing a full Hreflang audit using screaming frog?

Screaming Frog posted a tutorial called: How To Audit Hreflang
https://www.screamingfrog.co.uk/how-to-audit-hreflang/

By Eric Mandell

Eric Mandell is a an SEO professional since 1999 and is currently a Director of SEO at Catalyst Performance Marketing Services out of Boston.

Leave a comment

Your email address will not be published.