Discover & Crawl hreflang URLs for Specific Pages with Screaming Frog

This Screaming Frog configuration will allow you to discover and crawl the hrfelang URLs on a specific page or group of pages. I have used this crawl to help discover and trouble-shoot indexing issues on global websites that have the same pages but are localized for specific languages and country markets.

The output of this crawl will include only the URLs you requested to crawl and the hreflang URLs that those pages point to.

Time needed: 5 minutes.


Hreflang Crawl Settings for Screaming Frog

  1. Reset to the default configuration

    Go to menu: File > Configuration > Clear default Configuration

  2. Set to list mode

    Go to menu: Mode > List

  3. Open crawl configuration settings

    Go to menu: Configuration > Spider

  4. Update spider configuration “Crawl” tab

    Open Spider Configuration “Crawl” tab and check “Hreflang” and uncheck all “Resource Links” and “Page Links”. Uncheck “Crawl All Subdomains” under “Crawl Behavior” and uncheck “Crawl Linked XML Sitemaps”.hreflang sf spider configuration crawl tab

  5. Update spider configuration “Limits” tab

    Uncheck “Limit Crawl Depth” then click OKhreflang sf spider configuration limits tab

  6. Upload URL(s) and run the crawl

    Menu: Upload > Enter Manually. Put in the URL list you would like to crawl and click “Next”sf upload urls in list mode

IF YOUR HREFLANGS ARE ON DIFFERENT TLDs:

If your hreflang URLs point to different TLDs than the ones you pasted in, you will need to add those additional TLD URLs to the CDN Configuration: Menu > Configuration > CDN

How do I crawl just the hreflang URLs for a particular page or group of pages using Screaming Frog?

The steps outlined in this article walk you through setting up the Screaming Frog configuration to to extract and crawl the hreflang URLS.

Where can I find additional resources for Screamingfrog?

Screamingfrog YouTube channel
Screamingfrog User Guide
Screamingfrog Tutorials

By Eric Mandell

Eric Mandell is a an SEO professional since 1999 and is currently a Director of SEO at Catalyst Performance Marketing Services out of Boston.

Leave a comment

Your email address will not be published.