• 25 Nov 2020
• 3 Min read

Google launches new Crawl Stats report in Search Console

• by Marc Swann

On 24 November, Google announced that it would be launching a new Crawl Stats report in Search Console. The goal of this is “to help website owners better understand how Googlebot crawls their sites”.

This feature is still rolling out but, if you already have it, you’ll be able to access this new information here.

What’s new?

Google believes its new Crawl Stats report will make it much easier for developers, site owners, and SEOs to identify any issues with Google’s crawling. Here are the four new features they’ve introduced:

1. Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type
2. Detailed information on host status
3. URL examples to show where in your site requests occurred
4. Comprehensive summary for properties with multiple hosts and support for domain properties

As part of this update, they’ve added “over-time charts”. These can provide you with data about your total crawl requests during a particular time period, including on a single day. You can access information about your total requests, total download size, and average response time.

The new version of the report also gives you data on your crawl requests, broken down by response, the file type of the fetched URL, the purpose of the crawl request, and the Googlebot agent. If you would like to see example URLs of each type, you can simply click on the relevant row in the grouping table.

You’ll also be provided with detailed information about any host status issues. This will tell you if and why Google has had trouble accessing your site over the last 90 days. If you have domain properties with multiple hosts, you can also check the status for each of the top hosts you see in the report summary view.

What does this mean for you?

This is a significant improvement on the old Crawl Stats report, and one that most site owners and SEOs will be very happy about. When it comes to SEO and running a website, the more information you have about its performance, the better. And this new report will give you plenty of new stats and alerts that will keep you informed.

With Google giving us more transparency in this area, it’s sure to help build confidence with site owners. The new information we’re being provided with will also help webmasters and SEOs to diagnose technical issues much more easily, without having to invest in complex tools like log file analysers, although those can still be very useful.

In future updates, it would be great if Google could give us the option to filter Search Console crawl reports by URL so we’re able to look at Googlebot activity for specific areas of our clients’ sites. This is definitely a move in the right direction, though.

Here at Glass Digital, we specialise in helping business owners to improve their online visibility through SEO. Our technical SEO team can also help you to ensure your site is performing as well as possible. Get in touch today and we’ll be more than happy to talk through your requirements!

---