Improving Google Crawler Speed

How to get Google to crawl our site faster?

There are a few things you can do to improve your crawl speed.

Google crawlers are also know as Googlebots or Google’s web crawling bot, sometimes also called a spider. The more Googlebots crawl your website the better it will be indexed. The better your Google indexing is directly reflected in how high your website appears in a Google search.

Google wants to show a Googler (a person searching on Google) the best results for the search terms they search for. So Google ranks (indexes) websites for search terms. Google works this out with it’s ever changing Analytics.

Have an authoritative website

An authoritative website is basically a website that is recognised buy Google to;

  • Have relevant and recent information, this can be achieved by adding regular, informative, relevant well written (short, well formed sentences with good grammar)
  • Have back links from similar or related website, showing that this website is an important part of this topic being searched on Google.
    Note: irrelevant or spammy links from non related websites or on non related topics can ban a website from any Google search.
  • Be a site with little or no error of coding faults
    – bad links or 301 redirect pages

Google also looks at

  • Your websites security
    – are all the software that your website has up to date?
    – Do you have an SSL Certificate for your website?
    Google now penalisers a website that collects any persons details (like a contact form) if they done have https:// as the URL prefix
  • a XML sitemap installed, so Google can find it’s way around you site easily
  • Pages need to load fast,
    – so good website coding is important,
    – smart plug in are important to improve caching & file loading speed
    – images with the smallest file size is critical
  • Not being Responsive (mobile friendly website) Google penalises websites that are not mobile friendly when searched on a mobile devices, thus your website needs to be Responsive.
  • Too many URL on the one page to the same page etc, as this can slow Google crawlers down, as do spammy links.

Tips to increase your crawl speed

A couple of simple tips to increase your site’s crawl speed:

  • Using the methods above, find and fix all the errors.
    – Especially making your site Responsive (mobile friendly)
    – Have an SSL Certificate – having URL that starts with https://
    – Having a website that all the software is up to date
  • Keep your site recent and relevant with fresh quality well written content
  • Make sure your site is fast.
  • Add an XML sitemap to your site and submit it to the search engines.
  • If all of that fails to improve your crawl speed, start link building!

If you have any questions or need a hand please contact us