Crawl delay is supported by Bingbot but not by Google as per reports

One thing that is not necessarily related to a website’s SEO but related to the performance of your website is how fast or slow a website loads. And we know that the site’s load times depend on the server performance but when you are talking about websites which are indexed by search engines, it also depends on the crawl times. Basically, every website around the world which is submitted to search engines will have to be crawled by Googlebot. This is to see if any new content has been added to the site so that it can be indexed in the search results. Sometimes, it happens that a search engine’s crawler is so fast that your site’s performance can be affected.

This is because the crawler would be crawling too many times per second which the server might not be able to handle if it does not have the required size. It is known that Googlebot crawls a website 5 times per second and it is considered as standard crawl time. However, you cannot make it any faster or slower as Google’s crawler does not support crawl delay. But it is now revealed that Bingbot supports crawl delay which means you can make the crawl rate faster or slower as per your server and website needs.

The fact that Googlebot does not support crawling delay was well documented on its page but Google’s John Mueller has also clarified the same publicly on Twitter now. On the other hand, Bing’s Frédéric Dubut says that Bingbot does support crawl-delay. Having said that, you can still slow down the crawl rate of Google’s bot if it feels too fast for you. To do that, you will have to reach the official documentation from Google on changing the crawl rate from here.