We have seen that many websites are not comfortable in indexing a link on Google Search or any other search engine. Now, there are various reasons for doing so such as bad links, negative SEO impact etc. Therefore, what site owners do is that they place a noindex tag after the link in their robots.txt file which means that the Googlebot or any other search engine crawler does not index that link.
However, Google has just made a big announcement and it might concern you if you are using noindex links on your website. Google says that GoogleBot will not obey a Robots.txt directive which is related to indexing any longer. This means that it does not matter if you place a noindex tag or not in your robots.txt file, GoogleBot will still crawl the links.
The company says that website owners have time until September 1, 2019 in order to remove the noindex tag and if they want to not index the links, they will have to use an alternate directive. Now, we also have the reason why this directive is cancelled by Google. Recently, Google officially submitted a proposal to make Robots.txt the official directive for website crawlers.
For this reason, the noindex directive has been dropped because it is unofficial. In the past, Google supported this directive because of Robots.txt was not the official method of crawling but since it is now an official method, any unofficial directive cannot be included.
Here is what Google announced:
Today we’re saying goodbye to undocumented and unsupported rules in robots.txt
If you were relying on these rules, learn about your options in our blog post.
In the interest of maintaining a healthy ecosystem and preparing for potential future open source releases, we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.
WordPress 5.9 launch postponed until 2022
Google Ads App comes up with 3 new traits
Data breach intrusions at 6 more web hosts
Republished reviews not worthy for rich search results
Google explains when to use Rel Canonical or Noindex
YouTube co-founder not in favor of removing dislikes
Twitter planning to drop support for AMP
Domains2 years ago
8 best domain flipping platforms
Business2 years ago
Wix launches Editor X, website maker for designers and web agencies
Business1 year ago
8 Best Digital Marketing Books to Read in 2020
News3 years ago
Google Search Rankings showing early signs of an algorithm update
Internet Marketing5 months ago
Who is David Bond? A Look Into The Life Of A Travel YouTuber
Internet Marketing2 years ago
Snapchat’s new ‘Multi-Snap’ feature and New Sticker options seen in testing
How To's2 years ago
How to submit your website’s sitemap to Google Search Console
Domains2 years ago
Domains with .com extension are going to become expensive soon