We have seen that many websites are not comfortable in indexing a link on Google Search or any other search engine. Now, there are various reasons for doing so such as bad links, negative SEO impact etc. Therefore, what site owners do is that they place a noindex tag after the link in their robots.txt file which means that the Googlebot or any other search engine crawler does not index that link.
However, Google has just made a big announcement and it might concern you if you are using noindex links on your website. Google says that GoogleBot will not obey a Robots.txt directive which is related to indexing any longer. This means that it does not matter if you place a noindex tag or not in your robots.txt file, GoogleBot will still crawl the links.
The company says that website owners have time until September 1, 2019 in order to remove the noindex tag and if they want to not index the links, they will have to use an alternate directive. Now, we also have the reason why this directive is cancelled by Google. Recently, Google officially submitted a proposal to make Robots.txt the official directive for website crawlers.
For this reason, the noindex directive has been dropped because it is unofficial. In the past, Google supported this directive because of Robots.txt was not the official method of crawling but since it is now an official method, any unofficial directive cannot be included.
Here is what Google announced:
Today we’re saying goodbye to undocumented and unsupported rules in robots.txt
If you were relying on these rules, learn about your options in our blog post.
In the interest of maintaining a healthy ecosystem and preparing for potential future open source releases, we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.
SEMRush forced to ‘pause’ its Guest Post Service after controversy
Google explains how you can know which pages should be removed
Google says its indexing issues from last week have been fixed now
John Mueller from Google reveals how some guest posts can look unnatural
How to leverage the internet to market online gaming
Google has announced new ranking signals for its web search
Facebook ‘Collab’ launched as a mix-and-match app for collaborative music videos
Business4 months ago
Wix launches Editor X, website maker for designers and web agencies
Domains4 months ago
8 best domain flipping platforms
News1 year ago
Google Search Rankings showing early signs of an algorithm update
Internet Marketing12 months ago
Snapchat’s new ‘Multi-Snap’ feature and New Sticker options seen in testing
Domains4 months ago
Domains with .com extension are going to become expensive soon
News6 months ago
Google announces News Publisher Center and changes to approval of News websites
News12 months ago
DailyMail admits losing 50% traffic after Google June 2019 Core Update
News9 months ago
Google is reportedly still ranking syndicated content over original publishers