We have seen that many websites are not comfortable in indexing a link on Google Search or any other search engine. Now, there are various reasons for doing so such as bad links, negative SEO impact etc. Therefore, what site owners do is that they place a noindex tag after the link in their robots.txt file which means that the Googlebot or any other search engine crawler does not index that link.
However, Google has just made a big announcement and it might concern you if you are using noindex links on your website. Google says that GoogleBot will not obey a Robots.txt directive which is related to indexing any longer. This means that it does not matter if you place a noindex tag or not in your robots.txt file, GoogleBot will still crawl the links.
The company says that website owners have time until September 1, 2019 in order to remove the noindex tag and if they want to not index the links, they will have to use an alternate directive. Now, we also have the reason why this directive is cancelled by Google. Recently, Google officially submitted a proposal to make Robots.txt the official directive for website crawlers.
For this reason, the noindex directive has been dropped because it is unofficial. In the past, Google supported this directive because of Robots.txt was not the official method of crawling but since it is now an official method, any unofficial directive cannot be included.
Here is what Google announced:
Today we’re saying goodbye to undocumented and unsupported rules in robots.txt
If you were relying on these rules, learn about your options in our blog post.
In the interest of maintaining a healthy ecosystem and preparing for potential future open source releases, we’re retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019.
Google’s new update on the Search Console Crawl Stats Report
YouTube starts their experimentation on the automated video chapters to improve the navigation
Twitter is going to launch story-like Fleets for its users
YouTube Analytics comes with reports on the traffic sources
Google Ads is working on the Insights page to make business easier
Microsoft Bing shows up with the Green Secure Label to their search results
6 Step Guide for an Effective Online Onboarding Process
Business10 months ago
Wix launches Editor X, website maker for designers and web agencies
Domains9 months ago
8 best domain flipping platforms
News2 years ago
Google Search Rankings showing early signs of an algorithm update
Domains10 months ago
Domains with .com extension are going to become expensive soon
Internet Marketing1 year ago
Snapchat’s new ‘Multi-Snap’ feature and New Sticker options seen in testing
News12 months ago
Google announces News Publisher Center and changes to approval of News websites
Business4 months ago
8 Best Digital Marketing Books to Read in 2020
News1 year ago
DailyMail admits losing 50% traffic after Google June 2019 Core Update