If you are into SEO, you would know that there is a concept of Disallowing links. This basically means that you are telling Google not to index the URL in their search results. To do this, you have to add some tags to your robots.txt and then Google’s crawler bots will not be able to index those links.
Now, this is done when you know that a link is not important or if someone is trying to spam your website with harmful links. Thus, if any URL is disallowed then it means Google or any other crawler should not collect links from that URL. However, we now have a new explanation from Google’s John Mu regarding disallowed links.
In Twitter conversation with a user, John Mu said that “If a URL is disallowed for crawling in the robots.txt, it can still “collect” links, since it can be shown in Search as well (without its content though). This obviously caused some confusion in the SEO Groups around the world. Because we all had the notion till now that Disallowed links or URLs cannot collect links.
To explain this, we should take the example of Google Search results. You must have seen links in Google Search results which say that no information is available for this domain. Basically, they have disallowed links.
However, you see them because Google has found them relevant for that particular search results. If any website or multiple websites are linking to that particular URL, Google’s crawler sees it as an important URL. It does not matter whether it is a disallowed URL or not.
What this means is that you can build links for a Disallowed URL as well. However, it must be said that it is of no relevance as Google’s crawler will not be able to see content of the URL.
How to Pick the Best Packaging for Your Product
Facebook introduces a Collage Making App, E.gg, to let users design their webpages
Airbnb claimed that Google’s competing travel site has dragged down it’s listing in the search results
Instagram includes a new option for users regarding ad targeting via third-party data usage
Facebook introduces new video series exclusively on Instagram and Messenger
Google Webmasters is altering its name to Google Search Central
Spotify is expanding its podcast advertising options by acquiring Megaphone
Business10 months ago
Wix launches Editor X, website maker for designers and web agencies
Domains9 months ago
8 best domain flipping platforms
News1 year ago
Google Search Rankings showing early signs of an algorithm update
Domains10 months ago
Domains with .com extension are going to become expensive soon
Internet Marketing1 year ago
Snapchat’s new ‘Multi-Snap’ feature and New Sticker options seen in testing
News12 months ago
Google announces News Publisher Center and changes to approval of News websites
Business4 months ago
8 Best Digital Marketing Books to Read in 2020
News1 year ago
DailyMail admits losing 50% traffic after Google June 2019 Core Update