pPosted by a href=\”http://moz.com/community/users/155620\”Cyrus-Shepard/a/pp
The folks at Groupon surprised us earlier this summer when they reported the
a href=”http://searchengineland.com/60-direct-traffic-actually-seo-195415″ target=”_blank”results of an experiment/a that showed that up to 60% of direct traffic is organic./pp
In order to accomplish this, Groupon de-indexed their site, effectively removing themselves from Google search results. That’s crazy talk!/pp
emOf course, we knew we had to try this ourselves./em/pp
We rolled up our sleeves and chose to de-index
a href=”https://followerwonk.com/” target=”_blank”Followerwonk/a, both for its consistent Google traffic and its goodnbsp;analytics setup—that way we couldnbsp;properly measure everything. We were also confident we could quickly bring the site back into Google’s results, which minimized the business risks./pp
(We discussed de-indexing our main site moz.com, but… no soup for you!)/pp
We wanted to measure and test several things:/pol
liHow quickly will Google remove a site from its index?/li liHow much of our organic traffic is actually attributed as direct traffic?/li liHow quickly can you bring a site back into search results using the URL removal tool?/li/olp
Here’s what happened./ph2How to completely remove a site from Google/h2p
The fastest, simplest, and most direct method to completely remove an entire site from Google search results is by using the
a href=”https://www.google.com/webmasters/tools/removals” target=”_blank”URL removal tool/a.nbsp;/ppWe also understood, via statements by Google engineers, that using this method gave us the biggest chance of bringing the site back, with little risk. Other methods of de-indexing, such as using meta robots NOINDEX, might have taken weeks and caused recovery to take months./ppstrongCAUTION: /strongRemoving any URLs from a search index is empotentially very dangerous/em, and should be taken very seriously. Do not try this at home; you will not pass go, and will not collect $200!/pp
img src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53d9749576a803.74474426.jpg”/ppAfter submitting the request, Followerwonk URLs started
strong disappearing from Google search results in 2-3 hours/strong.nbsp;/pp
The information needs to propagate across different data centers across the globe, so the effect can be delayed in areas. In fact, for the entire duration of the test, organic Google traffic continued to trickle in and never dropped to zero./ph2The effect on direct vs. organic traffic/h2p
In the Groupon experiment, they found that when they lost organic traffic, they
a href=”http://searchengineland.com/60-direct-traffic-actually-seo-195415″ target=”_blank”actually lost a bunch of direct traffic as well/a. The Groupon conclusion was that a large amount of their direct traffic was actually organic—up to 60% on “long URLs”./pp
At first glance, the overall amount of direct traffic to Followerwonk didn’t change significantly, even when organic traffic dropped./pp
img src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53d9833bac15a6.07738371.jpg”/pp
In fact, emwe could find no discrepancy in direct traffic outside the expected range/em./pp
I ran this by our contacts at Groupon, who said this wasn’t totally unexpected. You see, in their experiment they saw the biggest drop in direct traffic on
stronglong URLs,/strongnbsp;defined as a URL that is at least as long enough to be in a subfolder, like a href=”https://followerwonk.com/bio/?q=content+marketer” target=”_blank”https://followerwonk.com/bio/?q=content+marketer/a./pp
For Followerwonk, the vast majority of traffic goes to the homepage and a handful of other URLs. This means we didn’t have a statistically significant sample size of long URLs to judge the effect. For the long URLs we were able to measure, the results were nebulous.nbsp;/pp
img src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53e935992f4c15.55941500.jpg”/pp
strongConclusion:/strong While we can’t confirm the Groupon results with our outcome, we can’t discount them either./pp
It’s quite likely that a portion of your organic traffic is attributed as direct. This is because of different browsers, operating systems and user privacy settings can potentially block referral information from reaching your website./ph2Bringing your site back from death/h2p
After waiting 2 hours,
strongwe deleted the request/strong. Within a few hours all traffic returned to normal. Whew!/pp
img src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53d979569ca5d1.18916645.jpg”/ph4Does Google need to recrawl the pages?/h4p
If the time period is short enough, and you used the URL removal tool, apparently not./pp
In the case of Followerwonk, Google removed over
a href=”https://www.google.co.uk/search?q=site%3Afollowerwonk.compws=0gl=usgws_rd=ssl”300,000 URLs/a from its search results, and made them all reappear in mere hours. This suggests that the domain wasn’t completely removed from Google’s index, but only “masked” from appearing for a short period of time./ph4What about longer periods of de-indexation?/h4p
In both the Groupon and Followerwonk experiments, the sites were only de-indexed for a short period of time, and bounced back quickly./pp
We wanted to find out what would happen if you de-indexed a site for a longer period, like
strongtwo and a half days/strong?/pp
I couldn’t convince the team to remove any of our sites from Google search results for a few days, so I choose a smaller personal site that I often subject to merciless SEO experiments./pp
In this case, I de-indexed the site and didn’t remove the request until three days later. Even with this longer period, all URLs returned within just
stronga few hours/strong of cancelling the URL removal request./ppIn the chart below, we revoked the URL removal request on Friday the 25th. The next two days were Saturday and Sunday, both lower traffic days./ph4Test #2: De-index a personal site for 3 days/h4p
img src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53d98f6e58e064.25092527.jpg”/pp
Likely, the URLs were still in Google’s index, so we didn’t have to wait for them to be recrawled.nbsp;/ppHere’s another shot of organic traffic before and after the second experiment./ppimg src=”http://d1avok0lzls2w.cloudfront.net/uploads/blog/53e9ce692ac148.39762140.jpg”/pp
For longer removal periods, a few weeks for example, I speculate Google might drop these semi-permanently from the index and re-inclusion would comprise a much longer time period./ph2What we learned/h2ol
liWhile a portion of your organic traffic may be attributed as direct (due to browsers, privacy settings, etc) in our case strongthe effect on direct traffic was negligible/strong./li liIf you accidentally de-index your site using Google Webmaster Tools, in most cases you can strongquickly bring it back to life/strong by deleting the request./li listrongReinclusion happens quickly even after we removed a site for over 2 days/strong. Longer than this, the result is unknown, and you could have problems getting all the pages of your site indexed again./li/olh4
strong/strongFurther reading /h4pMoz community member Adina Toma wrote an excellent a href=”http://moz.com/ugc/how-to-reinclude-your-content-in-googles-search-results-in-less-than-4-hours-28345″ target=”_blank”YouMoz post on the re-inclusion process/a using the same technique, with some excellent tips for other, more extreme situations./ppBig thanks to
a href=”https://twitter.com/petebray” target=”_blank”Peter Bray/a for volunteering Followerwonk for testing. You are a brave man!/pbr /pa href=”http://moz.com/moztop10″Sign up for The Moz Top 10/a, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!/pimg src=”http://feeds.feedburner.com/~r/MozBlog/~4/msT0HWQNME4″ height=”1″ width=”1″/