SEO WebMonkey » Google

rel=canonical: trying to get Google to understand

ndixon — Tue, 14 Apr 2009 20:49:41 +0000

Back in February, Google announced it had begun supporting the rel=canonical hint in determining the definitive URL for an item of content. This sounds like a one-stop solution to all our duplicate content problems.

For example, my previous post can be reached by the full URL, but also (because of the Wordpress system defaults) via

http://seowebmonkey.com/?p=113

If someone decides to link to my post using that URL, it might therefore also enter the search index in addition to the full version – thus duplicate content.

Placing a rel=canonical instruction in your page’s header can resolve this problem before it occurs. It tells the search engine which URL to use as the definitive (canonical) URL for that page of content, regardless of what was used to get there.

Here’s an example for the above page:

This sits within the section of your page’s html.

This link-tag is supported by Google, Yahoo, Ask.com and Microsoft Live Search.

Does it work?

Google describes its support of rel=canonical as a “hint”. This means it will use the information to determine a canonical URL, but reserves the right to do what it wants when it feels like it. This seems to be a way to cover for errors that slip through the net. Search engines are rarely predictable, and no single method should be trusted in avoiding duplicate content.

For all content management systems in particular, this is an essential addition to the page output. How rapidly Google will change any existing duplicative content URLs that are already in its index is yet to be clearly determined, and this alone will not enable webmasters to explicitly request removal of duplicative URLs via the Google Webmaster Tools interface.

Direct removal of pages from Google’s search index will be covered in my next post.

Duplicate content in web applications

ndixon — Fri, 10 Apr 2009 10:37:05 +0000

When first approaching a new SEO project, one of the very first research tasks is to assess the current status of the client site within the search engines – particularly, of course, Google. Right at the top of such tasks is a hunt for duplicate content and consequently, pages that have been filtered into the Supplementary Index.

Defining duplicate content

A search engine considers every unique URL to be the location of a single web page. It learns about such URLs by following links. Duplicate content occurs when a search engine follows more than one unique URLs that take it to a page of predominantly similar content – these pages do not need to be identical, just very similar (a defined ratio is something of a holy grail for optimisers and not clearly established).

Why is this important? A search engine will not display more than two pages from a website for a particular search. It makes an automated decision of which pages are the authoritative or originating version of that content, tucking the rest away at the end of the results: the Supplementary Index.

The damage to SEO comes from a number of factors, the two main causes being:

The page you most want to surface may become inadvertently pushed into the Supplementary Index, thus giving searchers an inferior, poorly converting page to visit.
A major factor in search result visibility is the number (and quality) of links pointing to a page, but multiple URLs to a page frequently dilutes the combined effectiveness of back-links as the links point to different unique URLs.

From my direct experience with sites over the past year in particular, duplicate content pages result in a cumulative deterioration of the entire site’s visibility in SERPs (Search Engine Result Pages).

If you have a site that is failing to rank well for even the least competitive search terms, such duplicate content is likely to be a contributing factor, and you will struggle to establish a firm footing until this problem is resolved.

Spotting the damage

Discovering duplicate content is relatively straightforward. Perform a search for your website on Google, like this:

site:yourdomain.com keyword

where yourdomain.com is your website’s domain, and keyword any terms for which you are trying to be visible. Run down to the very end of the results and if you see a message like this:

you have duplicate content!

The cause

Content Management

If you have a duplicate content issue, then I will bet your website runs on some form of content management. Blog applications are notoriously prone to these problems, with supplementary listing pages such as category summaries, tag summaries, archive lists, and search results, all in danger of generating pages with very similar content.

In addition, some content systems do not chose a single means of generating a URL for a item of content, linking to pages and posts in slightly different ways from different parts of the system. They may also fail to redirect all URLs to the canonical URL for that page or post.

Identical meta data

In addition, it is very common to come across sites with many – sometimes even all – their pages with identical titles and META descriptions . This is particularly prevalent once again with blogs, and also in the small business space where a company feels it must apply its company name and corporate blurb at the top of every page, thus not accurately reflecting that page’s actual content.

Page titles are very important in establishing the context of a page, and must be unique in order for the search engine to properly understand the page content.

Too little unique textual content

Pages with little unique text content can become regarded as duplicative because the majority of the content there is similar to everywhere else on the site. In these sparse content examples, the site-wide navigation, footer information, and other generic text, can form the majority of the textual content.

Search, archive and summary pages

As mentioned above, pages that summarise and list snippets of other content can easily appear to a search engine to be very similar but have unique URLs.

www and non-www domains

This one often surprises web developers, but www.yoursite.com is, to a search engine, a different website to yoursite.com. This means that if all your content is reachable via both those versions, your entire site is seen as being a duplicate!

It does not matter which you choose, but have one of those URL versions permanently redirected to the other.

Repairing the damage

Now that I have covered most of the indicators and causes, how about a means of fixing the problem? Watch out in a few days for specific techniques to repair and remove duplicated content within search engines.

Is Black-Hat SEO on the way out?

ndixon — Tue, 06 Jan 2009 09:39:21 +0000

As search engines evolved, so the methods each used to determine the results of a search were probed, prodded and tested to breaking point. Thus the Black-Hat SEO was born with the intent to achieve big profits by directly manipulating inadequacies in the search engines’ techniques.

What makes SEO Black-Hat?

The boundaries between “white” or “black” SEO techniques are thoroughly blurred with a range of grey tones. Boiling down to the core principles of Google’s thoughts on the matter, one might consider any technique deliberately and strategically employed with the intent to manipulate a search engine’s results could be considered Black-Hat. Google defines it as techniques which violate their terms of service.

But when do we step into the dark territory? Could it be the moment we simply become aware that adding a few strategic keywords to a piece of text and creating some links to it can enhance (manipulate) that page’s position in the search results?

Is Black-Hat in decline?

Once the stuff of dark alleys and locked away in grubby discussion forums, the secrets of the Black-Hatter are now to be found by anyone capable of performing a search. Try and find an article spinner, content scraper or forum posting bot a few years go and you would need the right connections and the right money. These days, just mention Black-Hat and the options are endless, often even free.

The Black-Hat world is opening up , and it’s the Black-haters themselves that are unlocking the doors.

Unlike hackers – who are motivated by the urge to crack because they can – Black-Hat internet marketers are motivated by profit, and a quick and sizable profit is the ultimate goal.

As the Black-Hat world opens up, however, the techniques they employ are being exposed for all to see, use, and, most significantly, abused. A credit card in the right places and you can grab a large handful of the tools you need to make a fat pile of cash on the internet selling other people’s products to people who don’t really need them. And that’s the god side of Black-Hat which can stretch from merely deceptive search engine manipulation techniques to all-out financial scams.

The result of all this is two-fold:

First, the tools are being employed by an ever increasing army of people ho don’t know how to use them and so are exposing themselves and their techniques just as reliably as i they had posted their web address on a billboard over the road from Google HQ.
Secondly, the saturation swamps the dedicated Black-Hat pro in sheer numbers. They have discovered the need to dig deeper and deeper to uncover the profit opportunities that once lay scattered about their feet.

Switching tactics

With hard profits more difficult to prise out of the internet, Black-Hatters are evolving from exploiting the search engines to exploiting the masses of would-be internet marketers hunting for a quick buck.

More and more closed, limited communities are opening up, where for a monthly fee anyone can learn the real secrets of Black-hat internet marketing. Tools to beat the search engines, cheat the affiliate networks, and spam every bog and forum on the planet. are available with the tempting promise that only 100 people will be permitted access to the treasured secrets.

You have to admire the Black-Hatter’s resourcefulness. But when one’s method of paying the mortgage is based on discovering and exploiting online opportunities, the switch is not surprising.

Google loves Black-Hat

You may be surprised to learn that Google is perfectly happy to permit pay per click advertisers blatantly promoting Black-Hat techniques to beat Google at their on game. But consider that th more information is available in the public domain, the easier it will be to understand and subvert the techniques being employed.

Is there still money in Black-Hat marketing and SEO?

I am sure there is. No matter how the space evolves, so the techniques, opportunities, and secrets also evolve. Spend any amount of time in a Black-Hat discussion forum and you will see ideas, techniques, scams, downright fraud being described for the purposes of making a fast buck. It is a very mucky world.

One factor that never ceases to be consistent is that big bucks are only achievable with big effort. Most apprentice Black-Hats enter the space searching for those tens of thousands for just an hour’s investment. But big, consistent earnings take as much effort – I believe more in the long run because of the additional energy exerted in need to cover one’s tracks – as any more legitimate techniques for online business.

Overcoming duplicate content filters – back-links are everything

ndixon — Mon, 29 Dec 2008 11:34:39 +0000

Now I bet you are wondering “Why on earth would you want to duplicate content that already exists?” Well, putting the darker flavours of SEO/SEM to one side, many sites in the web2.0 world aggregate content legitimately scraped from other sites. So to understand how the duplicate content filter can be overcome, I thought a little experiment was in order.

The Duplicate Content Filter

Google dislikes presenting multiple search results that contain the same content. Instead, it decides which is the most authoritative original source of that content from all the duplicates it has in its index, and presents just that one. The rest are filtered into the supplementary index – all that extra content you can see when you see something like this at the end of your search results:

In order to show you the most relevant results, we have omitted some entries very similar to the 1 already displayed.
If you like, you can repeat the search with the omitted results included.

An experiment to become the authority

I wanted to test the power of backlinks as an indicator of authority above all else. The outline of the experiment is straightforward:

Create a brand new website
Fill it with content that already exists on other, more established websites
Create back-links pointing to it
Do not market the site in any other way

I am not going to link to the site itself here, as I am now isolating it from normal, organic link targets to perform another experiment.

Content for the site was selected from some freely available PLR (Public Label Rights) articles which can be legally reproduced. Such articles are also generally already published elsewhere. In my case, a search for specific chunks of article texts showed most articles had already been published across 6-10 other websites, some new, some quite established.

Getting indexed

Adding a link to the footer of a very healthy blog got the brand new domain added to the Google index within 24 hours.

Building back-links

In addition to the main purpose of this test, I also used it to try out link building service Linkvana. A full review of my experience with Linkvana will be here soon, but in a nutsheell, it provides the ability to create unique backlinks from a plethora of specially managed blogs, but without the potentially damaging drawbacks of usinga link-farm.

Over a period of three weeks I used Linkvana exclusively to create just 15 back-links into the new content, both deep-linking and to the home page.

The outcome

After just a week of link-building, searching for specific chunks of my published PLR text returned my site at the top of the search results, with all the other pages containing the same PLR article, pushed into the supplemental index as duplicate content. This despite all the other sites having the advantage of greater domain age and having already published that content.

The conclusion

Duplicate content is one of the most discussed, and misunderstood, aspects of Google’s search alorhythm, but can be overcome with pure link bulding.

The number of quality links pointing at duplicate content, seems to be the primary metric for assessing that site’s authority. Of course, this assumption must be tempered against any existing authority held by other sites with the same content.

PageRank, influence, and the silo

ndixon — Tue, 16 Dec 2008 07:50:33 +0000

We are, of course, talking about Google toolbar Page Rank. Either obsess over it or regard it with indifference, it is the only bit of clear feedback Google provides about your site that has some level of stability (though merely because it only updates every quarter) and to an extent, predictability.

In a nutshell, just to catch you up to speed, PageRank has no direct relationship with how or where your site appears in Google’s search results. But that does not mean it has no value.

A web page temperature guage

PageRank (PR) has a direct association with the number of incoming links to your webpage, and to some extent is also affected by the number of outgoing links from that webpage. Incoming links increase PR, while outgoing “leak” it.

One of the most important factors in gaining search result visibility is the number of incoming links to a page from elsewhere. See the connection? If your PR is increasing, that’s a good indicator of the general health of your site or page. If it is decreasing, get out there and build more links!

Page rank sculpting and the passing of influence

With the number and potency of incoming links as important as they (currently) are, it is easy to see how PR can be considered an indicator of influence and authority. High PR pages “pass on” PR to those pages into which they link. This is easily tested: have several high PR pages point to a low PR page – using your selected keywords in the correct manner, of course – and watch not only that page’s PR increase, but also watch it climb the SERPs. You may not achieve the ultimate goal, but you will see positive movement.

With a large site containing many pages that have their own PR, it is possible to “push” this PR influence to target pages, thus raising thir SERPs visibility purely with internal linking. Internal linking… now is not that a great deal easier than external?

Creating silos, or living in the PR culdesac

PR “leaks” out to pages when you link to them. The rate of leakage is low so there is no fear of completely draining a page’s PR. Controlling the “Flow” of influence within a website is a very potent tool in building higher SERP visibility.

Silo pages take all the incoming PR they can get their grubby little hands on and hold it tight, not letting a single drop seep out particularly outside the site, but also within it. To build the influence and authority of a particular page on your site, have many incoming links to it while reducing its outgoing links to an absolute minimum. If you do need to link out, use the “nofollow” attribute to prevent passing (leaking) the PR influence.

There is some discussion around fears that silo pages may appear to be unnatural entities to a search engine and may be penalised as a result. In experiments with directly managing the PR flow around a site,I have not personally seen this, in fact, very much the opposite.

Do not keep it all in the family

Internal linking to control the PR flow around a site is increasingly important to ensure the pages you want to float higher in the SERPs, do. But it must never be an exclusive linking technique.

Incoming links from elsewhere remain vital, particularly from unconnected sites with highly relevant content – and a big, juicy PR to boot. Such links are hard to acquire, but when combined with strong management or internal linking, may land you within striking distance of the top.

Google Searchwiki – blackhat SEOs rejoice

ndixon — Wed, 10 Dec 2008 17:38:29 +0000

In Techcrunch’s report today regarding the planned inclusion of an “off button” for Serchwiki – I really don’t know why so many people got so hot under the collar over this point – they also mentioned a direct quote from Google’s VP of Search Product and User Experience Marissa Mayer who was speaking at this week’s Le Web in Paris:

in the future it’s likely Google will use the data to at least make obvious changes. An example is if “thousands of people” were to knock a search result off a search page, they’d be likely to make a change

Just gather enough friends

I can see the blackhat SEO community already grouping together to “bump-up” each others’ sites in the search results. Though I doubt it is just blackhatters planning such activity. Watch the callout for reciprocated votes spreading like wildfire through any tech or marketing oriented social network! If Google have made buying links unacceptable behaviour, what about buying votes?

Just like any form community voting, it is open to abuse and manipulation. One might consider that Google are intelligent enough to measure such voting effectively, but it will instantly become another weapon in the SEO’s search manipulation arsenal.

Weakening SEOs power

Searchwiki itself, in my opinion, is a direct stab at the SEO industry to weaken its stronghold on the way search results are presented to the searcher. Personalised search results where the individual is in control, removes the effectiveness of SEO. One cannot determine with confidence what search results an individual might be seeing for a particular keyword, because Searchwiki allows a user to place whatever search results they wish at the top positions for a particular search.

If uptake of Searchwiki is broad enough, SEO will need to become increasingly focused on long-tail search terms – those that are searched less often and I would suggest are less likely to see the searcher feeling the need to manipulate their results. Highly competitive terms – where a searcher might be more likely to keep searching through page after page of results to find the one they want – are more likely to be clicked to the top of the list in Searchwiki.

Swings and roundabouts

There seem to be some opportunity here on all sides. Users may find their search results more acceptable for terms they regularly hunt, while pure on and off-page optimisation could become less important.

I wonder when we might begin seeing page title and description content on search results specifically designed to encourage a searcher to click their little Searchwiki icon to move that result to the top..?

Google teaches SEO

ndixon — Sat, 29 Nov 2008 08:36:16 +0000

Google has a number of stock answers for SEO related questions, most of which revolve around advice to build content for users, that’s easy to navigate, is relevant and accessible. Makes a great deal of sense when you think about it.

They want to reward sites that do well in providing quality, unique content for their visitors. But with the growth of the web, only an automated solution is viable, and automation leads to algorithms, variables, and ultimately, attempts to circumvent and/or manipulate the technology.

Out of the Horse’s mouth

I get asked daily to give an overview of the fundamental basics of SEO. What the majority of these questioners want is the one, secret technique – because there must be one, right? – that pro SEO’ers use to get top listing in Google’s SERPs. They are usually disappointed with my answers, which allude to lots of effort creating and marketing original content.

With the release of Google’s SEO guide, I just point them at its pages. You can download here (pdf).

Who better to tell you what they want from you?

Google are in the best position to let you know what they want to see on (and off) a web page to consider it for a good ranking in search results. If you know anything about SEO, then there really are no surprises within these pages. Even SEO pros should give it a read or two to remind ourselves of the basics and most important elements in ranking well for Google.

Hurting the professional SEO business?

What could be the motive of Google releasing this document, could they be trying to hurt the professional SEO business? It’s not much of a stretch of the imagination to consider, as a friend recently suggested to me, that by openly spreading around what are the fundamental principles of Google SEO, it weakens the efforts of professional SEO’s as now anyone can run with these techniques.

But the “secrets” of SEO are no secrets at all. The knowledge of what to do is freely available across the web and though no professionals cling to the idea that professional SEOs hold the most powerful techniques close to their chests, this simply in not true.

In fact, the secret to SEO can be boiled down to two words: focused perseverance.
Success with achieving top search engine positions, and the resulting traffic if you have targeted your audience correctly, is a matter of time, effort, and slowly inching your way up the SERPs. There simply are no shortcuts.

The bottom line of this is if you follow the directions offered in the Google SEO document, you stand a very good chance of ranking well on the Google SERPs.

It’s that “hard” part that puts people off

The SEO industry can rest comfortably in their beds. My personal experience is that no matter how much information you offer people, it’s the “hard work” element that stalls their progress – not necessarily that they do not want to engage in the time investment, but that they generally simply do not have the time to invest.

It takes time to see the results of day-after-day grinding at the various on and off page techniques, and the tasks do seem worthless, particularly when ranks appear to be slipping (not uncommon at the immediate outset of strategic SEO activity).

Setting out on a strategic SEO plan, one must look six months or more ahead before seeing strong, stable results of directed effort. Now and then results appear sooner. But more often than not, rapid results fluctuate week after week; stable, predictable results take time.

Google pagerank update 27 September 2008

ndixon — Sun, 28 Sep 2008 09:45:15 +0000

Page Rank is, of course, irrelevant to your actual SERPs ranking, but is an indicator of the general health of your site (if your rank is increasing) or ill-health (if it is decreasing). Exact details on how Page Rank is calculated remains a secret that Google keeps close to its chest, but it is certainly related to the number of links pointed at a website, and the quality (influence) of those links. Worth keeping an eye on as an indicator that your site is gaining strength.