SEO WebMonkey » Optimisation

rel=canonical: trying to get Google to understand

ndixon — Tue, 14 Apr 2009 20:49:41 +0000

Back in February, Google announced it had begun supporting the rel=canonical hint in determining the definitive URL for an item of content. This sounds like a one-stop solution to all our duplicate content problems.

For example, my previous post can be reached by the full URL, but also (because of the Wordpress system defaults) via

http://seowebmonkey.com/?p=113

If someone decides to link to my post using that URL, it might therefore also enter the search index in addition to the full version – thus duplicate content.

Placing a rel=canonical instruction in your page’s header can resolve this problem before it occurs. It tells the search engine which URL to use as the definitive (canonical) URL for that page of content, regardless of what was used to get there.

Here’s an example for the above page:

This sits within the section of your page’s html.

This link-tag is supported by Google, Yahoo, Ask.com and Microsoft Live Search.

Does it work?

Google describes its support of rel=canonical as a “hint”. This means it will use the information to determine a canonical URL, but reserves the right to do what it wants when it feels like it. This seems to be a way to cover for errors that slip through the net. Search engines are rarely predictable, and no single method should be trusted in avoiding duplicate content.

For all content management systems in particular, this is an essential addition to the page output. How rapidly Google will change any existing duplicative content URLs that are already in its index is yet to be clearly determined, and this alone will not enable webmasters to explicitly request removal of duplicative URLs via the Google Webmaster Tools interface.

Direct removal of pages from Google’s search index will be covered in my next post.

Duplicate content in web applications

ndixon — Fri, 10 Apr 2009 10:37:05 +0000

When first approaching a new SEO project, one of the very first research tasks is to assess the current status of the client site within the search engines – particularly, of course, Google. Right at the top of such tasks is a hunt for duplicate content and consequently, pages that have been filtered into the Supplementary Index.

Defining duplicate content

A search engine considers every unique URL to be the location of a single web page. It learns about such URLs by following links. Duplicate content occurs when a search engine follows more than one unique URLs that take it to a page of predominantly similar content – these pages do not need to be identical, just very similar (a defined ratio is something of a holy grail for optimisers and not clearly established).

Why is this important? A search engine will not display more than two pages from a website for a particular search. It makes an automated decision of which pages are the authoritative or originating version of that content, tucking the rest away at the end of the results: the Supplementary Index.

The damage to SEO comes from a number of factors, the two main causes being:

The page you most want to surface may become inadvertently pushed into the Supplementary Index, thus giving searchers an inferior, poorly converting page to visit.
A major factor in search result visibility is the number (and quality) of links pointing to a page, but multiple URLs to a page frequently dilutes the combined effectiveness of back-links as the links point to different unique URLs.

From my direct experience with sites over the past year in particular, duplicate content pages result in a cumulative deterioration of the entire site’s visibility in SERPs (Search Engine Result Pages).

If you have a site that is failing to rank well for even the least competitive search terms, such duplicate content is likely to be a contributing factor, and you will struggle to establish a firm footing until this problem is resolved.

Spotting the damage

Discovering duplicate content is relatively straightforward. Perform a search for your website on Google, like this:

site:yourdomain.com keyword

where yourdomain.com is your website’s domain, and keyword any terms for which you are trying to be visible. Run down to the very end of the results and if you see a message like this:

you have duplicate content!

The cause

Content Management

If you have a duplicate content issue, then I will bet your website runs on some form of content management. Blog applications are notoriously prone to these problems, with supplementary listing pages such as category summaries, tag summaries, archive lists, and search results, all in danger of generating pages with very similar content.

In addition, some content systems do not chose a single means of generating a URL for a item of content, linking to pages and posts in slightly different ways from different parts of the system. They may also fail to redirect all URLs to the canonical URL for that page or post.

Identical meta data

In addition, it is very common to come across sites with many – sometimes even all – their pages with identical titles and META descriptions . This is particularly prevalent once again with blogs, and also in the small business space where a company feels it must apply its company name and corporate blurb at the top of every page, thus not accurately reflecting that page’s actual content.

Page titles are very important in establishing the context of a page, and must be unique in order for the search engine to properly understand the page content.

Too little unique textual content

Pages with little unique text content can become regarded as duplicative because the majority of the content there is similar to everywhere else on the site. In these sparse content examples, the site-wide navigation, footer information, and other generic text, can form the majority of the textual content.

Search, archive and summary pages

As mentioned above, pages that summarise and list snippets of other content can easily appear to a search engine to be very similar but have unique URLs.

www and non-www domains

This one often surprises web developers, but www.yoursite.com is, to a search engine, a different website to yoursite.com. This means that if all your content is reachable via both those versions, your entire site is seen as being a duplicate!

It does not matter which you choose, but have one of those URL versions permanently redirected to the other.

Repairing the damage

Now that I have covered most of the indicators and causes, how about a means of fixing the problem? Watch out in a few days for specific techniques to repair and remove duplicated content within search engines.

Stumbleupon Firefox bar gets an update

ndixon — Thu, 27 Nov 2008 17:38:55 +0000

If you are not familiar with the service, Stumbleupon is a service where a connected network of users submit their favourite web content which is then fed to other users – based on their subject preferences – which they can then vote with a thumbs up, or a thumbs down. It’s a great way to discover new stuff based on popularity amongst the user network.

The new toolbar in particular offers greater control over what you might see in the future by not only enabling you to give a site a thumbs down, but also to give Stumbleupon some idea why it’s a thumbs down. Options are: “Not-for-me” (the standard thumbs down), “Stumbled this before”, “Too much like this lately”. There’s also a command to block a specific site from being presented in the future.

These added communications smack of Stumbleupon wanting a higher level of feedback to improve the web content they feed users. That’s a very good thing as it can sometimes be frustrating when it appears the algorithm is getting your selections all wrong.

Around ten other fixes and enhancement have also been included. You can see the changelog here.

The importance of on-page keyword focus

ndixon — Thu, 27 Nov 2008 09:45:17 +0000

Doorway pages existed because of the diverse way in which the various search engines handled their ranking algorithms differently. On-page optimisation would be tweaked slightly differently for each engine, for each keyword, and so multiple pages with similar content appeared on a site, each one attempting to flirt with a specific search engine. Pages even went so far as to include the search engine name in the page URL!

If you are still tempted to use such a technique, either for attracting different search engines, appearing to be an authority site on a particular keyword, or split testing content and on-page SEO, there are two primary negative aspects of multiple doorway pages that you must consider.

A single target for your keyword

Let’s imagine a scenario where both pageA and pageB are optimised for a particular keyword, each with slightly different content, but very similar URLs, page titles, and other on-page optimised elements.

Search engines will discover and index both pages, but do not generally want to provide more than one result to the searcher, particularly when the content on both is very similar.

They need to make a decision about which one is the originating, authoritative page for that particular keyword. Due to the number of ranking factors used in search engine calculations, they may select the wrong one!

Duplicate content

Much is discussed – and misunderstood – regarding duplicate content, but the key issue is similar to above. If two pages (defined as two web pages reachable by unique URLs) have substantially similar or identical content, a search engine will not display both pages in results for a particular search. It will decide on the authoritative page for that content and filter the other out of the search results.

How do you target more than one keyword on a single page?

Simply: research and strategy.

Each individual page must be focused around a specific set of keywords, with a single keyword – or “key-phrase” of course – as the primary focus. Keyword selection is a whole other discussion, but it is important to isolate what is th most important keyword for that page and optimise for that.

The additional keywords in the selection should then be used within the page content as they will be seen as associated terms and will be found in context – an increasingly important consideration.

Getting the balance right

Keyword research is everything in determining the most effective focus for a page. Balancing potential traffic volume against how difficult it might be to rank for a keyword is critical in determining the choice of primary keyword over the additional, secondary terms.

There is no hard and fast rule; it all depends on the broader range of of keywords within your niche – the “keyword space”.

For example, KeywordA may potentially generate the most traffic, but could also be the term for which most of your competition are fighting. Ranking well for KeywordB might prove far more lucrative in traffic generation than ranking poorly for KeywordA – this is very often the case where KeywordA is a broad, generic term. Because KeywordA and KeywordB are associated with each other, ranking well – becoming an authority – for KeywordB may also help to drive some traffic for KeywordA.

Focus on the most effective

A single web page that is optimised around a single, primary target keyword, will stand a much better chance of becoming an authority for that keyword.

Choose carefully how to split primary keywords between different pages.

Take care in not focusing more than one page on very similar terms.

Be strategic in page/keyword selection so that each individual page contains absolute minimal references to keywords you are targeting on other pages.

Can you please decrease my SERPs ranking?

ndixon — Fri, 24 Oct 2008 17:12:55 +0000

A site I help to SEO has a number of pages ranking well in the Google SERPs (Search Engine Result Pages) for some specifc, targeted keyword phrases. They also, inadvertently, rank well for a number of other phrases that related specifically to brand names of the site’s clients. This competition for the brand names has created some issues and so I was asked to reduce the rank for those brand searches – without reducing the ranking for all the other target keyword phrases.

De-optimisation?

The first thought was to remove as many instance of the troublesome keywords from each of the ranking pages. The difficulty is that these brand names are legitimately used in the pages, and often are part of the main target key phrases. Removing them would reduce legibility and context of the page for visitors.

Reducing the on-page frequency of the keywords was certainly possible – if the resulting effects on SERPs was unknown – but backlinks were another matter. Incoming lnks to these pages also contained the troublesome keywords, and getting all these changed would be tricky and again damage legibility and relevance of the link to users.

Sometimes SEO does not have the answer

It took a non SEO person to come up with a very different solution to the problem. Stepping away from an SEO-focused solution, what if the search traffic generated from the troublesome branded searches was simply redirected to the client sites? Not SEO, but may resolve the problem.

Whether this becomes a viable solution or not, it was a big lesson in how focusing too narrowly on the problem, and forming early assumptions on where the solution might lie, clouded the creative, problem solving process. Sometimes, you just need to step away from SEO and broaden the possible solutions.