The opening salvo of a sales pitch for a new FTP application that is part of the current MacHeist bundle offer, begins by criticising users for continuing with a tried, tested, trusted product that I have used for many years:

Are you still using Transmit, an app that dates all the way back to MacOS 8? Well, if so, then it’s time to make room for the new kid in town… Flow

That’s right, they open their sales pitch in that way, and continue with snipes at Transmit with reference to Transmit’s very basic text editor:

And we’re not talking about a gimped editor…

Panic’s Transmit is one of those apps that I never hesitate to upgrade. Along with hardened, mature tools like BBEDit, they have remained at the core of my day to day development work (BBEdit is even set to auto boot when my Mac starts).

Am I behind the times in my choice of tool? When I find a new tool that stands the test against those I trust, I’ll make the switch. But strip away the bells and whistles of most new tools and I generally find poor substitutes for those that already enable me to do my job.

Of course, tool choice is a personal one and there are many who would never make the same choices as me. But to go so far as to be critical of a user’s choice is an immature, insecure and ignorant form of marketing. Not the kind of company to which I want to make a professional commitment.

Right now we are working on upgrading all our themes to new Magento release. The good news is that all upgraded templates will be based on our new improved Magento template framework with new improved seo code. Most popular themes will be available till the end of the week. We are planning to finish upgrading all themes within 3 weeks.

Good to see they are on the case, and a three week wait is not unreasonable considering the number of templates they have made available.

Magento templates are not independent

Template providers for Magento tend to create their products using parts of the default Magento template to supplement their own code. Magento therefore uses parts of the default template, combined with the custom template, to build the site.

This method of easing template creation by not having to reproduce every last element from the default template is a cost-effective means of developing, and consequently brings the cost of each template down. But inevitably leaves developers at the mercy of the Magento developers. If they change the default template structure, it could easily break the third party templates.

This is no criticism of TemplatesMaster, this method of template creation is commonplace and quite acceptable, particularly for complex template structures such as Magento’s. But it does demonstrate how we remain at the mercy of core product developers of large, commercial products such as Magento. Their product development is commercially motivated (as it should be), but this can lead to the broader community of developers having to hop around when things change.

MTECH411 – for 20% off £35/$50 or more

MTECH412 – gets you a .com domain name with 30% off, including renewals

MTECH125 – 25% off $100 / £65 or more (expiration extended indefinitely!)

MINOR20H1 – for 20% off 1, 2 or 3 year hosting packages

MINORSSL – gives you a standard SSL Certificate for a mere $12.99

How do you use these?
Just pop over to GoDaddy.com, drop some items in your shopping cart, and insert the most appropriate code above into the clearly visible box in your shopping cart, just before you complete checkout.

Older Codes (expiring soon)

MTECH8 - gets you 10% off any order
MTECH9 - for 20% off £35/$50 or more
MTECH10 - gets you a .com domain name with 30% off, including renewals
MTECH25 - 25% off $100 / £65 or more (expiration extended indefinitely!)
MINOR20H - for 20% off 1, 2 or 3 year hosting packages
MINORSSL - gives you a standard SSL Certificate for a mere $12.99

Magentoexpert has the solution which requires a couple of lines added to /cron.php:

Magento uses a combination of xml layout directives and php/html (.phtml) template files to build the front-end interface (storefront) of a web shop. Fundamental changes to the look and feel of the store can and should be made directly in these files, all of which reside in the directory:

/app/design/frontend/default/

Smaller scale tweaks to page content can also be added from within the admin interface. Customising the content of an individual category page, for example, is achieved through adding xml code segments to the Custom Design tab of a category.

The difficulty we found was that any customisations for an individual category were added to the page in addition to the standard category layout definitions, rather than replace them with a new, unique set of directives. Resulting, in our case, to duplicated elements on the page.

Solution 1: Create an additional template

The admin allows selection of an entirely different template for an individual category. A powerful option, that’s for sure, but hugely impractical for ongoing maintenance. Multiple templates means multiple changes to layout, phtml and possibly css files if any tweaks are required in the future.

Solution 2: Add category-specific custom code to the master layout xml file

The default category layouts are defined in /app/design/frontend/default/-your-template-/layout/catelog.xml (just look for the <catalog_category_default> element).

To add code for a specific category, all one needs to do is create a new element containing the internal ID of that category (changing XX to the category’s internal ID):

<CATEGORY_XX>

  ....

</CATEGORY_XX>

A good solution which fails only in that it becomes less portable. These xml files are designed to handle the template as a whole. Having specific category designations here mean the template can only be used for a single store with the correct category ID values.

Solution 3: Add custom xml code to the category via the Administration interface

The most practical solution, without a doubt. Here the custom code affects only the single category (and its child-categories should that be selected as an option).

However, both Solution 2 and Solution 3 must solve the problem that additional xml is added to the default xml. In our case, however, we must override the default settings to avoid duplication.

The unsetChild method

Thankfully, Magento provides a means of overriding the default elements, as defined in the layout template, to be replaced by new elements. You can see the unsetChild methods implimented in the screenshot here.

unsetChild

The method tells magento to remove whatever definitions that have already been created for the named elements. Usually, these would have been defined in the main template xml files. It does not remove the named elements themselves, so you are then free to redefine those elements as you wish.

In our case, we wanted to replace the category.products, product_list, and product_list_toolbar elements to avoid duplicating the product grid.

Once unsetChild had been called, we were free to add our customised grid definition below it.

The results are a default, 3-column, 9 product grid for most of the catalog, but a 4-column, 12 product grid for some individual category pages.

The remove method

Magento also provides the ability to entirely remove elements from the xml definition.

<remove name="xxxx" />

This completely removes the defined element, by name. Inappropriate in our case, because it does not provide the option of re-inserting a new version of the element into the page; it removes that element name entirely.

Our web search (the well-known search at Google.com that hundreds of millions of people use each day) disregards keyword metatags completely. They simply don’t have any effect in our search ranking at present.

So there it is, no more wondering, no more endless speculation and scrabbling for evidence of whatever side of the fence one might sit.

Could this be a hint of more, clear-cut statements like this from Google?
I doubt it. The air of mystery works very much to their advantage and keeps the SEO world running rings around itself drying to second guess Google’s technology.

What it does signify is a move away from deliberately inserted page elements to a more whole-page approach, taking the real, visible content to determine page context and authority – in addition to many other factors.

Rapid indexing of new and relevant content seems to be of increasing importance to Google and it is this that might provide the difference in search results.

Exactly how Caffeine will integrate most recently indexed content amongst the aged, authoritative content previously given prominence in Google SERPs remains to be seen. We are likely to see results shifting rather rapidly. Hot topics will float to the top of listings, but only temporarily, sinking almost as rapidly to blend with the more stable results. But that is pure speculation at this time.

Is it all about twitter-fear?

Earlier this year, twitter began aligning itself as the future of immediate search. Google got understandably nervous, and Caffeine might be the result. I doubt Google will replace the place twitter has become to so many of us as a place to get literally up to the minute news about important events.

Though still nowhere near as immediate as searching the contents of the twitter timeline, Caffeine does index and present relevant results much sooner than Google before it.

How does this affect SEO?

The jury remains out on how this enhances or inhibits direct SEO activity. We are certainly expecting to see more flexible and dynamic results for many searches as new, fresh content is intermixed with the authoritative.

There are still apparent glitches in the system right now, so no Caffeine search results can be trusted as representative at this time.

MTECH410 10% discount on your entire order [copy code or use]

MTECH411 20% off your order over £35 (~$56) [copy code or use]

MTECH125 25% off your order over $100

MTECH412 30% off .com domain names (can also be used on renewals and multiple year registrations) [copy code or use]

MINOR20H1 gets 20% off web hosting [copy code or use]

MINORSSL standard SSL certificates for $12.99 [copy code or use]

Click any of the above codes to have the discount automatically applied. Your discount value will be clearly displayed on your shopping cart at checkout. You may try different codes to see which one provides the best discount, but only one code can be applied to a single order.

GoDaddy codes are used at checkout, entering them into your shopping cart before you complete your order. There are no limits on the number of times you can use them, but some products and services are exempt form the discounts – in particular, items that are already on discount promotion on the GoDaddy site.

If you already have them, these older codes are no longer valid:
10% MTECH8
20% MTECH9
30$ .com’s MTECH10
25% MTECH25
10% discount MTECH1
20% off orders over £35 (~$56) MTECH2
30% off .com domain names & renewals MTECH330

For example, my previous post can be reached by the full URL, but also (because of the Wordpress system defaults) via

http://seowebmonkey.com/?p=113

If someone decides to link to my post using that URL,  it might therefore also enter the search index in addition to the full version – thus duplicate content.

Placing a rel=canonical instruction in your page’s header can resolve this problem before it occurs. It tells the search engine which URL to use as the definitive (canonical) URL for that page of content, regardless of what was used to get there.

Here’s an example for the above page:

<link rel="canonical" href="http://seowebmonkey.com/duplicate-content-web-applications/"/>

This sits within the <head> section of your page’s html.

This link-tag is supported by Google, Yahoo, Ask.com and Microsoft Live Search.

Does it work?

Google describes its support of rel=canonical as a “hint”. This means it will use the information to determine a canonical URL, but reserves the right to do what it wants when it feels like it. This seems to be a way to cover for errors that slip through the net. Search engines are rarely predictable, and no single method should be trusted in avoiding duplicate content.

For all content management systems in particular, this is an essential addition to the page output. How rapidly Google will change any existing duplicative content URLs that are already in its index is yet to be clearly determined, and this alone will not enable webmasters to explicitly request removal of duplicative URLs via the Google Webmaster Tools interface.

Direct removal of pages from Google’s search index will be covered in my next post.

Defining duplicate content

A search engine considers every unique URL to be the location of a single web page. It learns about such URLs by following links. Duplicate content occurs when a search engine follows more than one unique URLs that take it to a page of predominantly similar content – these pages do not need to be identical, just very similar (a defined ratio is something of a holy grail for optimisers and not clearly established).

Why is this important? A search engine will not display more than two pages from a website for a particular search. It makes an automated decision of which pages are the authoritative or originating version of that content, tucking the rest away at the end of the results: the Supplementary Index.

The damage to SEO comes from a number of factors, the two main causes being:

1. The page you most want to surface may become inadvertently pushed into the Supplementary Index, thus giving searchers an inferior, poorly converting page to visit.
2. A major factor in search result visibility is the number (and quality) of links pointing to a page, but multiple URLs to a page frequently dilutes the combined effectiveness of back-links as the links point to different unique URLs. 

From my direct experience with sites over the past year in particular, duplicate content pages result in a cumulative deterioration of the entire site’s visibility in SERPs (Search Engine Result Pages). 

If you have a site that is failing to rank well for even the least competitive search terms, such duplicate content is likely to be a contributing factor, and you will struggle to establish a firm footing until this problem is resolved.

Spotting the damage

Discovering duplicate content is relatively straightforward. Perform a search for your website on Google, like this:

site:yourdomain.com keyword

where yourdomain.com is your website’s domain, and keyword any terms for which you are trying to be visible. Run down to the very end of the results and if you see a message like this:

you have duplicate content!

The cause

Content Management

If you have a duplicate content issue, then I will bet your website runs on some form of content management. Blog applications are notoriously prone to these problems, with supplementary listing pages such as category summaries, tag summaries, archive lists, and search results, all in danger of generating pages with very similar content.

In addition, some content systems do not chose a single means of generating a URL for a item of content, linking to pages and posts in slightly different ways from different parts of the system. They may also fail to redirect all URLs to the canonical URL for that page or post.

Identical meta data

In addition, it is very common to come across sites with many – sometimes even all – their pages with identical titles and META descriptions . This is particularly prevalent once again with blogs, and also in the small business space where a company feels it must apply its company name and corporate blurb at the top of every page, thus not accurately reflecting that page’s actual content.

Page titles are very important in establishing the context of a page, and must be unique in order for the search engine to properly understand the page content.

Too little unique textual content

Pages with little unique text content can become regarded as duplicative because the majority of the content there is similar to everywhere else on the site. In these sparse content examples, the site-wide navigation, footer information, and other generic text, can form the majority of the textual content.

Search, archive and summary pages

As mentioned above, pages that summarise and list snippets of other content can easily appear to a search engine to be very similar but have unique URLs.

www and non-www domains

This one often surprises web developers, but www.yoursite.com is, to a search engine, a different website to yoursite.com. This means that if all your content is reachable via both those versions, your entire site is seen as being a duplicate!

It does not matter which you choose, but have one of those URL versions permanently redirected to the other.

Repairing the damage

Now that I have covered most of the indicators and causes, how about a means of fixing the problem? Watch out in a few days for specific techniques to repair and remove duplicated content within search engines.