in SEO by Jeffrey_Smith

With SEO, sometimes you hear about duplicate content and duplicate content issues.  These “issues” are real and unless corrected, it is like splitting hairs and losing a large percentage of your potential SEO ranking factor within a website.

How To Avoid Duplicate Content

How To Avoid Duplicate Content

What are some of the types of duplicate content? And what can be done to manage, prevent or minimize their impact on rankings or site performance?

  • Canonical http:// or http://www variations of pages.
  • Shopping carts using hand me down descriptions from manufacturers.
  • Pages that use the same template with very little distinction from eachother,
  • Block level shingles (content copied from page to page or from other websites).

As you can determine from the brief list above, there are multiple species of duplicate content and each should be eliminated or managed from the level of your content management system, .htaccess file or through the uniqueness of content on each page.

Today we are only addressing the type created within your own website (as a result of content and indexing from lack of enough content) and ways to avoid or eliminate it.

We often emphasize the importance of uniqueness on a page by page level within a website. Having 100 pages and 800 of them having a 12% difference as a result of a different title, model#, photo and brief description doesn’t qualify as enough distinction significantly.

Common Duplicate Content Solutions

There are a couple of different things you can do to eliminate duplicate content, refine canonical issues or control which pages are indexed and which others are not.

1) Write enough content to shift the collective focus of the page.

2) Personalize data feeds or commonly used databases for makes and model#’s from mfg. for ecommerce products.

3) Add a custom field or segment on your ecommerce pages to create their own digital footprint.

4) Consider using meta-tags, or robots.txt to block or prevent indexing of extremely similar pages (which alleviates the penalty).

5) Use a canonical tag so that pages that are a linked like a daisy chain (or bread crumbs) all link back to the leader as the primary index page. This is very useful for deep categories where getting to the main category page is sufficient from an SEO perspective.

6) Personalize footer links to prevent diffusing the pages topic sitewide through “making each page similar”.

From the perspective of search engines, realize that if a page has nothing to contribute, if it already has 20 other pages from your website that have the same look, feel, code and context, you can’t expect them to index and waste space within their primary index on cookie-cutter pages.

To briefly address the points above:

For page to page level canonical issues use a 301 redirect by making adjustments to your .htaccess file (granted you are running a Linux server). More information on 301 redirects and how to implement them here from Taming the Beast.

Once fixed, depending on the type of site level duplicity, your link flow will consolidate within a site where you can then use internal linking to shore up segments of your website that may be depleted or starved for sufficient link flow.

Also, make sure if you have multiple versions of your homepage to consolidate them using the page level redirect method to one specific version http://www.yoursite.com/ versus http://www.yoursite.com/index.html or http://www.yoursite.com/default.htm |.aspx |.shtml | .cfm, etc.

Also, when linking to those pages (both within your own navigation and from other sites), make sure you use one consistent page / naming convention / URL.

Solutions for Multiple Pages with Similar Content

There are a few ways to remedy this. You can use a canonical tag to refer back to the original page or manage what gets indexed to avoid duplication.

If you have access to personalize the headers of the page, then you can implement a simple meta tag.

NOINDEX, FOLLOW – invites the spiders / user agents to crawl your page, but prevents that page from being indexed. Yet search engines will follow the links and include them for ranking factor or discover new pages based on the links present on the page.

This would be the preferred selection for ecommerce pages, if you wanted to emphasize the link back to the main category page and apply your SEO efforts to make the category your preferred landing page for rankings and traffic from natural search.

INDEX, NOFOLLOW – Tell search engine spiders / user agents to index the page, but not follow the links on the page to other pages or count them in their algorithm.

NOINDEX, NOFOLLOW – informs them, that you gave at the office and you would rather not be bothered. In other words, NOINDEX, NOFOLLOW means that the page will not be included in the search engine index and the links are not followed or treated with any significance.

The most common meta tag setting is INDEX, FOLLOW which is like leaving a plate of cookies out for Santa, it is the best way to ensure that permissions are granted and welcome them in to crawl, index, sort and earmark pages in their index for information retrieval. The same could be done at the server level with a robots.txt file as well. For more information on robots.txt follow the link.

If you have dynamically loading pages that insert tracking cookies, session ids, or other unique strings into the URL of your pages, then using the canonical tag is the best way to ensure your pages are not creating duplicity across your entire content management system.

A canonical tag is a tag that the three major search engines (Google, Yahoo and Bing) recently started to support that eliminates clutter in their index by not allowing multiple variations with slight differences to appear as more than one page.

By adding the canonical tag, it ensures that much like highlander, there can be only one, and the main page gets the credit, rankings and authority. For more information on canonical tags and how to implement them, follow the link.

Other Methods to Eliminate Duplicate Content

Adding unique content for pages that requires more emphasis, this includes rewriting your top selling products if you are using a shopping cart to avoid duplicate shingles from other sites selling the same products.

At first, you think, who has the time to rewrite hundreds or thousands of products? The answer, the companies that know the value of unique content, authority and what it means to rank heads and shoulders above the competition with a fraction of the backlinks or the SEO required to propel them there.

Every word you add to a document, it changes the landscape of that page, the more unique it is and the more frequently it references the primary keywords the page targets, the easier it is to separate that page from other pages in the template.

Navigation is also another area of concern for nested pages as they can cripple your near duplicate pages by convoluting the page with numerous (me too) links, like the other pages and completely diffuse the UNIQUENESS of the page.

You can tuck navigation behind an IFRAME, use of nofollow, or redirecting the navigation links from behind a /CGI-BIN/ script where all of the redirects happen in a page not followed by robots or blocked by robots.txt.

However, despite if they are followed or not, the presence of all of those links creates a unique page impression that for lack of better terms skews your s.k.u’s (stock kept units) by slightly obscuring their focus.

The advantage to the CGI-BIN method is, you could then insert includes or use contextual links (links in the body copy of the document) to connect relevant pages using keyword-rich anchor text to increase relevance and ranking position.

Footer links or breadcrumbs would then carry more weight in the algorithm since every link is not WIDE OPEN on your pages. Although that falls into the category of page sculpting, it is still important to understand the premise behind the topic.

Another Advanced SEO Tactic

1)      Create a custom text field somewhere on the page through the content management system.

2)      Populate that content through an XML feed or pull data from a blog. The blog could be set to noindex, follow and link to the new destination landing page and the landing page showcases the feed data as its own.

3)      Or for a low tech solution, simply write 250-400 words on the topic and insert that into the custom text field.

As you can see, where there is a will, there is a way. The blog method mentioned allows you to easily implement a CMS, keep the content from showing up in search engines (on the blog itself) as you can set it to archive or noindex, then the aggregated data, which you created can be utilized to rank on the page of your choice.

You could also tag the content, and aggregate it through additional means like Google Base and other RSS based feeds for additional leverage.

The real take away here is, if you want to distinguish your website in search engines, be ready to take an extra step and implement solutions that  support granular changes to url naming conventions, on page content, link structure and CMS architecture. All of these changes are secondary to thoughtful planning and can be eliminated entirely from building it properly in the first place.

However, since that is not always an option, having the ability to implement distinct modifications and functions to produce optimal ranking factor means creating custom programs, hacks and workarounds to adapt, modify and optimize your CMS, add content on the fly, change or modify header information, rewrite titles and more.

In the event that a redesign is not applicable, there are plugins that can help to distinguish your money pages through using some of the tactics we discussed above.

Also, did I mention it was free? If you use the WordPress platform, then we invite you to download our SEO Ultimate Plugin which has the ability to rewrite titles, meta data, tag and archive pages, adjust headers, track 404 errors, alter the robots.txt or .htaccess file, toggle which modules are active and numerous other SEO features.

As stated above, it’s better to build it right the first time, but just in case you can’t there are always solutions for common SEO problems. 

Read More Related Posts
Are you building the right type of links?
While link building is an important for SEO, the million dollar question is, are your building the right type of links for your website? The truth is, not all links ...
READ MORE
How to Dethrone Competitors using SEMRush, Compete, SpyFu and Alexa.
Looking for a quick way to data mine competitors keywords to determine if they are worth the hassle to optimize? After reading this quick SEO tip and using a few ...
READ MORE
SEO Tips to Step Over Competitors
Contained in this post are practical SEO Tips to step over complacent competitors by targeting the least common denominator with SEO. This, in turn, churns lackluster contenders off the page ...
READ MORE
With SEO, Timing is Everything!
If you have ever wondered how a particular competitor used SEO to distance themselves distinctly in the top 3 positions for a competitive search term. How is it that they ...
READ MORE
Don't Disrupt Trusted Nodes of Relevance
Disrupting the trust a website has can be catastrophic to SEO. For example, if you have aged legacy content in a website that is indexed, then that page is contributing ...
READ MORE
How to Double Up on Traffic and Conversions
Where you rank matters! And whether or not your listing is above the fold in natural search results or on page two or beyond impacts the click through rate. The ...
READ MORE
What is the Purpose of SEO?
Life is about cause, effect and purpose, search engine optimization and using a search engine are no different. When you take the experience of a normal search into account, it ...
READ MORE
Getting Back to Page1
Have you ever checked Google and your most cherished SEO ranking was missing in action? Even more disturbing, there is some unforeseen new website or several new sites in your ...
READ MORE
SEO Tips To Optimize a New Website
What are the most critical SEO modifications to implement during the preliminary phases of optimizing a new website? Sitemaps and site architecture, links, naming conventions or D) all of the ...
READ MORE
Usability Solutions to SEO Challenges
A couple days ago Jeffrey Smith of SEO Design Solutions asked me to look at a client's eCommerce site to help solve an interesting problem. The site offered an popular ...
READ MORE
Stop Building Worthless Links!
SEO Tips to Find Competitors’ Keywords
SEO Tips to Step Over Competitors
With SEO, Timing is Everything!
SEO Tips for Retooling Legacy Content
How to Double Up on Traffic and Conversions
What is the Purpose of SEO?
SEO Tips for Getting Back to Page 1
SEO Tips to Optimize a New Website
Usability Solutions to SEO Challenges

About Jeffrey_Smith

In 2006, Jeffrey Smith founded SEO Design Solutions (An SEO Provider who now develops SEO Software for WordPress).

Jeffrey has actively been involved in internet marketing since 1995 and brings a wealth of collective experiences and marketing strategies to increase rankings, revenue and reach.

20 thoughts on “Duplicate Content and SEO
  1. dlgod says:

    very informative for me thanx for sharing…

  2. Your very welcome, thanks for visiting…

  3. Fundoo says:

    Definitely bookmarked! And thanks for sharing the article.very informative blog..wish u good luck

  4. John Vargo says:

    Great article on duplicate content and how to improve search engine optimization. Thanks for the SEO tips!

  5. YoVz says:

    For of all, thank you for such an informative article. I am a newbie at all this. I’m in the process of creating a website for my dad. I understand how you described to clean up duplicate pages on the site. However, my dad has several domains because he has a multifaceted business. Here is the example:
    Imagine a grocery store. They have a domain “grocerystore”. They also have these domains “apples”, “steak”, “bread”. Essentially, the websites would be almost identical, except for the landing page of each domain which discusses the term they searched on. However, the other pages discuss everything else in the grocery store. Is this considered duplicity? if so, how do I get around this? Thank you.

  6. Derek says:

    Hello. Great site full of useful information. I have a question and would greatly appreciate if you could help. Please note that I’m not very good with coding.

    I just added some footer pages to my site. I wanted the page names to be the same as the header page names. Should i “no follow” and “no index” these footer pages or just “no follow” them?

    I don’t want the search engines to punish me for having duplicate pages since both the header and footer pages carry the same name and link to the same page.

    Or should I just name the footer pages a little differently?

    Thanks for any help :)

  7. Mike says:

    Very useful post, thank you very much, will introduce these techniques on my Sky Cards website

  8. sima says:

    Hello
    I need help please,am a student,i was asked to create a micro questionnaire website…my problem is how to make sure that a survey our enters his data once and how to secure annonymity.Need help please.tnx

  9. edyzen says:

    morning sir, I have trouble with duplicate description each page. I use seo ultimate plugin. can you help me ? thanks

Comments are closed.