in SEO by

With SEO, sometimes you hear about duplicate content and duplicate content issues.  These “issues” are real and unless corrected, it is like splitting hairs and losing a large percentage of your potential SEO ranking factor within a website.

How To Avoid Duplicate Content

How To Avoid Duplicate Content

What are some of the types of duplicate content? And what can be done to manage, prevent or minimize their impact on rankings or site performance?

  • Canonical http:// or http://www variations of pages.
  • Shopping carts using hand me down descriptions from manufacturers.
  • Pages that use the same template with very little distinction from eachother,
  • Block level shingles (content copied from page to page or from other websites).

As you can determine from the brief list above, there are multiple species of duplicate content and each should be eliminated or managed from the level of your content management system, .htaccess file or through the uniqueness of content on each page.

Today we are only addressing the type created within your own website (as a result of content and indexing from lack of enough content) and ways to avoid or eliminate it.

We often emphasize the importance of uniqueness on a page by page level within a website. Having 100 pages and 800 of them having a 12% difference as a result of a different title, model#, photo and brief description doesn’t qualify as enough distinction significantly.

Common Duplicate Content Solutions

There are a couple of different things you can do to eliminate duplicate content, refine canonical issues or control which pages are indexed and which others are not.

1) Write enough content to shift the collective focus of the page.

2) Personalize data feeds or commonly used databases for makes and model#’s from mfg. for ecommerce products.

3) Add a custom field or segment on your ecommerce pages to create their own digital footprint.

4) Consider using meta-tags, or robots.txt to block or prevent indexing of extremely similar pages (which alleviates the penalty).

5) Use a canonical tag so that pages that are a linked like a daisy chain (or bread crumbs) all link back to the leader as the primary index page. This is very useful for deep categories where getting to the main category page is sufficient from an SEO perspective.

6) Personalize footer links to prevent diffusing the pages topic sitewide through “making each page similar”.

From the perspective of search engines, realize that if a page has nothing to contribute, if it already has 20 other pages from your website that have the same look, feel, code and context, you can’t expect them to index and waste space within their primary index on cookie-cutter pages.

To briefly address the points above:

For page to page level canonical issues use a 301 redirect by making adjustments to your .htaccess file (granted you are running a Linux server). More information on 301 redirects and how to implement them here from Taming the Beast.

Once fixed, depending on the type of site level duplicity, your link flow will consolidate within a site where you can then use internal linking to shore up segments of your website that may be depleted or starved for sufficient link flow.

Also, make sure if you have multiple versions of your homepage to consolidate them using the page level redirect method to one specific version http://www.yoursite.com/ versus http://www.yoursite.com/index.html or http://www.yoursite.com/default.htm |.aspx |.shtml | .cfm, etc.

Also, when linking to those pages (both within your own navigation and from other sites), make sure you use one consistent page / naming convention / URL.

Solutions for Multiple Pages with Similar Content

There are a few ways to remedy this. You can use a canonical tag to refer back to the original page or manage what gets indexed to avoid duplication.

If you have access to personalize the headers of the page, then you can implement a simple meta tag.

NOINDEX, FOLLOW – invites the spiders / user agents to crawl your page, but prevents that page from being indexed. Yet search engines will follow the links and include them for ranking factor or discover new pages based on the links present on the page.

This would be the preferred selection for ecommerce pages, if you wanted to emphasize the link back to the main category page and apply your SEO efforts to make the category your preferred landing page for rankings and traffic from natural search.

INDEX, NOFOLLOW – Tell search engine spiders / user agents to index the page, but not follow the links on the page to other pages or count them in their algorithm.

NOINDEX, NOFOLLOW – informs them, that you gave at the office and you would rather not be bothered. In other words, NOINDEX, NOFOLLOW means that the page will not be included in the search engine index and the links are not followed or treated with any significance.

The most common meta tag setting is INDEX, FOLLOW which is like leaving a plate of cookies out for Santa, it is the best way to ensure that permissions are granted and welcome them in to crawl, index, sort and earmark pages in their index for information retrieval. The same could be done at the server level with a robots.txt file as well. For more information on robots.txt follow the link.

If you have dynamically loading pages that insert tracking cookies, session ids, or other unique strings into the URL of your pages, then using the canonical tag is the best way to ensure your pages are not creating duplicity across your entire content management system.

A canonical tag is a tag that the three major search engines (Google, Yahoo and Bing) recently started to support that eliminates clutter in their index by not allowing multiple variations with slight differences to appear as more than one page.

By adding the canonical tag, it ensures that much like highlander, there can be only one, and the main page gets the credit, rankings and authority. For more information on canonical tags and how to implement them, follow the link.

Other Methods to Eliminate Duplicate Content

Adding unique content for pages that requires more emphasis, this includes rewriting your top selling products if you are using a shopping cart to avoid duplicate shingles from other sites selling the same products.

At first, you think, who has the time to rewrite hundreds or thousands of products? The answer, the companies that know the value of unique content, authority and what it means to rank heads and shoulders above the competition with a fraction of the backlinks or the SEO required to propel them there.

Every word you add to a document, it changes the landscape of that page, the more unique it is and the more frequently it references the primary keywords the page targets, the easier it is to separate that page from other pages in the template.

Navigation is also another area of concern for nested pages as they can cripple your near duplicate pages by convoluting the page with numerous (me too) links, like the other pages and completely diffuse the UNIQUENESS of the page.

You can tuck navigation behind an IFRAME, use of nofollow, or redirecting the navigation links from behind a /CGI-BIN/ script where all of the redirects happen in a page not followed by robots or blocked by robots.txt.

However, despite if they are followed or not, the presence of all of those links creates a unique page impression that for lack of better terms skews your s.k.u’s (stock kept units) by slightly obscuring their focus.

The advantage to the CGI-BIN method is, you could then insert includes or use contextual links (links in the body copy of the document) to connect relevant pages using keyword-rich anchor text to increase relevance and ranking position.

Footer links or breadcrumbs would then carry more weight in the algorithm since every link is not WIDE OPEN on your pages. Although that falls into the category of page sculpting, it is still important to understand the premise behind the topic.

Another Advanced SEO Tactic

1)      Create a custom text field somewhere on the page through the content management system.

2)      Populate that content through an XML feed or pull data from a blog. The blog could be set to noindex, follow and link to the new destination landing page and the landing page showcases the feed data as its own.

3)      Or for a low tech solution, simply write 250-400 words on the topic and insert that into the custom text field.

As you can see, where there is a will, there is a way. The blog method mentioned allows you to easily implement a CMS, keep the content from showing up in search engines (on the blog itself) as you can set it to archive or noindex, then the aggregated data, which you created can be utilized to rank on the page of your choice.

You could also tag the content, and aggregate it through additional means like Google Base and other RSS based feeds for additional leverage.

The real take away here is, if you want to distinguish your website in search engines, be ready to take an extra step and implement solutions that  support granular changes to url naming conventions, on page content, link structure and CMS architecture. All of these changes are secondary to thoughtful planning and can be eliminated entirely from building it properly in the first place.

However, since that is not always an option, having the ability to implement distinct modifications and functions to produce optimal ranking factor means creating custom programs, hacks and workarounds to adapt, modify and optimize your CMS, add content on the fly, change or modify header information, rewrite titles and more.

In the event that a redesign is not applicable, there are plugins that can help to distinguish your money pages through using some of the tactics we discussed above.

Also, did I mention it was free? If you use the WordPress platform, then we invite you to download our SEO Ultimate Plugin which has the ability to rewrite titles, meta data, tag and archive pages, adjust headers, track 404 errors, alter the robots.txt or .htaccess file, toggle which modules are active and numerous other SEO features.

As stated above, it’s better to build it right the first time, but just in case you can’t there are always solutions for common SEO problems. 

Read More Related Posts
The Phases of SEO Part I
According to Wikipedia - A phase is one part or portion in recurring or serial activities or occurrences logically connected within a greater process, often resulting in an output or ...
READ MORE
Puzzled About Landing Page Performance?
The saying you can lead a horse to water, but can’t make them drink is the equivalent of bringing a visitor to a landing page through paid search, banners, contextual ...
READ MORE
Web Design Trends 2010
We are taking a break from SEO related posts today and passing the blog to one of our colleagues Mr. Jim Brody who has some great insights regarding the topic ...
READ MORE
Which keyword will hook the best consumer?
There are two critical facets to any SEO or SEM campaign (1) the keywords you target and (2) how effective they are  for producing tangible sales,  leads, opt-ins or their ...
READ MORE
How to Implement GEO-Targeting and Local Search
Too often you hear about horror stories of people trusting the wrong SEO companies only to fall flat in the search engine result pages. To avoid this, there are a ...
READ MORE
Missed Opportunities are Unlikely with the Proper Incentives
Incentivising an audience to encourage click throughs and conversion is necessary regardless of the marketing channel or medium used to delivered that traffic to a website. Regardless of the fulcrum used ...
READ MORE
Don’t Let Broken Links and Errors Devalue SEO Efforts
When making viable shifts to your website or site architecture, here are a few things to take in consideration that can affect existing SEO. Like Murphy 's Law, anything that ...
READ MORE
Are we on page one yet?
It’s understandable why many feel the same way as the woman in the picture about organic SEO. You'll need patience to see results and cultivating enough patience to reserve judgement ...
READ MORE
Where is the Off Page Link Tipping Point?
Is there such a thing in SEO as the off-page inbound link sweet spot? Meaning, finding the tipping-point for inbound links per URL, per keyword… Based on our heuristic testing, absolutely! ...
READ MORE
Using .htaccess for 301 Redirects, SEO and More
First, the question; why is eliminating duplicate content important to SEO? The answer is predicated on the conditions that are promoted if duplicate content is allowed in place of a ...
READ MORE
The Phases of SEO Part I
Why Most Landing Pages Fail
Web Design Trends 2010
Find the Best Keywords to Bait New Consumers
SEO Only Works If You Do It Right!
Click Throughs and Incentives
Don’t Let Broken Links and Errors Devalue SEO
10 SEO Tips to Shorten the Time to
Where Is The SEO Backlink Sweet Spot?
Using .htaccess for 301 Redirects, SEO and More

About

In 2006, Jeffrey Smith founded SEO Design Solutions (An SEO Provider who now develops SEO Software for WordPress).

Jeffrey has actively been involved in internet marketing since 1995 and brings a wealth of collective experiences and marketing strategies to increase rankings, revenue and reach.

20 thoughts on “Duplicate Content and SEO
  1. dlgod says:

    very informative for me thanx for sharing…

  2. Your very welcome, thanks for visiting…

  3. Fundoo says:

    Definitely bookmarked! And thanks for sharing the article.very informative blog..wish u good luck

  4. John Vargo says:

    Great article on duplicate content and how to improve search engine optimization. Thanks for the SEO tips!

  5. YoVz says:

    For of all, thank you for such an informative article. I am a newbie at all this. I’m in the process of creating a website for my dad. I understand how you described to clean up duplicate pages on the site. However, my dad has several domains because he has a multifaceted business. Here is the example:
    Imagine a grocery store. They have a domain “grocerystore”. They also have these domains “apples”, “steak”, “bread”. Essentially, the websites would be almost identical, except for the landing page of each domain which discusses the term they searched on. However, the other pages discuss everything else in the grocery store. Is this considered duplicity? if so, how do I get around this? Thank you.

  6. Derek says:

    Hello. Great site full of useful information. I have a question and would greatly appreciate if you could help. Please note that I’m not very good with coding.

    I just added some footer pages to my site. I wanted the page names to be the same as the header page names. Should i “no follow” and “no index” these footer pages or just “no follow” them?

    I don’t want the search engines to punish me for having duplicate pages since both the header and footer pages carry the same name and link to the same page.

    Or should I just name the footer pages a little differently?

    Thanks for any help :)

  7. Mike says:

    Very useful post, thank you very much, will introduce these techniques on my Sky Cards website

  8. sima says:

    Hello
    I need help please,am a student,i was asked to create a micro questionnaire website…my problem is how to make sure that a survey our enters his data once and how to secure annonymity.Need help please.tnx

  9. edyzen says:

    morning sir, I have trouble with duplicate description each page. I use seo ultimate plugin. can you help me ? thanks

Comments are closed.