It’s no secret that WordPress is one of the most popular, easy-to-implement content management systems online. But just because it’s easy doesn’t mean that the out-of-the-box WordPress settings are ideal for optimal on page SEO. In fact, there’s nothing farther from the truth.
How to Block Bots for Optimal SEO
In this post, we cover simple solutions to prevent redundant pages (pages with duplicate or minimal content) from getting indexed from the roving tarsus of search engine spiders.
Search engines use a bag of words model to deduce market-focus (what each page is about) and topicality and relevancy are often lost when boilerplate templates, category pages, tags or archives pages are left wide-open for spiders to crawl.
By eliminating digital stragglers (thin pages), you’ll bolster your websites relevancy by having a higher percentage of unique indexed content.
While users might observe a tag page, a search engine is programmed to view that page as redundant content with very few classifiers to distinguish itself from the boilerplate template.
In fact, this is one of the prime directives of the Google Panda algorithm (to eliminate low quality content through normalization within a body of documents) hence, pages like these often get ignored or suppressed.
Avoid Rookie SEO Blunders
Websites with a high percentage of duplicate or minimal content on pages (such as category, archives or tag pages) can trigger algorithmic suppression.
If your an avid SEO and understand HTML then you could simply implement a meta-robots command such as nofollow or noindex, but unless you want to do that painstakingly page by page, you’ll need a plugin such as SEO Ultimate (The Premiere All-In-One WordPress SEO Plugin) from SEO Design Solutions that allows you to quickly and easily implement control over robots (search engine spiders) and canonicalization on the fly.
If you are already using SEO Ultimate, chances are you have already implemented these changes. If not, then let’s get started by making your website canonical and segmenting any potential duplicate content penalties. This is particularly important to new websites so they can make a great first impression (by preventing non-canonical taxonomies or duplicate content from being indexed).
With Google constantly updating their algorithms (Penguin, Panda and the other zoo animals running amuck) you’ll need to avoid tripping any algorithmic filters. Sharing content across multiple pages or thin content on boilerplate templates can devalue the integrity of your website architecture and spring these booby traps into action.
A splintering effect occurs dynamically by default when using WordPress. For example, when you create a post, that post also gets added to a category archive and if you use tags, then each tag also becomes a page with a link “via the excerpt” to the original post.
While implementing canonicalization helps, selective indexation via the Meta Robot Editor module is an ideal countermeasure to prevent multiple pages from being indexed and potentially creating penalties. Keep in mind, if its not indexed, there is no penalty (which is the basis of the Meta Robot Editor).
Example of Category Archive Page with Cyclical Posts Under Category
From an end-user’s perspective, tag pages are not ideal landing pages, which is why canonicalization is important. You want the search engines to show them the original page or post (not some tag or category archive).
Search engines have been known to suppress websites which allow tag pages and category archives to leach through into their index by relegating them to the supplemental results.
Taming the Search Engine Spiders
First proceed to the SEO Ultimate module manager and make sure the Canonicalizer, Meta Robot Tags Editor module, Miscellaneous module and Nofollow Manager module are all enabled >>> then click the save changes button.
If you are wondering about what canonicalization is or why it matters, here’s a brief synopsis in layman’s terms.
What it does: Canonicalizer will point Google to the correct URL for your homepage and each of your posts, Pages, categories, tags, date archives, and author archives.
Why it helps: If Google comes across an alternate URL by which one of those items can be accessed, it will be able to find the correct URL and won’t penalize you for having two identical pages on your site.
Next, click the Canonicalizer Module and under the Canonical URL Generation setting tick the first two boxes:
- Generate
<link rel=”canonical”/>meta tags and - Send
rel=”canonical”HTTP headers
This will ensure that any potential splintering of pages or posts are redirected to the primary page (helping Google avoid any potential confusion about your website architecture or data hierarchy).
Also, make sure you tick the redirect requests for nonexistent pagination checkbox to add an automated 301 redirect for categories or archives that have recently changed due to adding or deleting content within the category or archive.
Please Note: Canonicalizer improves on two WordPress features to minimize possible exact-content duplication penalties. The <link rel="canonical" /> tags setting improves on the canonical tags feature of WordPress 2.9 and above by encompassing much more of your site than just your posts and Pages.
The nonexistent pagination redirect feature fills a gap in WordPress’s built-in canonicalization functionality: for example, if a URL request is made for page 6 of a category archive, and that category doesn’t have a page 6, then by default, depending on the context, WordPress will display a blank page, or it will display the content of the closest page number available, without issuing a 404 error or a 301 redirect (thus creating two or more identical webpages).
This duplicate-content situation can happen when you, for example, remove many posts from a category, thus reducing the amount of pagination needed in the category’s archive. The Canonicalizer’s feature fixes that behavior by issuing 301 redirects to page 1 of the paginated section in question.
Implementing the Nofollow and Meta Robot Tags Editor
Since you are already in the Miscellaneous Module scroll down until you see the Nofollow Manager module>>> Click All the Boxes and save changes and your done with this step.
What is the Nofollow Manager?
The “Nofollow Manager” module lets you set the nofollow attribute on entire swaths of links on your site. Although Google has deprecated the use of nofollow for PageRank Sculpting in 2008, if you’re transitioning from another SEO plugin to SEO Ultimate, Nofollow Manager can help you maintain your existing settings if you’d rather take an “if it ain’t broke don’t fix it” approach.
Regardless of whether or not you elect to use the Nofollow Manager module as an on page SEO tactic, you can implement the Meta Robot Tags Editor as a catch-all solution.
Control default settings, pages, posts, media or term archives with the Meta Robot Tags Editor
Once complete simply save your changes and that will discourage bots from crawling the directories, pages, posts, categories, etc. listed above.
To take it a step further, you have the ability to granularly assign index or noindex or nofollow status to pages, posts, media, tags and post format archives (through either using the default settings or individually). Once default values are assigned, they will carry through to the other sections e.g. categories, tags, etc. unless you specify and overwrite the option by visiting the tabs specific to each type.
For a more detailed explaination of all the SEO Ultimate modules feel free to watch the video here.
In closing, there is one more thing we suggest in the Settings>>>Permalinks>>> and changing from the default parameter to custom and using /%category%/%postname%/ to create keyword-rich URL structure instead of the default URL output. This will give you more bang for your buck by creating keyword rich URL structures instead of the default P=ID# setting that WordPress generates by default.
Later in the week, we’ll discuss how to use the SEO Ultimate Siloed Categories Widget in tandem with the Deeplink Juggernaut Module to quickly implement website silo architecture quickly and easily on your blog.
Make sure to check back for more SEO tips and tactics to distinguish your website in Search Engines from the SEO Design Solutions Blog.
Thanks for the great seo insights. It’s a pleasure using the seo ultimate plugin. Really, no other plugin can come close to it.
Can’t wait to use Ultimate theme. By when do you think it will be launched?
Best Wishes
This is a really great plugin…
NICE WORK
+1
we’re using it on termopane mures.
@Chetan:
Thanks for asking about WP Ultimate. We are still looking at launching this month, just wrapping up some stress-testing, but I know, we need to stop obsessing and stop the madness and launch.
We will keep you all posted when you can click the download button. All the best,
Jeffrey
Thanks Jeffrey. I appreciate everything you guys do over here! It seems that no one is actually coming out with products with an affiliate perspective. You guys truly rock! I am tired of reading white hat marketing blogs (seomoz etc) preaching google ways & content is king stuff. They are right in a way but there is a lot more to SEO than just content & user friendly sites. Your plugin & the new theme are way above the rest. I even love the blog posts here, cuz it serves information that is actually helpful for real affiliates like us.
Best wishes
@ Chetan:
Thanks again for chiming in on this. I fully agree that SEO is more than just content, unicorns and rainbows and putting your eggs in one basket (Google). It’s safe to say that the devil is in the details, which most keep close to the chest.
Our goal has always been to help pioneer better ways for affiliates, small to medium size business owners and people genuinely out there trying to get ranked have a competitive advantage and get the upper hand and achieve ROI on their time and money.
I can speak from experience on this one, there is nothing more satisfying than conquering a market defining keyword while others scratch their head trying to figure out how we did it. Its not just content, its site architecture, link-flow, indexation, internal linking and so much mrre.
Yet, even when this happens, it’s only a matter of time before Google changes something else and the whole cat and mouse game starts over again.
We will be rolling out some powerful strategies soon dealing with RSS syndication and using multiple web properties as extentions of your brand that essentially create an unblockable blanket of relevance that search engines simply cannot deny.
You have to get away from the one site mentality and use other authority sites as mediums to drive traffic through social, search and all profitable channels.
Our theme and plugin are only the tip of the iceberg for what we have in store for the community we have started and I thank you for being a part of it and speaking your mind.
All the best Chetan.
Thanks again for your insights. Look forward to all the new tactics & features.
Can’t wait to get my hands on the new theme!
Best of Wishes,
Chetan
Thanks for this informative post. SEO Ultimate is the best wp seo plugin, i love it more than yoast seo plugin. Thumbs up to you guyz.