in SEO Tips by Jeffrey_Smith

Just like the real world, you have to crawl before you walk, the same applies for SEO and getting pages indexed in search engines that need to be crawled and cached before they can rank…

SEO Tips for Sitemaps

SEO Tips for Sitemaps

Managing crawl rates and ensuring your website has enough internal links or sitemaps to get search engine spiders deep into the sites architecture is a critical component of search engine optimization. In order for pages in your website to rank or in this case even matter to a search engine, it needs to be present in the cache as a result of being indexed.

Countless horror stories occur when pages fall out of the index. This can happen from (a) the wrong use of the .htaccess file which may have inadvertently blocked spiders from crawling the page (b) changes to navigation  or (c) changes to file name and / or naming conventions which result in broken links.

Although rankings typically resume when the broken links are fixed and re-crawled, or when the .htaccess file is restored, it can be a harrowing experience for any business.

There are certain protocols you should remember about sitemaps, if you do not have a cron (script) that monitors the addition or removal of new pages in your website and automatically updates the sitemap, then, when you make changes to a site, make sure to upload a fresh copy.

Google webmaster tools allows you to track how frequently your sitemaps are crawled, so this is your first line of defense to ensure that your pages get in and stay in the index.

Otherwise, one way to expedite crawl rates is to acquire links from sites with a higher crawl frequency. This means any website capable of having a do-follow link that spiders will traverse will be suitable in igniting a fresh crawl.

The larger the site the less frequent the crawl for pages further away from the root folder. Particularly in regard to sites with similar elements such as shopping carts or pages that do not have enough content to truly stand on their own as a genuinely unique contribution.

Using a tiered approach if you have hundreds of pages tucked away in subfolders works well. This means that you simply have a master sitemap that links to subordinate sitemaps. So, with a site with 10 sub folders, each folder should have its own sitemap; than all 10 should be linked from the main sitemap.

That way, when one is crawled, the others are included in that crawl and any changes you may have made to any of the pages all make their way into the search engine result pages.

If you find your website has become unruly and has too many sub folders, then flattening site architecture is another solution that can remedy deficiencies in link flow and revitalize languishing pages.

For example, if you have a dynamic site, you could simply copy the contents of a page and reposition it. If the page were domain.com/products/electronics/dvd-players/black-sony-model234.html you could create a static page close to the root and flatten the site / subfolders by consolidating the naming conventions found in the entire url string.

The revised page could be domain.com/electronic-products-black-sony-dvd-player-model1234.html which puts that page in the original root folder (which inherently has more link flow that other areas of the site).

As a result, that page could hold its own, enjoy a more frequent crawl rate (granted it is included in sitemaps, internal links or navigation) but the takeaway is, it is easier for user agents a.k.a search engine spiders that crawl the web to locate, update and index and changes to the file.

The last step would be to 301 redirect the old file location domain.com/electronic-products-black-sony-dvd-player-model1234.html to the new location domain.com/electronic-products-black-sony-dvd-player-model1234.html which can often be accomplished with a shopping cart setting.

After that, you still need to get spiders back to the page since files that far away have a longer crawl cycle, so by either building links to the old location or the new location, you will toggle the positive SEO effects of the pages recent transition. 

Read More Related Posts
Carbon or Silicon What Defines Intelligence?
Search engines keep a running keyword tally of which pages you have, who links to them, what they are about and how popular they are. If somewhere on some page ...
READ MORE
SEO Ultimate Version 0.8 Now Features a "File Editor" Module
SEO Design Solutions is pleased to announce another release / module for the SEO Ultimate All-in-One SEO Plugin. SEO Ultimate 0.8 includes two new features: robots.txt editing and htaccess editing, ...
READ MORE
When to use 301 redirects for SEO
When acquiring competitive keywords the rules of engagement change. Today’s SEO tip revolves around using 301 redirects, internal links and search operators to leverage previous content for present-tense SEO objectives. There ...
READ MORE
Does your online marketing make dollars and sense?
Making money with your website is not as difficult as the gentlemen in this graphic depicts.  However, it does require that you understand the basic premise of relevance, positioning and ...
READ MORE
Landing Pages and the Value of Propositions
During the several hours of perusing the internet each day, I come across many value propositions. Like most web surfers, I can't help but try my luck at winning a ...
READ MORE
Reviving The Long Tail of SEO
Depending on how far along in the sales cycle a consumer is, the keywords they use to find products or services in search engines drastically vary. Statistically, someone "closer to ...
READ MORE
How Does Your Website Measure Up?
Google officially updated the TPBR Page Rank last night. For those in the SEO community, the significance is met with mixed emotions ranging from dismissive to celebratory. Although page rank ...
READ MORE
SEO: Understand the Window of Opportunity
There are two common dynamics that can play out when seizing the SEO window of opportunity in your market. You are either  (1) the most relevant / grandfathered search result ...
READ MORE
WordPress 3.0 Compatible SEO Ultimate 2.9.1 Update
In the last month, SEO Ultimate (the free WordPress SEO plugin from SEO Design Solutions) has been updated with no less than 8 major functionality upgrades. SEO Ultimate now stands at ...
READ MORE
3 SEO Tools for On Page and Off Page SEO
When you’re tired of playing childish games like leap-frog in the search results with your competition and are ready to change the subject from recess to biology and literally dissect ...
READ MORE
What Do Search Engines Know, That We Don’t?
SEO Ultimate 0.8 Released!
SEO Tips to Acquire Competitive Keywords
SEO Marketing Meets Dollars and Sense
Landing Pages: The Value of Propositions
Reviving The Long Tail of SEO
Google Toolbar PageRank Update October 2009
Seizing the SEO Window of Opportunity
SEO Ultimate WordPress SEO Plugin Version 2.9.1 Released
SEO Tools for On Page & Off Page

About Jeffrey_Smith

In 2006, Jeffrey Smith founded SEO Design Solutions (An SEO Provider who now develops SEO Software for WordPress).

Jeffrey has actively been involved in internet marketing since 1995 and brings a wealth of collective experiences and marketing strategies to increase rankings, revenue and reach.

14 thoughts on “SEO Tips for Sitemaps
  1. Star says:

    Thanks for sharing nice information. Is there is any tool that generate sitemap with more than 500 urls for free

  2. is there any tool that generate sitemap with more urls for free

  3. Hi,

    I learn a lot from your above three point (a) the wrong use of the .htaccess file (b) changes to navigation or (c) changes to file name and / or naming conventions…

    It is a very information post for the entire webmaster regarding Site map!

    Have A Good Day!!!

  4. Frank Lynch says:

    These are all common sense SEO approaches that are still very valid. good post indeed.

  5. Helpful Guy says:

    Hi there,

    To be crawled, or not, we still need to read lots and lots of SEO tips in order to be on the right path of those spiders and search engines.

  6. No Name says:

    Well i really liked the way you have explained the concept.It has certainly increased my knowledge on SEO.

  7. bobby mm says:

    thanks for this post. Its help me a lot about SEO.

  8. Sure thing Bobby, thanks for visiting.

Comments are closed.