in SEO Basics by

Search engine spiders thrive on fresh content and websites with an abundance of PageRank, authority and pizazz. Instead of making SEO more complicated than necessary, sometimes you need to “go back” to basics to move forward and simplify your search engine optimization approach.

SEO Tips for Search Engine Spiders

Feeding Search Engine Spiders

Aside from all of the strategies, methods and tactics, tweaking pages and waiting for spiders to re-index those changes on your web pages is 50% of the SEO process.

With that in mind, here is a simple but effective way to increase crawl-frequency and feed the search engine spiders…

What is SEO aside from:

  • Creating Relevant Content on a Solid SEO Friendly Platform using Relevant Titles, Meta Data and Internal Links
  • Developing Trust, Domain Authority and Link Popularity Over Time and
  • Revisions and Fine-Tuning


On the last point (revisions and fine-tuning) is often where many ranking battles are won or lost. If at first you don’t succeed, then refine the page; add more pages to augment your preferred landing pages or build links to gain additional traction.

Addressing crawl frequency and indexation issue directly implies that the more pages you have, the more page rank you can have working to deliver more traffic through a broader array of keywords.

After you make a change to a web page, you have to wait for spiders to re-crawl the page and the search engines repository and index (which is what queries hit) to update their cloud data before your rankings rise or fall based on the revisions.

In an interview with Eric Enge, Matt Cutts (Google’s head of web spam team) confirmed that PageRank equates to crawl priority crawling the web from the top-down Page Rank model from both (a) in the collective repository of Google’s cached data and (b) using PR as an indicator of page priority to determine the depth of the crawl.

So, aside from (1) looking at stagnant pages and considering if they rank or need revisions or deep links to provide buoyancy you need to (2) make revisions to the on page and (3) get spiders to revisit those pages.

This brings me back to an old school tactic to (a) pull pages out of the supplemental index that haven’t changed or been linked to in a while and (b) get pages re-indexed after making changes. The technique is simple – just link to them from a high PR site that allows you to post your link.

Who would have thought that something as simple as linking from a website like Craigslist.org, Folkd.com or any website with a PR7 or higher (ensuring it is within the top tier seed set of authoritative domains online) is all it takes to expedite a re-crawl to pages nested deeper in a website.

All most pages need are 3-5 links to those individual pages (to distinguish them from the millions of pages that have zero links) to improve search engine placement for something specific keywords nested in the content. Getting an authoritative link does two things (1) it improves the link citation of your link profile and (2) helps spiders dig deeper to re-index changes and gain more contexts about your website, its theme and where it belongs in the search engine index.

So, the next time you tweak a page, make sure to drop a few links from a website with a high crawl frequency that has do-follow links to get things out of the supplemental index and ranking on your behalf. 

Read More Related Posts
SEO Requires Planning and Patience
Search Engine Optimization (SEO) is all about layering processes over time. Often we receive calls from prospects who (a) want to rank for the most competitive keywords and (b) do ...
READ MORE
SEO Tips to Take Usability & Design to The Next Level
What is the point of driving traffic to a website if you send them away from bad design once they get there? Sure, SEO is important but so is usability, ...
READ MORE
SEO Web Design Revisited
Its time to bring back the timeless concept of "superb design" ; the synergy of fusing SEO, web development and web design to appease search engine spiders with proper content, ...
READ MORE
Google's Love Affair with Authority Sites
In short, Google's love affair with authority sites is based on the premise of reciprocity, relevance and commercial viability. It’s no secret, that Google’s search engine algorithms are stacked in ...
READ MORE
What SEO can Do For Your Business
If you understand the psychology of supply and demand, search engine positioning and more importantly what consumers want, you can devour entire markets as a result of superior SEO and ...
READ MORE
Alt Attributes and SEO: This image could be a link for the keyword butterfly
Today, I would like to share a powerful SEO technique (using server side includes SSI and alt attributes) to recapture a website from sagging rankings, orphaned pages, theme diffusion and ...
READ MORE
Hmmm, Smells like a Sponsored Link to Me...
In the competitive realm of SEO, honest business people are often tempted to resort to less-than-honest forms of internet marketing. We’ve all heard the rumors about competitors clicking on your ...
READ MORE
Block Segmentation, Duplicate Content, SEO and IR
Often you here about the need to prevent duplicate content within your own website as it applies to SEO, but why? This is our take on why creating uniquely distinct ...
READ MORE
Rankings Slip? Assess Metrics to Minimize SERP Volatility
If you closely monitor your SEO rankings, from time to time a ripple enters the SERP’s (search engine result pages) and shakes things up a bit in the index. You ...
READ MORE
WordPress SEO Tips: Overcoming Wildcard Categories
The process of SEO frequently requires that you conduct periodic sweeps of the various platforms that make a website operational and either contribute to or impede search engine rankings. Sweeps like ...
READ MORE
SEO Requires Planning and Patience
SEO Tips to Take Usability & Design to
SEO Web Design Revisited
Google’s Loves Affair With Authority Sites
Search Engine Positioning: Consumers, Supply and Demand
Using Server Side Includes and Alt Attributes for
SEO Links: The Way They Were Meant To
Block Segmentation, Duplicate Content, SEO and IR
My Rankings Fell: Dealing with a Slip in
WordPress SEO Tips: Permalinks, Categories and Duplicate Content

About

In 2006, Jeffrey Smith founded SEO Design Solutions (An SEO Provider who now develops SEO Software for WordPress).

Jeffrey has actively been involved in internet marketing since 1995 and brings a wealth of collective experiences and marketing strategies to increase rankings, revenue and reach.

10 thoughts on “SEO Basics: How to Bait Search Engine Spiders
  1. John says:

    Wow!, i will try this one, this page is very helpful. thank you webmaster.

  2. To bait Search Engine Spider :

    1)Directory Submission
    2)Article Submission
    3)Press Release Submission
    4)Social Bookmarking
    5)Blog Comments
    6)Forum posting etc…

  3. Great tips. I think I will tweak some of my pages so I can get more traffic. Thanks for sharing.

  4. people pay good money on some really rubbish products,your free page has helped me alot. a big thank you from the auto seeds bank team.

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>