If you answered yes to this question, then you are not alone. In fact, may businesses who rely on organic SEO (natural search engine traffic), constantly obsess about their SERP (search engine result page) positions. Not only are in many cases the SEO rankings the livelihood of their business, but acquiring a top ranking position for a competitive keyword is quite the endeavor which could take months or years depending on the amount of time it takes to create relevance or overcome competition.
Why Does Ranking Higher Matter?
Let’s face it, consumers are entrenched in the pervasiveness of convenience and those who appear higher up in the search results win the first chance to pitch and possibly close a prospect who used those keywords for that search query.
Conclusive studies have been performed in great length to determine the click through percentages of the websites that appear above the fold (in the top 3 or top 5 results). It is no secret that if your website is one of those results (ideally with a relevant page, title and enticing description / snippet), you are well on your way to reaping the rewards of that keyword and a multitude of stemmed or synonymous phrases.
Each time your website gains enough relevance to appear in the top 10, you have already stepped over 990 other websites to attain that position. This means that many of the ranking factors that are deemed a prerequisite for a less competitive ranking can also be scaled and consolidated to target more competitive phrases.
The way that you would conquer a competitive ranking is through systematically targeting keywords within your reach, gaining a foothold and then leveraging that position (regardless of competing pages) to elevate multiple related pages within your website.
Through using the relevance / internal linking / buddy system, you can effectively channel that relevance to other pages which will in turn (if optimized) become buoyant as a result of the authority your collective pages are garnering.
Relevance is the core of competitive rankings and what was relevant 6 months ago, may not be now, so, you have to constantly refine your content or add more content to remain relevant in competitive arenas.
10 Things You Can Do to Diagnose the Problem.
With so many variables at play in search engines such as (1) search engines applying filters to neutralize increased barrages of spam (2) the introduction of new algorithms or (3) the competition simply is becoming more competitive. You need to look for on page and off page patterns that can provide more insight to the phenomenon.
In other words, to assess the competition, check the search engine result pages:
1. Is there an array of new contenders on the page? (if so, look at their crawl data and determine if those pages are fresh). Have you ever seen those pages? Or where they 10-20 results buried deeper in the search engine index? If a page jumped that high that you have never witnessed there, then either that search engine is testing a new filter or you need to see what they did to get that kind of buoyancy.
3. Look at the occurrence for the keyword and where it appears in the title, description and on the page (how many times, etc). If the search engine shifted their algorithm to pay more attention to title tag continuity and your lacks relevance, you could have slipped on that metric alone.
4. Use Google search operators to check your own allintitle:keyword, allintext:keyword and allinanchor:keyword ranking position (by entering them in a search box) or enter the keyword “in quotes” in a Google search box to see how all three of those metrics deem your website relevant. If your off the first page with any of those, chances are your competition turned up the heat. You will also see the new arrival there in either the allintext, allinanchor or allintitle command to see if it may be temporal or if they earned their new spot. NicheWatch is a great tool for this.
5. Does the search result in Google display a cached date for the page? Look under the search result for a small link named “cache“. If not, it may just be a news/fresh resource that has gained relevance that the search engine may be displaying through universal search and will probably roll back when the link velocity has worn off.
6. Is it the homepage or a specific page being displayed? if the result is a homepage vs. a website.com/the-keyword.html, multiple variables within their own site and from other sites may be in play that resulted in increased relevance. If it is an individual page, then that page could have acquired a larger percentage of link flow from the site or deep links (links from other websites to that page). Look at the top 5 results to see what the threshold might be (200 links, 500 links, 20 links, etc.) You can also use this on page SEO tactic which uses an array of SEO tools to delve into the on page metrics.
7. Are you using a CMS (content management system), if so, make sure that all third party plugins, applications or modules are functioning as planned. If your permalinks (naming conventions) break, or some aspect of a program that controls dynamic rewrites or things of this nature break, then your site could have dozens of broken links which could result in a diminished search engine position.
8. Have some pages went into a supplemental index? Although search engines rarely discuss this anymore, the fact is, with the rate of new information is being introduced only leaves a finite room of space available in the search engine index. If you have a large number of pages in your site, and many of them are 60-70% similar in context (due to a template or lack of content to differentiate the pages) then many of the pages providing internal links to your preferred landing pages could have taken a hiatus. The solution, get some deep links to the pages that have not been updated or crawled in a while, or consider going back to reinvigorate pages or posts with more content or internal links or just 301 redirect them to provide more buoyancy to related / more relevant pages.
9. Check your server logs to look for anomalies, at least you can find the time the problem occurred. Often server outages can result in jeopardized rankings as well, since if a search engine user agent tries to crawl your site and the result either times out or is unavailable, there goes a percentage of the pages retrieved for the index. Less pages equal less ranking factor if those pages are integral as hubs to support other landing pages.
10. Build more internal links to the page that dipped. Consider this shoring up relevance just in case. To find the page with the most relevance use the Google advanced search operator site:yourdomain.com keyword [replace yourdomain with your site and keyword with the keyword] Also, there is no space between the word site and the domain. When you hit enter, you will see a list of the most relevant pages (according to Google) for that keyword within your website. Your page you select should be first, if it isn’t, then start building links from the other 9 pages on the list to your preferred landing page (with the keyword that slipped). Repeat as often as need throughout the site for multiple variations of keywords to strengthen and optimize internal links.
However, sometimes you just have to wait things out. Google and other search engines are populated on multiple crawlers who are all uploading results from an array of sources, data centers, etc. I often refer to this as “the weekend index“.
You should also consider there is no significant way for search engines to perform “maintenance mode” on the fly, so often what you are seeing is older cached data standing in place of “the real search results” which will get blended back into the SERPs over time.
The main thing is, stay calm, become systematic in your approach, make periodic backups and don’t do anything too radical as you may initiate a new ripple of cause and effect consequences that may be harder to diagnose that the original dilemma in addition to other valuable metrics.