Using Statistics for better Search Engine Optimization
Matthew Henry, Portent Interactive, Seattle, WA
Edited by Ian Lurie
September 22, 2004
A note from Ian: Search Engine Optimization may seem more art than science. And it definitely requires a bit of both. But you can use some basic statistics to focus your campaigns, separate reasonable goals from unreasonable ones, and more efficiently use your search marketing dollars.
Matthew Henry is Portent Interactive's resident search engine optimization genius and math wizard. Read on to learn how two of his creations - Relative Position Index and Click-Through Prediction - help target our SEO campaigns.
RPI: Your Search Engine Optimization Potential
RPI, or Relative Position Index, is a metric we use to track search engine optimization potential and progress. Your RPI score shows how well you're doing relative to your competitors - it's an indicator of how you're doing at outranking them. It is based on a logarithmic scale, so an increase of 1 to this value translates into a tenfold increase in ranking strength.
The formula for RPI is:
RPI = Log10[competition/rank]
competition is the number of competing sites in a search for this term.
rank is the rank that your site gets on a search for the term.
In case you're not a math geek, here's another way to look at it:
An RPI of 0 means you rank dead last.
An RPI of 1 means you rank in the top 10%.
An RPI of 2 means you rank in the top 1%.
An RPI of 3 means you rank in the top 0.1%.
An RPI of 4 means you rank in the top 0.01%.
An RPI of 5 means you rank in the top 0.001%.
and so on...
Why do we need RPI? Because it's a good way to judge good search engine optimization targets. For any two keyphrases, doing the same amount of work will get you roughly the same RPI, but the actual ranks may be quite different. So we can use RPI to figure out which terms will get you the highest rank, the fastest, for the same amount of work.
This makes RPI an essential strategic tool - we can figure out which shorter-term 'money' terms will get you a quick burst of traffic, and plan our longer-term campaign for higher-profile, more competitive terms that will require more work but also offer a better payoff.
What really makes RPI valuable, though, is that we can look at a web site and judge the maximum attainable RPI for that site, at that time. It also gives us goals to shoot for to improve RPI, as well. In my experience, the following RPI values are realistic goals:
An RPI of 4 is possible with a medium-sized site with a moderate SEO campaign.
RPI of 5 is possible for a large site with an aggressive, year-long SEO campaign.
RPI of 6 is within reach with a huge site with and an aggressive, multiple-year SEO campaign.
RPI of 7 or 8 is only possible if you happen to run a "Household word" site, like Ebay or Amazon.
Assume a 'medium-sized' site has 20-40 pages of quality, text content. A 'large' site has 40-100 pages. A 'huge' site will have at least 100-200 pages of text content.
So, if we find a set of terms and calculate the RPI necessary to achieve a high rank for that term, we can use RPI to predict what is or isn't possible for your search engine optimization campaign, given your current site. We can also plan out how we might grow your web site, and bring terms requiring a higher RPI within reach.
Using RPI to Predict Search Engine Rank
RPI makes it possible to estimate the rank you will achieve for a given keyword, on a given web site with a given number of pages of text content.
To estimate your potential rank, after search engine optimization, use this formula:
Rank = competition/10RPI
Round up to the next integer.
competition is the number of competing sites in a search for this term.
rank is the rank that your site gets on a search for the term.
Important caveat: This is an educated guess only! Your actual rank will be determined by a large number of factors, many of which you have no control over. Your actual rank might be higher or lower than this formula will tell you. The main idea is to make the best guess possible with the information you have.
Here's an example: Suppose you have a site that can reasonably achieve an RPI of 4 or 5 - it's a fairly good-sized site, but not over 30-40 pages. Then assume you conduct a moderately aggressive SEO campaign - you optimize your site, add some targeted content and get a few incoming links, so you can achieve a target RPI of 5 for a keyphrase that has 200,000 competing sites. Plugging these numbers into the formula gives you:
200,000 / 105
= 200,000 / 100,000
= 2
The formula predicts a rank of #2.
Note that RPI can be used to make predictions about rank, but it doesn't tell you anything about the amount of actual traffic you will receive for a term after search engine optimization. For that, you need to use Click-Through Prediction.
First, your rank is estimated, based on target RPI and the amount of competition. (See RPI for more about this.)
The next step is to calculate the share of clicks that will go to a listing with the predicted rank. In reality, this is determined by many hard-to-measure factors like how well-worded each description is, the number of results that can be seen without scrolling, and the particular tastes of the searcher. Rather than attempt to model all this I use a highly simplified falloff model.
I assume that each position below #1 receives 75% of the click-throughs that are received by the position above it. Position #1 gets about 25% of the total clicks. Position #2 would then get 18.75%, position #3: 14.0625. The 75% figure is based on current statistics on user click-through from search engines. The formula for this is:
share = 0.25 * 0.75(rank - 1)
The last step is to multiply the share of clicks by the number of searches per day for the given term. You can obtain this data from a keyword stat service such as WordTracker. This gives us:
CTP = share * searches
Some important caveats:
This is an educated guess only! Your actual number of click-throughs will be determined by a large number of things, many of which you have no way of knowing about. Your number of click-throughs will probably be more or less than this formula will tell you. The main idea is to help you make the best guess you can with the information you have.
Keyword search stats from WordTracker are approximations themselves. Major engines like Google & Yahoo do not make their search logs available to the public. To get around this, WordTracker bases its stats on search data from meta crawlers like Dogpile. These stats are scaled up to give you figures for Google, Yahoo, etc. This method is not 100% accurate.
Because of its approximate nature, CTP is primarily useful as a relative measure. If you get a CTP of 100 for a term, do not assume you will actually get 100 clicks. You will, however, probably get roughly twice the clicks you'd get for a term with a CTP of 50.
Conclusion: What does it all mean?
CTP is totally dependent upon the RPI you can achieve. So, the best 10 keywords for a site that can achieve an RPI of 4 may be completely different from the best 10 keywords for a site that can achieve an RPI of 5.
Why? Because CTP can be high because of low competition, or because of a very high number of searches per day.
If your site has a low RPI, you'll want to target terms with lower competition. The high-competition terms may get more searches, but this won't bring you any traffic if you rank #400. If you target low-competition terms, you'll have a good chance of getting a high rank and getting relevant traffic.
If, on the other hand, your site has a higher RPI, you can shoot for more competitive terms that have far higher search frequency (they get more searches per day) and therefore get more search traffic after optimization.
At lower RPIs, the best keywords will be those with low competition. At higher RPIs, the best keywords will tend to be the ones with high search frequencies.
Portent approaches any search engine optimization campaign from two directions. Any good search engine optimization campaign starts with a realistic estimate of your RPI, and then chooses target terms based on a balance of CTP and RPI. By targeting terms for which you know you can achieve a strong rank, you can generate good, relevant traffic in a relatively short time.
Then we consider how we can increase the RPI of your site, with content strategies that may include corporate blogging, conversion of legacy content to new pages on your site, or completely new sections and information.
This balanced approach is the best way to achieve good short- and long-term search engine optimization results.
About the Author
Matthew Henry is the search engine optimization specialist at Portent Interactive, a full-service internet marketing agency in Seattle, WA. Portent offers complete Internet marketing support, including search engine optimization, e-mail marketing, and web design and development. Recent projects include www.dessy.com, alfredsungdresses.com, princesslodges.com and SEO for ModernBride.com.
About the Editor
Ian Lurie is an Internet marketer in Seattle, WA. He started his Portent Interactive in 1995. Ian has a law degree from UCLA and has successfully avoided practicing law for almost ten years.