Web Analytics Tutorial

Lesson 3 – Search Engines

Search Engine Ranking

Search engines are all built on the same principles, but use different techniques and technologies to achieve their goals. For a search engine to be effective it must produce the most relevant web pages for a given topic or topics as the first few items that a user sees. Most search engines use a robot (sometimes called a ‘spider’) to build their index. This robot is an automated program that follows links in web pages all over the Internet. For each page, it remembers the URL and determines, by many techniques, what the relevant concepts are on that page.

The ‘simplest’ way to get your site ranked first in each search engine is to make it the most relevant page on the Web for your given keywords. In order to do this you need to know which keywords visitors are using to find your site. You also need to know what other sites are out there and you need to decide what makes your site more relevant than others, sometimes by understanding how individual search engines determine relevancy.

Keywords and META Tags

Figure 5. The Search Words report tells you what keywords search engines are listing you under.

The Search Words report shows you what keywords most visitors are using to find your site. When you are listing your site with search engines, you can use this report to decide which keywords are most important to your visitors. Figure 5 shows the top search words as manager, widget and holder. When choosing keywords to represent your site, you will want to make these most prominent. The Search Word report can also highlight gaps in your current keyword listings. If you would expect some keywords to be common and they are not listed, then perhaps you have not taken the necessary steps to ensure that search engine robots can find those keywords on your site.

When a search engine robot looks at the pages on your site (“crawls your site”), it looks at different elements of the HTML file to determine what topics it is relevant to. Most search engines put extra weight on the words used in the page title. Also, many search engines display the page title on their search results page, so it is imperative that your keywords are included in the title and that the title is meaningful to the humans who are reading it in that context. You can also help most search engines by giving hints about the keywords for a given page. This is done by including an HTML tag in the header of the page. This ‘META tag’ is never seen by most web users, but robots read it to find your keywords. The tag is in a form like this, giving a comma- or space-separated list of words or phrases that the engine should index you page with:

 <meta name="keywords" content="keyword, a key
phrase, another phrase"> 

Each page on your web site should contain one of these tags. Some sites like to use the same keywords for each page, other like to customize the keywords to make the pages most relevant to the search. In addition to title and META tag, some robots will look at the content of the page. Usually, they only inspect the first 100 to 200 words. So you should make sure that the text that starts each page includes those same keywords and any additional information that defines the page or site.

Some search engines also include support for a description META tag. This description is used in place of the first paragraph or so of text when shown to the reader. If you can summarize your page in a few lines, it may be advantageous to add this to some pages. Also, some search engines, such as Google, put extra weight on keywords in the description META tag. The format for this tag would be something like this:

  <meta name="description" content="Insert a few sentences 
  here to describe or summarize the content of the page.">

Figure 6. Other metrics in the Search Words report can help qualify important keywords.

When choosing which keywords to include in your report, you may want to select a different metric than Hits. Figure 6 shows that the Search Words report also shows the number of Average Steps, Goals and Value for each word. In Summary, you can click on the columns header to have it sort the report by that unit. Summary also has specific reports that rank Search Words by Goal and Search Words by Value. Whatever metric you choose to qualify your keywords by, you should be able to select the top 10 or 15 words and use those when building your META tags and deciding what to include in your titles and first few paragraphs of content. Goals and Values are covered in further detail in Lesson 9 - Incorporating Business Goals and Lesson 5 - Revenue Modeling, respectively.

Competitive Analysis

Figure 7. Concentrate your analysis on the top few listings in the Search Engines Report.

Now that you have determined what keywords are important to your site, you will want to find out how you are ranked and how other sites are ranked with the same keywords. Using the Search Engine report, Figure 7, look at the top five to ten entries to start your analysis. For each listed engine, go to the search site, type in one of the keywords or phrases you have chosen and navigate through until you find a link to your site. If it is not in the top five pages, it is as good as not listed. In fact, very few people will even navigate to the second page of results. Do this for each keyword or phrase at each engine and see where you rank. While you are finding your listing, look at the top two or three sites for each search. These sites should give you a good idea about what kind of content the engine finds relevant to the topic and may provide insight on how to improve your page to get a higher ranking. You can also use the “View Source” or equivalent function in your web browser to see what keywords they put in their META tags.

Improving Your Ranking

Getting your listing near the top of major search engines is a complex science. There are whole books written about it and services that can provide expert advice and tools to assist you. However, with the knowledge you have gained from your search reports, you can start doing some of it on your own. Now that you know what keywords are important, make sure that they are included in the title, META tags and first paragraph or two of content on each relevant page. Resubmit your site for update with the engines and monitor the results (we will talk more about submissions in the next section).

The other major factor in search engine listings is ‘link popularity.’ Popularity refers to the number of other relevant sites that link to your site. If your site is referenced by other sites with the same topic, then yours is assumed to be more relevant. This technology was pioneered by Google and is now used by most major search engines. If you can get other sites that discuss your topics to link to your site, that is a good start. Of course if your site is the most relevant site, then it is in the interest of those other sites to link to yours.