It is important to remember that these are what manual raters are asked to look for. Machine learning algorithms like Panda are completely separate, but both manual raters and the Panda classifier are probably working to roughly the same goals.

Raters are given a query and a location and are then asked to rate pages for that particular query and location based on the presumed user intent for a particular query. Each sites gets one of the following ratings: Vital, Useful, Relevant, Slightly Relevant, Off-Topic or Useless, and Unrateable.

Sites can also be flagged as Spam, Pornography and Malicious.

Raters are expected to evaluate local intent based on the query location and query itself.

Based on screenshots within the document, a number of raters will rate a particular query/page and they are supported by administrators and moderators.

Here are some highlights.

Six Months From Now, Lists Will Be a Spam Signal

The document identifies a class of queries called “queries that ask for a list.” So go spam it to death.

After typing a query, the search engine user sees a result page. You can think of the results on the result page as a list. Sometimes, the best results for “queries that ask for a list” are the best individual examples from that list.The page of search results itself is a nice list for users.

A landing page that provides links to many good individual results can also be very helpful to users.

“Queries that ask for a list” may be typed in singular or plural form. For example, the query may be [bank], English (US) or [banks], English (US)

The document lists credit cards, banks, bikes, airlines hotels and London Boutiques as examples of queries that ask for a list.

So now everyone will be running to create lists for this sort of query which will quickly make lists a negative quality signal forcing Google to focus on providing the list in the search results page rather than ranking pages with comprehensive lists.

The Relationship between Ratings and Spam

Spam flags do not depend on a relationship between the query and the landing page. A page should get a Spam flag if it is created using deceptive techniques – no matter what the query is or how helpful the page might be…

In some specific cases, it is also possible for a page to receive a Vital rating, and also be assigned a Spam flag.

This isn’t really a big surprise, but it shows how big sites like JC Penny and BMW can spam Google and get away with it because they also get Vital or Relevant ratings for a lot of queries.

Keyword Stuffing

Not that I’ve ever felt the need to stuff a page full of keywords, but for clients that need some convincing…

“We ask you to assign a Spam flag if you think the number of keywords on the page is excessive and would be annoying and distracting to the real user.” p. 99

and, somewhat more interestingly

“URLs may also contain keyword stuffing. These URLs are computer-generated based on the words in the query and are often formatted with many hyphens (dashes) in them. They are a strong spam signal.” p. 99

Sneaky Redirects

Redirects to domains under different registrants is a spam signal.

If you are changing domain names, make sure you use the same registrant for both domain names. This is a particular danger for freelancers who start under their own name and then create a proper business entity or businesses that change names.

What is Spam

Most of the answers shouldn’t come as a big surprise, but I thought the below technical definitions to be quite instructive.

Feed-driven sites with PPC ads are a no-no.

“A page that just contains freely available feeds and PPC ads, and was created just to make money, is spam.” p. 102

And raters are asked to watch for template-driven sites that use keyword suggestion tools to generate related pages.

Some websites use templates to mass-reproduce webpages automatically. The content is usually copied from sources that provide such content. You will learn to recognize templates, which usually follow a generic format or pattern. Look for slight keyword variations that suggest automated use of a keyword suggestion tool. p. 103

Not all Affiliates are Thin

I know a lot of people are struggling with Google’s definition of “Thin Affiliate.” So, straight from the behemoth’s mouth…

Some affiliates are created to help users. Anyone can become an “affiliate” of merchant sites such as Amazon and link to Amazon products. Webmasters may do this to show products they like or to help users find a good deal.

For example, if the affiliate offers price comparison functionality, or displays product reviews, recipes, lyrics, etc., it is usually not a thin affiliate, and, therefore, not spam. Some websites that offer price comparisons or other helpful shopping features, in addition to the affiliate link p. 106.

The lesson here is to be a good website first and an affiliate second. I know, it’s not a great revelation, but it’s worth repeating.

Unique Content is King, Or at Least it is not Spam

Some webpages with content are created just for the purpose of putting ads on them; writers are paid by spammers to create articles on a wide range of topics. Often the articles are very generic and don’t provide a lot of good information, but they are original. You won’t find the articles on another website. Although you may be convinced that the intent is to deceive, if the content makes sense and appears to be original, you will not be able to assign a Spam flag to such pages. You will have to use your judgment. p. 107

Of course, having unique low quality content can still get you a low rating, you just won’t get a spam flag.

Phishing≠Malicious

Ooh boy, I’m not sure how I feel about this. I know it will make some people very angry. But I don’t often see Phishing sites in Google, so should I care about what they tell manual raters?

This landing page should make users (and raters) very suspicious and cautious. The spelling and grammar are bad and unprofessional, and the page feels “spammy”. What is most worrisome is that the page asks for the user’s bank password and pin number!

Even though we would not want to interact with the page, this type of phishing does not go against the Webspam Guidelines and the page should not be flagged as spam or malicious. p. 108

Conclusion, as in Google’s Conclusion, Not my Own

Remember to look at the page as a whole. Spam pages usually have some of these characteristics:

• PPC ads are usually very prominent on the page, and it is obvious that the page was created for them.
• If you do a text search, you will find that the content has been copied.
• If you visually remove all of the spam elements from the page (PPC ads and copied content), there is nothing of any value remaining

Good pages usually have these characteristics:

• The page is well-organized. There may be ads on the page, but they are well identified and not distracting.
• If you do a text search, the original page is usually the first result displayed.
• The page will have value to the user. A good search engine would want the page in a set of search results

I laughed at the good pages have ads that are not distracting part. So basically every single newspaper site in the world is a bad site for those terrible flyover ads.

I found this document on another site, it is not available at that site anymore. If you are looking for the document, I recommend you take an excerpt from this post and search for it in quotes in either Bing or, if you love irony, Google. Please don’t ask me for the document.

Photo Credit: Brent Moore via Flickr

Highlights from the Leaked 2011 Google Rating Guidelines by SEO on Tap

I’ve also been thinking about how Google+ changes SEO and wanted to expand on Chappell’s post with a few thoughts of my own.

Profiling SEOs is Simple Thanks to Google+

It’s not news that Google profiles SEOs and puts their projects under enhanced scrutiny. Google tries to discount viral link building by SEOs while allowing viral links by non-SEOs.

Well, now Google has it really easy. Even if you avoid self-identifying as an SEO in your online presence, if you are on Google+ and a friend puts you in their SEO circle, then you’ve got your own three scarlet letters.

If I were half as smart as I think I am, I’d have a Cafe Press link here selling “Friends don’t add friends to their Google+ SEO circle” t-shirts and mugs.

Google Gets to Find Subject Matter Experts

How does an SEO expert look on Google+? They probably get put in a high proportion of the Circles labeled SEO.

Now when one of these experts uses a +1 button, tweets or links to another document with a related keyword, Google would be well-advised to give that document some more weight.

If I were a betting man, and I am so pony up, Google would give expert opinions extra weight in image and video search until they can do a better job of indexing audio and non-text content.

Of course, if you want to be identified as an SEO expert, you have to contradict my previous advice and identify yourself as an SEO. As an SEO, if you described your relationship to Google on Facebook, it would definitely be “it’s complicated.”

Profiling Authors Becomes Easier

Google has already shown enough interest in giving content creators reputation scores and ranking documents that they have already published a patent on ranking authors.

Let Circles identify the subject matter experts, then combine this information with the agent rank patent and the recent Google-Bing schema.org agreement and you can confidently rank and boost the best authors.

The only real obstacle I see is spammers using schema.org markup to identify their documents as being written by better known authors in the subject area. If you see Google encouraging you to link your own content across domains from you Google + profile, then you know they are struggling with it.

More Anti-Spam

Chappell talks about creating networks of social media accounts to artificially inflate social signals, but there is more to anti-spam than just that.

Setting up multiple sites to manipulate search results also gets a lot more difficult if trusted sites are expected to be owned by a well-networked Google+ account while not associated with the other spam sites.

Google is a long way away from making a Google+ account mandatory because they can’t be dropping small businesses with a minimal web-presence, but if a site smells funny and isn’t owned by somewhat active Google+ account, then there’s a good chance that it is spam.

In fact, I fully expect the next generation of Black Hat SEO to include third-party social media services that employ hundreds of third-world social media account owners to Tweet, +1 and Like so that black/grey sites can benefit from social media signals without being tied to a narrow set of IP addresses or end URLs.

Let me know how you think Google Plus changes SEO in the comments.

More on How Google Plus Changes SEO by SEO on Tap

The great news is that you can see in Google which queries are returning synonyms and use this information to find synonyms that can rank you for the maximum number of key phrases.

Have a look at these results for “configure remote desktop” in Google.

The first thing to look at is the number of results returned (labeled “1″ in the image). If Google returns results in the millions for a long-tail search term, there is a good chance you are looking at search results that have been expanded through synonyms.

The next thing to do is to look for highlighted synonyms (labeled “2″ in the image) in the SERPs. While the example just goes to page 1, you will sometimes need to go deeper than the first page to find query expansion via synonyms.

For this particular query, “enable,” “setup,” and “set up” are all synonyms that Google is using. Because Google is checking for synonyms and it is returning millions of results, this query is being expanded with synonyms. For more competitive queries, you won’t see any query expansion.

The optional third step is to check the Google cache of these pages to see if the synonym candidate appears anywhere on the page. The reason we check the Google cache rather than the actual page is that if a page is ranking for a term because of backlinks, then Google shows a message at the top of the cache page.

In this particular example, the third document is ranking for configure because of anchor text from links pointing to the page. Configure does not appear anywhere on the page hosting the YouTube video and a number of the results in the bottom of page 1 only have the term way down in the text.

All results in this set either have configure on the page somewhere or in links pointing to the page except for the video search results which would suggest that synonyms are even more powerful when optimizing for video.

The next step is to swap out the synonyms in the query to see if we can find a key phrase that is getting enough volume not to need query expansion. What you are looking for is key phrases that are not being expanded by synonyms.

Based on this data, set up remote desktop is the most competitive term as it returns the highest number of pages without query expansion. While it would be advisable to have configure, setup and enable appear on the page, you can rank for these terms with links.

One thing to note, “set up,” “setting up” and “setup” are all highlighted in the “set up remote desktop” and “setup remote desktop” data sets. This is not query expansion via synonyms, but rather an example of query expansion via stemming.

There are few interesting points with this data set. The first is that in “set up remote desktop” SERPs, “set” always appears adjacent to “up” in some form or another.

This can’t always be the case because if they were truly equal, then they would return roughly the same number of documents, but “set up” returns 14,500,000 while “setup” returns less than half the number documents. The reason is likely that documents where “set” and “up” appear separately are also included in the “set up remote desktop” count; however, Google doesn’t show any results past 1000, so it is impossible to check with this query.

One really interesting query that shows how awesome Google is is “set remote desktop up” which properly stems with “set up,” “setting up” and “setup.”

As SEOs, we should note that “set remote desktop up” doesn’t appear as an exact match anywhere in the search results, or at least not in the top dozen or so pages. So not only is optimizing for mis-spelled words no longer effective, but optimizing for bad grammar doesn’t work anymore either.

In the previous Query Forensics post on synonyms, the data used the same number of keywords and better illustrates the pattern of higher-volume key phrases returning fewer results because of the lower-volume key phrases getting expanded through synonyms.

In this data set, “cycling routes netherlands” gets the highest volume of searches, but it returns the fewest documents. So the best target in this set is “cycling routes netherlands” while trails and paths can either be secondary on-page targets or in backlinks.

Key Takeaways

The technique doesn’t work for every query and in many cases I can’t figure out why one query returns more documents than another. However, for sufficiently long-tail and low-volume queries, the highest-volume key phrase is usually the one that returns the most documents while not showing signs of query expansion via synonyms.

Google can detect and fix poorly formed grammar and pages with poorly formed grammar exactly matching the searcher’s poorly formed search will not be returned.

Pages can rank via synonyms if the SERPs page is showing query expansion via synonyms, but you will probably need links with that synonym. It is still better to have the synonym present on the page.

Image Credit: gorbould

Query Forensics: Choosing the Right Synonyms for Long-Tail Key Phrases by SEO on Tap

When Google indexes web pages, it stores both the URLs of pages that contain a particular word and keyword position information. To better understand how exact-match terms and synonyms affect SERPs, we’ll map their positions in the target document similar to how Google might index the document using the following notation: T+ = in title; T- = stem in title; Ts- = synonym in title; B+ = prominent in the body; B- = present in the body; Bs = absent from body, but a synonym is present; url = in URL; A = in backlink anchor text (when not otherwise present on page). PA and DA stand for Page Authority and Domain Authority via SEOmoz.

Google could certainly store more information, for instance separating the keyword in the domain from the keyword URL path or mapping the keyword in hx tags and strong tags, but I don’t want this to get any more complex than it already is. Also, Google has only stated that they store information about where the term appears on a page in its posting lists, query expansion through synonyms could occur separately from the posting list that stores exact-match keyword data.

The first query we look at is Cycling Trails Netherlands (guess what I’m planning on doing for my holidays).

The terms Cycling and Netherlands are obviously very important while synonyms seem to carry as much weight as Trails.

The lone page that is hyper-optimised for this query (http://en.wikiloc.com/trails/cycling/netherlands) doesn’t do very well. I wouldn’t be surprised if this site were hit by Panda and I suggest you look at it as a guide for what not to do, even though it is not as bad as many similarly over-optimized resources.

Substituting Routes for Trails (in a different browser to avoid personalised results from previous clicks) gives Cycling Routes Netherlands and it returns the same top five resources from the initial query, albeit in a different order, and six out of ten resources from the initial query appear again.

“Cycling routes Netherlands” (1,300 global monthly searches via AdWords Keyword Tool) is a more popular query term than “cycling trails Netherlands” (not enough volume to display data). As a result, documents containing the term “routes” are a lot more prominent in the search results than “trails” in the first query.

What Google may be doing, is it is requesting the posting lists for each of “cycling,” “trails,” and “Netherlands;” the query processor sees that trails returns too few documents and then looks for synonyms to include more documents. If this is the case, documents included via synonyms are clearly not penalized for not containing the overly-restrictive keyword.

This doesn’t mean that synonyms are always treated equally, only when one keyword is overly restrictive.

The more common “cycling routes Netherlands” query only returns 129,000 documents. The less common “cycling trails Netherlands” query returns 61,700,000. What this shows is that query expansion only occurs when necessary and the query processor only adds synonyms when necessary (rather than adding them automatically, but not ranking them as well).

As an SEO, you should be checking query volume against the number of documents returned by Google when deciding whether to create a new page and comparing them against other synonymous key-phrases.

If a query has a low query volume but returns a lot of documents, then look for a synonym outperforming an exact-match term in the results and target that term instead.

Based on the higher number of documents returned by the lower volume query, query expansion via synonyms in Google likely works like in the diagram below.

Synonym Query Expansion in Google

The obvious next step is to try to figure out where the threshold for synonym-based query expansion lies and what rules govern it so that we can know when to recommend creating separate pages to target synonymous keywords, when to target multiple synonyms on the same page and what synonym to choose.

Query Forensics: Beginning Look at Synonyms in Google by SEO on Tap

Bold? yes. Interesting? sure. Successful? not a chance.

The problem with both Wolfram Alpha and Yahoo?’s new direction is that if they suck, they’re going to fail, and if they rock, they’re going to fail.

Let’s pretend Wolfram Alpha improves to the point where it works well for most queries and Yahoo? is able to do the same. Let’s even extend the fantasy to the point where both search engines gain traction and a large market share.

What’s going to happen? Most content producers are going to lose the incentive to produce good content.

Search engines crawl the web getting free content, skirting the fringe of copyright by allowing webmasters to block them if they choose, and in exchange they send free traffic to the content producers who are able to make money by selling crap or showing ads for crap.

A successful Wolfram Alpha and Yahoo? won’t be sending the content producers free traffic, they’ll take the best content while preventing the people who create the content from profiting.

Dead-tree media companies already howl about Google crawling their and making it accessible to people even though they get preferential treatment. Stop giving them their cut and get ready to see them unleash the dogs of law before finally going all dinosaur on the world.

While Wolfram Alpha and Yahoo? are fighting, and losing, legal challenges, there’s only one realistic outcome. Content producers are going to block Wolfram Alpha and Yahoo? from crawling their sites.

It may be killing a great and useful idea, but, if someone can kill the electric car, then killing these two seems rather trivial and profitable.

Photo Credit: Moore Fun

Wolfram Alpha and Yahoo!: Destined for Failure by SEO on Tap

Google Crawl Rate

The Importance of Server Speed by SEO on Tap

In case you can’t be bothered to click the above link, it seems that for a short while the G-spot was disclosing whether a domain was penalized in the SERPs when you entered hyves.domain.com.

This is my third site and it has the lowest starting page rank of any site I’ve created: PR 0. I’m curious why.

Of course big G has plugged this evil leak so I don’t know if I’ve been penalized. But I’m left wondering if I’ve been penalized and wondering why.

I haven’t done anything to deserve a penalty. I don’t really believe that I’m suffering a penalty because Google sends me traffic for long-tail terms that I deserve to rank for and nothing for competitive terms that I didn’t expect.

But why can’t I check anyway?

I’d like to know if my early auto industry post which generated spammy, non-reciprocated, track-backs that soon disappeared signalled my new domain as spam.

I originally purchased my name as a domain about 2 years ago with Bluehost.

I then let the registration lapse and Network Solutions started spamming me telling me my-name.com was available.

I was forced to do business with Network Solutions when a domain I wanted was pre-sold through an exclusive domain affiliate deal.

Let’s just say that imo they are part of the problem, not the solution.

What I’d like to know is if I’ve been penalized because Bluehost or, more likely, Network Solutions put up a spammy parking page between when I first registered my name and now.

I’m not bitter at Google. They are sending traffic in volumes consistent with new sites like mine. And they obfuscate Page Rank so it’s possible there isn’t any real penalty.

Still, I’d like to know.

What’s Wrong with Penalty Disclosure? by SEO on Tap

I was looking for some independent information eCommerce in a recession earlier today, so I googled ecommerce recession.

eCommerce Recession SERPS

The first page of results consists mostly of yay eCommerce pages, but I give Google credit for the #3 comScore report on an eCommerce recession. Too bad the author doesn’t know the definition of recession (hint: 2 consecutive quarters of negative growth, not 1 month).

The second page is even more revealing. Most of the posts cite the same two reports, one by Forrester Research and the other by Sandford-Bernstein. Unfortunately I couldn’t find the Bernstein report, and I didn’t feel like registering to maybe find the Forrester report.

As long as search engines use links as a ranking factor, search results for Internet-related queries are going to be biased towards pages touting the benefits of eWhatever because Internet-savvy people are the ones doing the most writing and linking and have a vested interest in promoting content that reminds them of their greatness.

Internet Bias in Search Results by SEO on Tap