I appreciate that it might be difficult to manage the problem, however if a site like Squidoo can’t get it under control then it completely erodes the usefulness of the site in my opinion. The Squidoo spam problem has been happening for quite some time and the owners are aware of it, however it seems that they have yet to find a way to curb it.

It’s a shame really, as I hate spam so much that it has now tainted my opinion of Squidoo as a useful service. Fortunately, I find the site pretty much useless, so at least I don’t have to put up with it while browsing their site.

Related posts:

1. Offline “Home Business Opportunity” Spam
2. Winning The Fight Against Spam
3. Snail Mail Spam

• http://domain.com/mysuperdooperproduct.html
• http://domain.com/my-super-dooper-product.html
• http://domain.com/my_super_dooper_product.html

For quite some time now, search engine optimisers all over the internet pondered whether using dashes versus underscores was the best performing technique. It wasn’t long before it became clear that using a dash as a word separator out performed the humble underscore.

Matt Cutts, a Google engineer has stated at WordCamp 2007 that very shortly Google will support an underscore as a word separator as well as the existing dash. Although the difference between the two seems so subtle, for many web sites it has proved a significant thorn in their sides as their content management systems produced URL’s which utilised an underscore and not the recognised dash.

This news is going to make a lot of people very happy in the near future.

Related posts:

1. Search Engine Optimisation (SEO): Dashes Versus Underscores
2. Search Engine Optimisation (SEO): Demystifying The Black Art
3. IIS & Invalid Folder Names

At the moment, there isn’t an option on the advanced search page to restrict images to that imgtype, so you’ll need to add it into the URL manually.To give you an idea of what it might be useful for, compare to two search results:

• Alistair Lattimore (standard)
• Alistair Lattimore (faces)

The first set of results are anything that has been associated to ‘Alistair Lattimore’, where the latter is meant to filter the results to contain faces. In that particular example, it isn’t perfect however it’s quite easy to see where this is going. This new functionality is apparently the by product of a Google acquiring Neven Vision last year who were developing specialist facial recognition software.

Related posts:

1. Google Image Labeler Included In Google Webmasters Tools
2. Google Search Revolutionised Through Vertical Integration
3. Search Engine Image Traffic Recovering

Google have supported removing content from their service for a long time, however the process was often slow to take. With the recent addition of the URL removal into Google Webmasters Tools, its now possible to process the removal of a page quite quickly.

As with everything associated to Google Webmaster Tools, the web site to act on first needs to be verified. Once verified, there is now a URL Removals link under the Diagnostics tab. The removal service supports removing URL’s in the following ways:

• individual web pages, images or other files
• a complete directory
• a complete web site
• cached copies of a web site

To remove an individual web page, image or file – the URL must:

• return a standard HTTP 404 (missing) or 410 (gone) response code
• be blocked by the robots.txt file
• be blocked by a robots tag

Removing a directory has less options available, it must be blocked using the robots.txt file. Submitting http://mydomain.com/folder/ would remove all objects which reside under that folder including all web pages, images, documents and files.

To remove an entire domain from the Google index, you need to block it using a robots.txt file and submit the expedited removal request. Google have once more reinforced the point that this option should not be used to remove the wrong ‘version’ of your site from the index, such as a www versus non-www version. To handle this, nominate the preferred domain within the Google Webmaster Tools and optionally redirect the wrong version to the correct version using a standard HTTP 301 redirect.

Cached copies of web pages can be removed by setting the <meta> robots attribute with a noindex on the given page(s) and submitting the removal request. By using this mechanism, Google will never re-include that URL so long as the robots noindex <meta> data is present. By removing the robots noindex <meta> data, you are instructing Google to re-include that URL, so long as it isn’t being block by alternate means such as a robots.txt file. If the intention is to simply refresh a given set of web pages, you can also change the content on those pages and submit the URL removal request. Google will fetch a fresh copy of the URLs, compare them against their cached copies and if they are different immediately removed the cached copy.

After submitting requests, it’s possible to view the status of the request. They will list as pending until they have been processed, denied if the page does not meet the removal criteria and once processed they will be moved into the ‘Removed Content’ tab. Of course, you can re-include a removed page at any time as well. It should be noted that if you remove a page and don’t manually re-include the web page(s) after exclusion, the removed page(s) will remain excluded for approximately 6 months – after which they will be automatically re-included.

Being able to remove content from the Google index so quickly is going to come in handy when certain types of content are indexed by accident and need to be removed with priority.

Related posts:

1. Google Duplicate Content Clarified
2. Search Engine XML Sitemap Improvements
3. Google Malware Warning

Google universal search is going to allow you to search, as you did before with the familiar single search box; however many additional sources will be used to formulate the search results. As most people are aware, Google houses many different indexes of information:

• web sites
• news
• books
• local
• images

which have been available to internet users through different search locations such as http://www.google.com or http://news.google.com. While separating out various types of search information into different web sites might have made sense from a development and technical level initially, Google were not leveraging their various indexes to their potential. Even with the initial release of the universal search service, I’m sure there will be significant improvements to come in the near future.

The key to the Google universal search is that their disparate search indexes have been vertically integrated. For those that aren’t aware, vertical integration typically refers to taking totally separate sets, be it a business, process or data and combining them into a single unified service. By removing the barriers between their various search indexes, Google have knocked down the information silos they helped build during development.

To the average user, this will mean they are more likely to find the information they are looking for on the Google home page. When a user searches, results will be returned from various sources and combined based on relevance. It will now be common place to see:

• web sites
• news
• books
• local
• images
• video

all within a single search results page. Of course, it is unlikely that a search would return results from all indexes at the same time. After all, the algorithms are looking to return the most relevant content to the user – not the most sources. As such, if the algorithms deem it appropriate then you may only see web and image results with no video or book content.

This is an exciting space and it is going to be interesting watching how the search engine optimisation landscape changes now that Google universal search has been released into the wild!

Related posts:

1. Google Image Search Supports ImgType=Face
2. Google Blog Search Accepting Pings
3. Yahoo! Search Marketing Launches Sponsored Results

Being the diligent computer user, I uninstalled MSN Messenger 7.5 and the original Windows Messenger that comes with Windows XP Professional. Not knowing the web address for MSN Messenger, I googled msn messenger to be presented with the search result to the left.

After glancing at the advertisement and seeing “Msn Messenger” as the advertising text, I clicked the link expecting to be taken to the Messenger home page on the Microsoft web site. No, that isn’t what I got at all – instead it redirected me to the new Microsoft Live Search web site, with my “MSN Messenger” search already performed. Not only that, they had a nifty JavaScript sliding panel with some useful advertising promoting Microsoft Live Search and telling me that it is “the ducks nuts”. After a few seconds, the useful advertising panel automatically slided away to leave the standard Microsoft Live Search page.

When the biggest software company in the world is required to participate in pay per click advertising on a competitors network to drive traffic to their own search engine, I think it is a pretty sure sign that their competitor is doing something right. I can understand that someone like Google and Yahoo! might advertise on their competitions web sites for pay per click marketing services but I’m yet to see an advertisement on Google or Yahoo! telling me that I should be using their competitors search engines.

Related posts:

1. Microsoft Invests USD$240 Million In Facebook
2. Google, Yahoo! & Microsoft Collaborate For The Greater Good
3. Yahoo! Search Marketing Launches Sponsored Results

Implementing an XML sitemap for a web site is a simple way for a webmaster to inform the search engines what content exists on their site that they absolutely want indexed. The XML sitemap does not necessarily need to include all content on a site you want indexed, however the content that exists within the XML sitemap is looked upon as a priority for indexing.

When the XML sitemap protocol was initially released by Google as a beta, webmasters needed to inform Google of its existence through the Google Webmasters Tools utility. When Yahoo! and Microsoft joined the party, all vendors accepted a standard HTTP request to a given URL as notification of the XML sitemaps location. These methods have worked fine, however required a little bit of extra work for each search engine. It was recently announced that you can now specify the location of the XML sitemap within a standard robots.txt file.

It’s a small change to the robots.txt file, however it’s an improvement that makes so much sense since the robots.txt file is specifically for the search engine crawlers. If you want to use this new notification method, simply add the following information into your existing robots.txt file:

• Sitemap: <sitemap_location>

It is possible to list more than one sitemap using this mechanism, however if you’re already providing a sitemap index file – a single reference to the index file is all that is required. The sitemap_location should be the fully qualified location of the sitemap, such as http://www.mydomain.com/sitemap.xml.

Related posts:

1. Search Engine Image Traffic Recovering
2. Search Engine Optimisation & Referral Tracking
3. Google Blog Search Accepting Pings

Traffic to your web site is generally a good thing, no matter how it gets to your site. Unfortunately, the cost of being prominently placed within the search engine results had a downside and my site exceeded its monthly bandwidth allocation. I contacted my web host and they graciously re-enabled my account for the rest of the month.

The table below shows the monthly image search referrals. As you can see, the search engine image referrals double between December 2005 and the following month. The referrals continued to increase steadily, until half way through April it started to jump again and by May it was completely out of control, saturating my monthly bandwidth allocation. In case you’re wondering what or who was the culprit, it was a whole swag of particularly lazy MySpace folk.

To make sure the bandwidth theft didn’t happen again, I took some pretty drastic measures by blocking all search engines from indexing my /images/ folder using the robots.txt file and implemented hot link protection via the .htaccess file. By half way through October 2006, the change had fully kicked in, dropping my monthly image search referrals from over 50000 to under 100 per month.

Since the mania surrounding Chuck Norris faqs has subsided, I decided that it was time to remove the heavy handed restriction placed over my /images/ folder. I’m currently allowing everyone to index everything once more and am even participating in the Google Image search beta, which can be enabled through the Google Webmasters Console.

The restrictions were removed at the end of March 2007 and half way through April, the search referrals are already on the increase again. Once this month is finished, I would expect to have approximately 4400 search referrals, which is back inline with where the site was in February/March 2006.

Onward and upward I say.

Related posts:

1. Google Image Search Supports ImgType=Face
2. Search Engine XML Sitemap Improvements
3. Search Engine Optimisation (SEO): Demystifying The Black Art

Physically setting up an affiliate program is quite straight forward and you have two choices in handling the inbound links and referral tracking:

Single entry point

Using a single entry point in your site, where everyone links to with their respective referral code which then shunts the user to the desired page. Using this method, you might end up with:

• http://mydomain.com/tracking.php?ref=abc&destination=1
• http://mydomain.com/tracking.php?ref=abc&destination=2
• http://mydomain.com/tracking.php?ref=abc&destination=3

Multiple entry points

Allowing multiple entry points facilitates deep linking. If you allow multiple entry points, you might end up with:

• http://mydomain.com/page-1/?ref=abc
• http://mydomain.com/page-2/?ref=abc
• http://mydomain.com/page-3/?ref=abc

Both of these methods will work but which one is better for search engine optimisation? If you use a single entry point, you end up in a position where you’ll have hundreds or thousands of inbound links to a particular page. Unfortunately, the page that they are linking to isn’t useful to a search engine for indexing – it simply redirects to another page. You do however get the benefit of being able to effortlessly reorganise a web sites structure and only have to worry about updating destination URL’s in a single location.

Using multiple entry points allows your marketing or affiliates to link directly to their intended page with their referral code, which can make a difference on various levels:

• it’s convenient for the people linking to the page
• it’s less error prone, as the linker can simply copy the URL from the browser
• the linked URL will begin to gain inbound links, which is critical for effective search engine optimisation
• the person clicking on the URL can hover the URL and see where it is going

The last point might seem like something you might otherwise gloss over, however as internet users become more savvy – they are becoming acutely aware of their online actions. Letting the user clicking on the link see the destination URL will help build trust between your web site and them, as they will be less inclined to think the link is spam.

My personal preference is towards deep linking, it’s just so convenient. If you allow deep linking, the next problem you have is your affiliated links making their way into the search engine result pages; which is definitely not what you want. Fortunately, through the use of a robots.txt file it is possible to drop the affiliated URL’s from being indexed. In the above multiple entry point example, you could stop those URL’s from being indexed by including the following line into your robots.txt file:

1. User-agent: *
2. Disallow: /?ref

Unfortunately, your work isn’t quite done, as all of the inbound links are linking into distinct URL’s (ie, with the referral code). As far as a search engine is concerned, these are totally separate web pages which could/should have unique content. To leverage the most out of your inbound links, you want to make sure the link ends up pointing to the permanent URL for the content (ie, without the referral code).

Remembering that you are tracking referral codes, the web site must first do something useful with the referral code. Useful might be placing the referral code in a cookie for later use or storing it in a database, but something generally needs to happen with it. Once the useful action has been completed, you need to send a standard HTTP redirect to the user agent (browser, bot, ..) to tell it the permanent URL for that content exists at a different URL – in this case the same URL without the referral code. Consult the documentation for your favourite server side language about handling HTTP response codes.

By implementing these two simple techniques, you now only have a single copy of any of your web pages indexed in the search engines and any inbound referral links will ultimately be attributed to the permanent URL for the actual content.

You can now sleep easily at night knowing you have search engine optimised referral tracking.

Related posts:

1. Search Engine Optimisation (SEO): Dashes Versus Underscores
2. Search Engine Optimisation (SEO): Demystifying The Black Art
3. Search Engine XML Sitemap Improvements

The game aspect of the Google Image Labeler involves a pair of people. The contestants are chosen at random to play against one another based on who is online at any point in time. Each game lasts for 90 seconds and the contestants are shown the same series of images which they have to tag or describe with words or phrases. The contestants gain points when they match words or phrases with their opponent.

By gaining points when you match words with your opponent, Google are assuming both contestants consider the image to reflect the same object. At some point, Google will end up using this information in Google Images to provide a better quality of service to their customers.

The service aspect of the Google Image Labeler is of course about providing a higher quality of service to the Google user base. At the moment, Google rely on webmasters providing context around any images that they use on their web sites. As a simple example, a webmaster might:

• provide a meaningful name for the image
• provide a useful alt attribute, which describes the image in text format
• provide captions for the image, which might be a more in depth text description of the image
• talk about the image in the main content on the web page

Whilst this mechanism is very useful and in most cases accurate, it can also be inaccurate or abused. By relying on random Google users to categorise the images, the chances of an image being misrepresented are vastly reduced.

Having humans categorise the images also lends itself to Google producing software that learns how to recognise images. Google could attempt to identify what the images are on their own and use the tags or labels provided by the Google user base to essentially compare or validate the results.

When logged into the Google Webmasters console, you are now able to select whether or not you want the images on your site to be visible to the Google Image Labeler service. At this stage, I’m not quite sure why you would opt out of it – however Google are giving webmasters the option should they choose to.

If Google do end up walking the learning machine path, it could be interesting times ahead for the image searching service.

Related posts:

1. Google Image Search Supports ImgType=Face
2. Google Webmasters Facelift
3. Google Adsense & Text Link Ads Irony