Google Duplicate Content Clarified

Internet, Search

Apparently there are a lot of people that don’t understand what duplicate content is when publishing content online and/or when applied to search engine optimisation. The fine folk over at the Google Webmasters Blog have clarified some points but I’ll recap if you can’t be bothered in the jump.

Google define duplicate content as blocks of text replicated across multiple pages, either within the same domain or across multiple domains. The content does not need to be a direct copy, a similar but slightly different copy would also be considered duplicate content.

Google report that most duplicate content they encounter is served unintentionally. The most common culprits for duplicate content are alternate versions of the same page such as for printing or a mobile device. The other example they cited is an online store, listing the same item more than once on distinct URLs which are actually linked.

Fortunately, Google also advised what isn’t duplicate content. If you were to translate your page using the Google Translate service, then that wouldn’t be considered duplicate content. That does beg the question though, what if I personally translate an article into German, French and Spanish but it doesn’t match what Google translate produced; would that still be considered duplicate content?

What will Google do if they find duplicate content? In the vast majority of cases their preference is towards filtering the content, however they can/do also adjust their algorithms from time to time as well. If you’ve got two copies (normal, printer) available on your site and you aren’t blocking either one; Google will decided which one they think should be listed and not include the others. This is sub-optimal because you could find that Google chooses your printer version over your normal version, eck!

The Google Webmasters have also provided some helpful hints too:

  • Blocking is the most effective mechanism for a webmaster. If you know you’ve got duplicate content and you don’t want to have the wrong versions listed, then block certain copies using a robots.txt file or a noindex meta tag.
  • 301 redirects should be used if you’re restructuring your site. This item isn’t all that pertinent if you’re was a traditional static HTML site, where you would physically move files around on the server. It is however a good point if you’re using a content management system of some sort, where restructuring your site might actually leave the content on the old URL.
  • Link consistently, don’t link to /mypage, /mypage/ and /mypage/index.html – pick one method and stick to it.
  • Use top level domains to indicate regional content if you can. Google will have a better chance of knowing that .de represents a German version of your site over a de subdomain or /de/ in the URL. I wonder how Google handle it if you specifiy using meta tags that the page content is in German?
  • Syndicate carefully by making sure that the syndicated content provides a link back to your original content.
  • Don’t repeat yourself on common things like copyright or legal fluff, just provide a link to a page describing it in depth.
  • Preferred domain feature in Google Webmaster Tools is useful.
  • Don’t publish stubs such as pages on topic X with no actual content on them.
  • Understand your content management product, so you’re aware of what/how it does what it does.

I’d like to know a little more about using Google Translate and translating your content in general. I also think there has to be a nicer way to handle regional sites than simply getting the regional domain as well; for a lot of people that simply isn’t possible.

Testing Your Unit Tests

Programming

Someone in the Ruby community thought it was a good idea to write a piece of software that tested your unit tests.

Heckle runs your standard Ruby unit tests and reports as per normal. Heckle then alters a certain amount of your Ruby source code and then re-executes your unit tests, reporting on the subsequent run against the altered source code. If your unit tests don’t fail after Heckle has mutated your initial source code, then it proves that certain sections of your application either aren’t covered by tests at all or that they aren’t thorough enough.

I don’t know how many people would use a unit testing utility like Heckle, however anything that can be used to increase the robustness of your code is a good thing I think.

WordPress 2.1 Upgrade

Blogging, Software

This evening I went through the process of upgrading WordPress to the recently released 2.1 series.

As a force of habit, I follow the guidelines for upgrading WordPress. Depending on the site of your blog, following the guidelines can be a little time consuming; things like backing up your database an other important files. Regardless of that small inconvenience, I follow it through and I’ve not had a problem during any WordPress upgrades in the past. I’m sure that the day I don’t follow it through will be the same time that I encounter my first problem, so I’ll just keep following the guidelines and live a pain free life.

Once all the formalities of taking backups and unloading plugins was handled, the actual upgrade was painless and very fast. Once the new copy of WordPress was on the server, I went through and reactivated all of my plugins and made sure they were all still working with the new WordPress 2.1 source code and to my surprise, everything was functioning as expected.

If you’re looking to start a simple web site for yourself or a small company, I’d recommend giving WordPress a look for your content management solution. It really is an elegant and extensible platform for publishing that just works.

Out Of Control

News

Yesterday Allen Jasson, a 55 year old IT worker from Melbourne was denied access onto a Qantas flight destined for London because of his shirt. The mans shirt had a picture of George W. Bush, the President of the United States of America with an overlay saying something to the effect of ‘Worlds #1 Terrorist’.

Qantas have stated they refused Allen Jasson onto the flight as they considered it a security risk and that it was offensive to other passengers on the flight. How they managed to take that rather large leap into terrorist, I don’t understand. Unless you were an absolute advocate of George W. Bush, I don’t even think that it could have been considered offensive.

When will the bullshit stop.

PostgreSQL Dynamic SQL & Quote_ident Gotchas

Database, Programming

PostgreSQL provides two useful functions to aid in the safe development of dynamic SQL:

quote_ident
The quote_ident function accepts and returns a text type. Whatever text you pass into quote_ident will be suitably escaped such that you could safely use it as an identifier within a dynamic SQL statement.

According to the documentation, you should always pass table and column identifiers into the quote_ident function for safety. Calling quote_ident('mytable') will return mytable, however calling quote_ident('MyTable') would return "MyTable".

quote_literal
The quote_literal function accepts and returns a text type. Whatever text you pass into the quote_literal function will be escaped so that you can safely use them in dynamic SQL.

You should always pass values or literals into the quote_literal function. By doing so, all special characters such as quotes will be safely dealt with. Calling quote_literal('mycolumn') would return 'mycolumn' whilst quote_literal('my\'column') would return 'my''column'.

Both of these functions work a treat, however there is a caveat with the quote_ident function which isn’t well documented. When creating objects in PostgreSQL, they object names are automatically lowercased unless you create the object using double quotes. As a simple example:

  1. CREATE TABLE MyTable (id integer, name varchar); would result in an object mytable being created.
  2. CREATE TABLE "MyTable" (id integer, name varchar); would result in an object MyTable being created; note the casing.

Now lets assume you wanted to create some dynamic SQL to fetch information out of the first example table above. If you issued quote_ident('mytable'), your dynamic SQL statement will execute because the value returned from quote_ident is lowercase which matches the table name. If you called quote_ident('MyTable'), your dynamic SQL statement will report an error stating that it cannot find the table or relation.

Creating dynamic SQL in PostgreSQL to fetch data out of the second example above, you would run into the reverse scenario. Issuing quote_ident('mytable') would produce an error, while quote_ident('MyTable') would execute without error.

If you create your database objects without using double quotes, then it’s important to remember to not pass capitalised parameters into quote_ident. The opposite is of course true as well, if you create your objects using double quotes then you must remember to pass in the same casing to quote_ident. If quote_ident applies double quotes (be it from capitised letters, spaces or special characters), the SQL engine within PostgreSQL will assume that an object exists with the explicit name matching the returned value of the quote_ident function.