Mediarun Search Marketing News

Search engine marketing is ineffective if crawlers are blocked

July 1st 2010

Search Engine Optimisation News

Search engine optimisation has been let loose on the whole of the world wide web in recent times. By this we mean that the days of posting search engine-friendly content solely on your company's website are over. As web 2.0 technologies continue to develop and integration of multiple platforms becomes a necessity, marketing professionals need to widen their search engine marketing net.

Whether it is using search engine optimisation techniques for content posted on Facebook, releasing exclusive videos on YouTube or using a corporate blog for website optimisation purposes, companies need to formulate a multi-pronged search engine marketing attack if they are to be successful and boost web traffic.

And as well as using multiple platforms to promote their search engine optimised landing pages and blog content, companies also need to pander to multiple search engines. This means search engine marketing should be aimed at Bing, Yahoo and others - as well as market leader Google.

However, some SEO companies may claim to carry out website optimisation for every search engine on the internet. A valiant claim, but a misguided one. It may be worth tracking down the odd users of Wolfram Alpha if you create mathematical graphing software for research departments, but for the majority of businesses, this depth of search engine marketing is unnecessary.

In fact, a good barometer of who your search engine marketing material is targeted at - in terms of search engines - is Facebook's robot.txt. This has a complete list of search engines that are allowed to index the social networking website. And it is very limited. Prior to the implementation of a robot.txt, Facebook was open to anyone, so any search engine could index the site.

Facebook chief technical officer Bret Taylor said: "We will white-list crawlers when legitimate companies contact us who want to crawl us."

As Facebook is the second most popular site on the internet, it makes sense to include it in any search engine marketing strategy, but because Facebook is intent on limiting the access routes to its site it is also a company to watch in terms of which search engines your marketing content should be aimed at.

To find out about the performance solution we can offer you and your business, speak to one of our consultants on 0207 843 2265 or contact us.