The three homes every webmaster needs to visit
As many SEOs may say, search engine optimization is about making you don’t do the wrong things; about making sure that your site is well structured and has all the elements in place so that your website appeals to the three powerful spiders (more powerful and Spiderman!) GoogleBot, Yahoo! Slurp and MSNBot!
But how do you know how the spiders sees your site?
Well, it’ll be good if the spiders would be even half as heroic and eloquent speaking than Peter Parker, coming to your website’s rescue milliseconds before you get into trouble, but unfortunately for us webmasters, the three very powerful bots are kinda deaf and dumb…
Urgh… in fact, they maybe the worst citizens in the web! If they see you in trouble they’d be the first to erase off your web address from their address books, and even go around feigning ignorance of your existence!
How would you know?
You’d have to pay their homes a visit! That is where you can rummage through their diaries and personal log files to find all their thoughts, so you can make the necessary structural changes to your site, before they decide to isolate you from your peers and leave you stranded on a no man’s land.
Shhh… don’t tell anyone we’re plotting this intrusion. Here are their addresses:
For the naughty GoogleBot, go to Google Webmaster Tools.
For Yahoo! Slurp, go to Yahoo Site Explorer
and for MSNBot, go to Live Webmasters
They leave their doors wide open 24 hours a day, 365 days a year.
What should you look out for?
Here are three main things you need to look out for as you rummage through their homes, especially from GoogleBot’s home:
- The times they’ve tried to pay you a visit, couldn’t find you. They knocked on your door a couple of times, and but nobody answered. Most of the time, they write these cases on the 404th page of their diary.
- Check out what their favorite food is! From my knowledge, their favorite dessert is called “sitemap”. Maybe you’d like to serve them with that for start. You can order that from here, or if you have a WordPress blog just ask for the Google XML Sitemaps Plugin
- Top search queries. Find out what it is in your site that the spiders love, so you can work on giving more of that to them.
Go now. Go. So you can make sure your site remain in their address book for a long time to come. Check out what they think about you. You might be surprised… and shhh… don’t tell them I told you this.

Tan Kian Ann is blog marketer based in Singapore. He now works with individuals and small businesses reach out to the world using blogs. 



November 19th, 2008 at 2:13 am
Visit Giovanna Garcia
Hey Kian Ann
Thanks for the reminder about MSNBot-Live Webmasters.
You have a great post.
Giovanna Garcia
November 19th, 2008 at 9:48 am
Visit Kian Ann
Hi Giovanna,
Thanks. Yes, many of us forget about Live Webmasters (me too!), even though many say that SEO is all about getting rankings on Google, we should not miss out the smaller engines.
So long as a search engine has more traffic than you, it makes sense to make sure that the search bot likes your site, right?
November 20th, 2008 at 9:55 pm
Visit Calvyn
Is there any other home we can visit? this 3 i have visit before… I believe still have a lot out there, just cant found it
November 20th, 2008 at 10:10 pm
Visit Kian Ann
Hi Calvyn,
I do not know of any more spider homes, to be frank. Ask.com is the 4th largest Search Engine as far as what i know.
But then again, if I were to want to spend that time, I’d rather be creating a facebook page or myspace or YouTube channel.
Why? According to Alexa (which you might argue that the data is inaccurate), Ask is 48th, and Facebook is 5th.
So I’m better off optimizing for Facebook traffic