For many people, most of these things will be old hat. However, this post is just intended to recap four necessary steps for any webmasters trying to get their Web sites indexed by various search engines.
Create a Site Map
First and foremost, you must have an XML sitemap. Almost all search engines (certainly the three big ones) utilize a sitemap to become aware of all pages on your site. If you have good link structure, a sitemap might not be necessary, but, it doesn’t take much effort to put one together, and the benefits could be fantastic.
Many content management systems (CMS) will automatically generate a sitemap for you, but, if you aren’t using a CMS that does that, you have a few other options. If your site has an auto-generated RSS feed, that can be a good place to start. Most search engines will treat an RSS feed the same way an XML sitemap is treated, so that can make a big difference.
If you don’t have an auto-generated sitemap or RSS feed, there are quite a few tools you can use to create a sitemap; but you’ll have to make sure you update it each time your site architecture changes. A site called “XML-Sitemaps” will crawl your site and create a sitemap for up to 500 pages for free.
Google Webmaster Tools
The next thing you need to do is sign up for a Google account (if you don’t already have one) and set up a Webmaster Tools account. Within that interface, you can submit your Web site to Google to make sure it’s being indexed, submit an XML sitemap to help Google keep track of your pages, set your preferred address (“www” or not), indicate whether your site is targeted at a specific geographical region, make sure that your site is configured properly to allow and disallow robots in the appropriate places and much more.
You can also keep track of how many pages on your site have been indexed by Google, see how often Google crawls your site, view any errors that Google’s crawler encountered, see where specific keywords rank your page, see how many pages on the Web are linking to pages on your site and how many links your site has to other pages on the Web.
One of the best tools in the Google Webmaster Tools is the “HTML suggestions” tool. This tool gives you a great deal of information about the title tags and meta descriptions on your site, including information about duplicates, missing items, items that are too long or too short and more.
Yahoo! Site Explorer
For the time being, Yahoo! is still a big player in the search game. That may change with the recent acquisition of Yahoo! by Microsoft, but we’ll just have to wait and see. Until that time, though, you should use Yahoo! Site Explorer to make sure your Web site is being indexed properly by Yahoo!’s search engine. Yahoo! Site Explorer offers some of the same tools offered by Google Webmaster Tools along with a few different ones.
Bing Webmaster Center
One more account you need to set up is with Bing, the new search engine from Microsoft. Once again, you can use many of the same types of tools that you would find within the Google Webmaster Tools. One nice thing about the Bing Webmaster Center is that you can view a “score” for your domain and for each of the pages that have been indexed by Bing. You can read a little more information about ways to optimize your site for Bing by reading this article over at Techgeist.