History of Search Engines

Search engines have undoubtedly changed the way we find and process information. People spend their days searching and adding to the rich databases that exist on the Internet. What we seek is almost always at our fingertips in a swift and timely process. Search engines, the universal mediums of receiving instant information, are considered to be concentration blockers for some and shapers of thoughts as well as ideas for others. Not only have search engines impacted our access to information, they are also at the forefront for helping businesses grow and succeed.

How Search Engines Work

Search engines are generally "crawler-based," meaning that they "crawl" the Internet for web pages, scan the information, and follow the links to other pages on a given website. These search engines operate using algorithms. What is found on the web pages is then stored in the index, or a catalog, of a search engine. If there are any changes to a site, the index is also updated.

When people search for key terms on a search engine, they will receive the results based on what the engine has crawled and stored. Search engines are programmed to sift through the stored pages and find matches to the specific terms. They will simultaneously rank these findings in order of what is most relevant. How a search engine ranks a website is a multifaceted process that ultimately impacts the popularity and success of the site.

The Early Days of Search Engines

Although the history of search engines is a short one, the amount of advancements search tools have experienced is tremendous. Archie, the very first search engine was developed in 1990 by a McGill University student named Alan Emtage. Archie, which is short for "archives," was designed to search and store directory listings of all files on file transfer protocol (FTP) sites. This ultimately created a database of filenames. The amount of content was minimal but could conveniently be searched.

A similar search tool called Gopher, which indexed text files, was created a year later. These files then evolved into web sites once the World Wide Web was established. Two new search engines called Veronica and Jughead were created and worked hand in hand to search the files in listings in Gopher. On the one hand, Veronica provided a keyword search of menu titles in Gopher while Jughead gathered information from Gopher servers. While these early forms of search engines did not utilize the major features of search engines we are familiar with today, such as crawling and indexing, they certainly helped pave the way.

The First Crawler-Based Search Engine

JumpStation is considered the first world wide web resource-discovery tool that operated using the process of crawling, indexing and searching. The resources on JumpStation's platform, however, were few in number. Only titles and headings on web pages were searched and indexed.

This led to the development of WebCrawler in 1994, which was the first actual crawler-based search engine that reached the masses. For the first time, users could search for any keyword and obtain results. After this development, Lycos entered the picture and provided users new features including results that rank according to relevancy.

The Branding of Search Engines

Once Yahoo!, the brainchild of Stanford students David Filo and Jerry Yang, was launched back in 1994, other developers began to vie for the spot for the next best search engine. Filo and Yang were interested in keeping track of their favorite links on the web, which led them to create lists and organize the items into categories and subcategories. This influenced their idea of a search engine and their aim was to help users find web sites related to their interests. Filo and Yang reached their goals and built a strong and growing audience in the small community of Internet users.

The two Stanford students were not alone in their ambitions. As Yahoo! gained in popularity, AltaVista was developed and started to grow as well. AltaVista, which means "a view from above," is another advanced crawler-based search engine that launched in 1995 and competed with the rest. Today, AltaVista is actually owned by Yahoo! This search tool was developed by researchers at Digital Equipment Corporation and the goal was to make it easy for people to find information on the web. It became successful with its advanced crawler system, which crawled an exhausting number of web pages.

In the meantime, a different group of Stanford University students planned and developed Excite, another significant brand in the world of search engines. They received funding from International Data group to develop this new tool. A year later, the next search engine giant that we currently know as Ask, was launched. AskJeeves was the former name of this search tool and was founded by Berkeley students Garrett Gruener and David Warthen. Similar to Excite, Ask received funding from various venture capital firms and grew as a portal for allowing users to obtain answers for everyday questions.

The Evolution of MSN Search

In the same year that Ask was created, Microsoft's Advanced Technology Group conceived the idea of The Microsoft Network (MSN). MSN was included with Windows 95 and users could take advantage of the network for news, weather, email, chatting, message boards and product support. However, users were unable to utilize MSN as a search tool initially.

With the growth of the Internet, Microsoft responded by launching MSN 2.0, which provided access to the Internet as well as multimedia content through the MSN Program Viewer. Microsoft refrained from offering a search feature through MSN, but with the development and growth of Google and other search engines, MSN participated in the competition with the launching of MSN Search in 1999.

MSN later evolved into Windows Live in 2005 in hopes to improve the overall services and features of the company. MSN Search was impacted by the change and was replaced with Windows Live Search and later Live Search. The evolution of the search engine did not end there. In fact, in 2009, Microsoft and Yahoo! made a deal to replace the Yahoo! search engine with Bing. Yahoo! Search, however, will continue to be its own user interface, but will be branded as Bing. The goal of this merger is largely considered to be a response to Google's growth and popularity as a search engine.

Google Opens New Doors

Google, the giant of search engines, made its breakthrough on the web by introducing a new system for search rankings. Rather than ranking results according to the number of times a search term appears on a page, Google ranks websites based on their relevancy and relationships to other websites. This new system is called PageRank and is heavily dependent on the number of pages that link back to a website and the value of those pages. This completely changed the understanding and perception of traditional search engines.

Although Google now offers many services, including advertisement programs that account for most of the company's revenue, the Google search engine is the most popular service to date. Trillions of web pages are indexed by Google and an emphasis on quality and relevancy is at the core of its service.

New and Alternative Search Engines

In addition to these widely known search engines, the web has also seen a handful of new and alternative search engines that may become just as successful. One of these search tools is Wolfram Alpha, which is a service that provides factual answers to questions. Unlike other search engines, Wolfram Alpha does not provide web pages with the answers but rather actual computational information.

Cuil, which is pronounced like "cool," is yet another example of an alternative search engine that provides web page results organized by content and even offers thumbnail pictures for the results. Unlike other search engines, Cuil does not store a user's search activity. However, this new search tool has an index of about 128 billion web pages and is managed by former Google employees as well as IBM.

These new search engines and the rapid growth of the Internet provide a glimpse into what may come in the future. The desire for developing and growing another great search engine is still alive, but advancements on the web pose questions and challenges for the search engines that are currently dominating the scene. How these search engines will respond to new changes and still provide the experiences users are demanding is something worth watching.