Nowadays it’s really hard to believe that there were times when search engines were not around or at least were not intensively used. In the age of Early Web, search results were often irrelevant and heavily influenced by spammers and big budgets from corporations. I’m not saying that it’s ideal now because if you invest lots of money in a proper way, it will give positive results in our day and age as well. but back in the day it was really cruel. 🙂 I’m gonna outline a few very important milestones in the development of SEO so that you can better understand why search engines work nowaday as they do.
The pre-historic age of search engines
As you may know, Tim Berners-Lee invented the World Wide Web back in 1991. And the very first online robot — the World Wide Web Wanderer — was launched in June 1993. The original task of the first web bot was to count the number of the servers connected to the network. Then it also started capturing actual URLs and added to the database that was entitled Wandex. The very first service that looks like today’s search engine—named the Repository-Based Software Engineering (RBSE) spider—was launched in late 1993. It was the first index that allowed to search based on relevance.
Then in 1994 the Yahoo Directory came to life. In the very beginning, it was just a collection of its inventors (David Filo and Jerry Yang) favorite links. Once it grew up, they decided to add more functionality. So, now users can search the Yahoo database for relevant sites they needed.
Lycos, Infoseek, and AltaVista used to perform the function of what Google and Bing do today. Sure thing, they were not that sophisticated or something.
The modern age of search engines
The mid to late ‘90s showed a huge growth of the Web. A slew of web pages were being created and indexed by search engines. By late 1996, Lycos had already indexed about 60 million web pages of content. The more content emerged the harder it became to provide the searcher with relevant content. The things started to look down.
The search engines of those times were capable to give reliable and relevant results when there were 10,000 pages but when the number grew to 100 million it all got to be another story altogether.
Two PhD students (Sergey Brin and Larry Page) at Stanford University (in 1996) started working together on a search engine they called BackRub. The advantage of BackRub was that it used sites that were pointing to a specific site to figure out which site is more relevant for a specific search query. They borrowed the basic idea from scientific books. They noticed that if a particular book is mentioned in a few books of a certain area of science, it means that book has a bigger authority. And they just were smart enough to implement the same idea, but in their search engine.
Sergey Brin and Larry Page launched Google in 1998. The engine had the idea of PageRank as its basis. Step-by-step. Year after year. Google started to finally dominate the industry of online search. Its dominance got really obvious in 2000.
In 2002, Google launched its AdWords, which allowed to display ads on the CPM basis. It steadily evolved to the modern cost-per-click (CPC) auction system.
Since 2003 all the major search engines were working pretty much the same way. I mean they used the same main idea. They were indexing web page content, and serving ads alongside search results. And even now the main search engines (Google, Bing, etc) have extremely similar search result page layouts.
The major search engines of today understand the meaning of a phrase and its context. As you can see the whole shebang got a bit more complicated. They don’t just search for the keyword that is mentioned multiple times (as they used to do). As a matter of fact, there are about 200 factors that influence the Google algorithm that is being used for ranking sites in a certain way. According to Google they do it for better user results but there are other explanations for why they do it that often and what actually it’s done for. Like to attract people to their own advertising services, for one. 🙂
On May 28, 2009, Microsoft launched Bing. I’m just wondering what took them so long to figure out that they need to do it. As we know, this company is notorious for its slow reactions to things in the industry. 🙂
As of now there are 3 main players in the nowadays SEO: Google, Yahoo, and Bing.
The bottom line is that Google is the king of the Web kingdom now. And it’s gotten way more complicated that it was in the very beginning.
To be continued …
