Google is so dominant, and has been for so long, because crawling the web is a natural monopoly and Google was the first company that overcame the barriers to entry into the search engine market. Investopedia defines a natural monopoly as follows:
A natural monopoly is a type of monopoly that exists due to the high fixed or start-up costs of conducting a business in a specific industry. Additionally, natural monopolies can arise in industries that require unique raw materials, technology or similar factors to operate.
Google Search has a monopoly on the search engine market based on their control of the natural monopoly that has developed around crawling the web. To understand how crawling the web is a natural monopoly, we will consider two types of costs: The costs to the organization doing the crawling, and the costs to the individual website operators being crawled.
The cost to crawl the web is high for three main reasons.
- A large amount of servers and computer hardware are required to crawl the Internet. Google was a pioneer in using large numbers of cheap, relatively small computers to do web crawling. Its competitors, at the time Google expanded in the late 1990’s and early 2000s, would use large, beefy mainframes to do most of their crawling.
- The labor involved in crawling the web is expensive. This labor is highly skilled, and the amount of work required is very high. Figuring out how to get a lot of cheap, error-prone computers to run more efficiently than custom built mainframes took a great number of smart people working long hours for years. Early on in its development, Google hired as many skilled and talented engineers in multiple fields as possible.
- Access to capital is itself expensive and hard to come by. Google received massive amounts of venture capital from Silicon Valley investors, with $12.5 million coming from Kleiner Perkins in 1999, a year after Google’s founding. The founders of Google were introduced to their investors by their professors at Stanford, an elite school that is expensive to attend and difficult to thrive in.
These high upfront costs represent a formidable barrier to any potential new entrants to the market.