The Billion Dollar Question For Programmers Out There

The latest billion dollar question for programmers out there is: who will win web search? Before we begin seeking answers to this question, we need to understand the business of search first.

And that is exactly where I will be beginning the discussion in this essay. Following that, I will be outlining how technology is currently transforming the web search landscape.

Finally, I will be exploring the future of web search and what role programmers can play in it. If these topics interest you, then I welcome you to join me in this discussion.

As I write this essay on a bright and sunny day, I have a lot of research material and personal files (from my experiences) laid out in front of me. All this information took me hours to compile. So, I certainly hope that the end result is worth it. Without any further ado, let us begin.

This essay is supported by Generatebg

The Value of Search

Human beings have always looked for answers to intriguing questions. In the olden days, Kings had the resources to send messengers across the country to gather the answers to their questions. It took days to months.

Pigeons and other messenger birds improved upon this by enabling faster transportation of messages. All of a sudden, people could get the answers to their questions within hours or days.

Fast forward a few hundred years, and human beings had revolutionized communication via networked technologies such as the telegraph, the RADIO, the telephone, etc., These technologies enabled people to get answers to their questions in a matter of minutes or seconds.

Today, whenever someone has an intriguing question, “Let me google it” is the first thought that comes to mind. What did Google, or more generally, the search engine do? It reduced the time delay between question and answer even further.

But why do this? Was the telephone or the telegraph not enough? Well, therein lies the value of search! The web and the internet increased human inter-connectivity by a great deal more than what the telegraph or the telephone did.

The more networked human beings become, the more demand there is for quicker, faster, and better answers to questions. People have questions and they need the best answers as fast as possible!

Whenever someone needs something fast, someone else is ready to deliver it at a price. It turns out that the modern web search is able to provide these hyper-fast, high-quality answers to questions at a price. And this price doesn’t seem to have a limit.

What is the current price for this commodity? And who is paying it? To find out the answers to these questions, we need to look no further than into the books of Google.

The Web Search Juggernaut

Google is currently the undisputed champion in the web search space. There is no question here. But this was not always the case.

Back in 2002, Overture was the search engine company to beat. In the next few years, companies like Excite, AskJeeves, AltaVista fiercely fought for the top spot in web search. With hindsight, we know who won this battle.

But why did Google win? The answer lies in their “PageRank” algorithm. Google optimised search results based on link popularity, which turned out to be a key differentiator. But this was not all.

Competitors often allowed advertisers to display images on their search engine. But Larry Page and Sergey Brin, Google’s founders, never allowed images in Google search results in the early days. The reason? Well, they figured that images take too much time to load.

In other words, they were hyper-obsessed with getting the best quality search results to the user as fast as possible. This obsession coupled with a technological advantage (PageRank) enabled Google to sprint ahead of its completion.

It was only after Google wiped out its competition that it slowly started rolling out advertisements (more on this later). As Google started rolling in profits, the company started investing in moonshot technologies.

The company now has a slew of successful market products such as Android, Youtube, Gmail, Maps, etc. The tech giant has a towering market cap of $1.4 trillion (at the time of writing this essay).

But here’s the thing. Even though Google has this slew of successful products and a diversified project portfolio in the works, web search contributes around 57% of Google’s annual revenue. That amounts to roughly $161 billion.

What Google actually has developed is a (mostly free-to-use) ecosystem of technologies that divert its users time and again to its search product.

Advertisers then pay to trend on top of search results, while content creators compete to feature (and profit from) the most relevant advertisements. In industry slang, we call this kind of a model a money-making-flywheel.

All this makes it sound like Google has built a fortress that will last for eternity. But nothing lasts forever. There are some cracks showing up and it is time for us to look into them.

---

He Who Fights Villains Long Enough Will Become the Villain He Fights

Remember that in the beginning, Google’s competitive advantage was that they didn’t let advertisers dictate terms and slow down search result delivery. But now that the competition is gone (Google commands 91% of search volume as of writing this essay), Google’s search has become bloated with advertisements.

Here’s Ginny Marvin’s breakdown of how Google’s ad-labelling strategy has changed over the years:

Do you notice the trend? Over the course of time, Google has gone from delivering the best quality search results to subtly peddling the best paid results to its search users.

Google does not deliver objectively fast and high quality search results like it used to. What’s more, search engine optimisation has plagued the web with bloated content that beats around the bush.

Have you ever googled the answer to a ‘yes or no’ question only to land on a web page that repeats the same search keyword over several headlines to deliver the answer at the end of 750 words?

How is this a fast answer to an intriguing question? It is safe to say, then, that something has gone wrong with web search over the course of time.

And yet, what alternative do we have? Don’t tell me that you use bing regularly. It’s worse (at least, in its current form). Erik Brynjolfsson, Felix Eggers, and Avinash Gannamaneni asked the research question of how much money users would be willing to accept in order to NOT to use web search and other related products such as maps.

The answer was $3600 for maps and a staggering $17500 for search (I’ve linked the research paper at the end of this essay). Remember the earlier section titled “The Value of Search”. Well, this is it in quantitative terms!

With so much at stake, there is an underdog looking very formidable after all this time. This underdog also seems to be paving the way for more underdogs!

Artificially Intelligent Search

ChatGPT has officially become the fastest digital product to acquire 100 million users (2 months). Just for comparison, it took TikTok 9 months to achieve this feat. So, what is all this buzz about?

Well, it turns out that ChatGPT is web search on steroids. And much more. Here is an example. I asked ChatGPT to help me write JavaScript code for a blog page. These are the results:

Not only did ChatGPT give me valid code, but it went above and beyond with relevant CSS styles as well. Also note that it asked me specific questions to narrow down and give me the best results possible.

All this without frustrating advertisements or keyword-bloated webpages. The price? Well, I did not pay anything for this query. ChatGPT servers are currently restricting access if the servers are at capacity while handling paid traffic.

But if you wish to access ChatGPT at anytime, OpenAI is planning a $20 per month paid subscription model.

The bigger picture here is that this AI-powered-personal-assistant-style search is not limited to programming. It can be extended to any dataset. However, it is not all good news though.

There have been reports of ChatGPT making up results, hallucinating academic papers, and providing plagiarised search results (example source: datanami). While these are certainly concerning issues, none of them are impossible to fix in the long run.

So, what does Google have to say about this?

---

Google is Panicking!

The funny thing about this disruption is that OpenAI would probably not have achieved this feat without the help of Google. Google was the pioneer in the field of transformer-based AI technology.

The company had been an early investor in transformer models. Google published state-of-the-art research material in the field of language modelling using recurrent neural networks (RNNs) back in 2016/2017. Companies such as OpenAI directly benefited from this research and built their technology on top of this knowledge.

So, in a tragic plot twist, Google has potentially sown the seeds of its own demise. But this is a plot twist only if you were Google. For a consumer, this might be something to rejoice about. Google has allegedly issued an internal code-red.

Google’s CEO, Sundar Pichai, has announced a competitor to ChatGPT named Bard. This is telling because Google responded to DALL-E (an image generating-AI model from OpenAI) with a research paper. But since the company is responding to ChatGPT with an actual product, the stakes must be higher.

Where is the Future of Search Headed?

On the one hand, we have the juggernaut that is Google. On the other hand, we have the underdog that is OpenAI. In a master-stroke, OpenAI’s CEO Sam Altman (and Y-Combinator’s former President) has struck a deal with Microsoft to build the technology further.

As part of the deal, ChatGPT features are slowly making their way into bing. Yes, we might just start using more bing in the future. But this story makes it sound like there are just two parties battling it out for the top spot in search.

In reality, the battle appears to be wide open. A couple of folks from OpenAI have jumped ship to start their own AI-search competitor named Anthropic. And guess what? Google is funding this project!

The buck does not stop there. There is a whole host of AI-assistant style search startups popping up. A couple of examples are Perplexity and Andi. So, it indeed appears like the market is wide open.

It is true that many programming jobs will be replaced by such services (to be fair, a whole host of other jobs as well). But if you are a programmer, you should be buzzing with energy and excitement, because the market seems to be opening up for alternative search technologies.

The need for faster and better search results has never been stronger before. Who knows what kind of technologies are waiting to be built on top of these ecosystems?

And there is STILL no limit to how valuable such services could be. So, anyone who is willing to put in the effort to deliver solid search-based solutions is likely to be handsomely rewarded.

Whether Google prevails or falls, only time will tell. But as a search consumer and product person, I can personally say that these are exciting times!

---

References and credit: Mario Gabrielle, and Brynjolfsson, Eggers, and Gannamaneni.

If you’d like to get notified when interesting content gets published here, consider subscribing.

Further reading that might interest you: 

• Startup Ideas: Is There Value In Ranking Creators?
• Entrepreneurs Are The New Economists. Here’s Why.
• Apple Charges More Money For Its Products In Europe. Why?

If you would like to support me as an author, consider contributing on Patreon.

---

JavaScript Code Snippet From Chat GPT:

<div id="blog-container">
  <!-- blog post 1 -->
  <div class="blog-post">
    <img src="thumbnail1.jpg" alt="Blog Thumbnail 1">
    <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. 
       Sed blandit, sapien vel molestie bibendum.</p>
  </div>
  
  <!-- blog post 2 -->
  <div class="blog-post">
    <img src="thumbnail2.jpg" alt="Blog Thumbnail 2">
    <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. 
       Sed blandit, sapien vel molestie bibendum.</p>
  </div>
  
  <!-- and so on -->
</div>

<script>
const blogContainer = document.getElementById("blog-container");
const blogPosts = document.querySelectorAll(".blog-post");

// Show only 10 blog posts at a time
const showPosts = (pageNumber) => {
  const startIndex = (pageNumber - 1) * 10;
  const endIndex = startIndex + 10;
  
  blogPosts.forEach((post, index) => {
    if (index >= startIndex && index < endIndex) {
      post.style.display = "block";
    } else {
      post.style.display = "none";
    }
  });
};

// Show the first 10 blog posts on load
showPosts(1);
</script>

<style>
  .blog-post {
    display: none;
  }
  
  .blog-post img {
    width: 300px;
    height: 200px;
    object-fit: cover;
  }
</style>

//Note: You would need to replace the thumbnail1.jpg and thumbnail2.jpg 
// in the HTML with actual URLs or file paths to your own thumbnails.