The AI Gold Rush for Infrastructure Companies

In the age of Generative AI, if developers are the miners, it's the tools like LLM and Vector Databases that help them strike gold.

“Robot wearing blue jeans in a gold rush town, old school style” - DALL-E

There is no shortage of talk around Generative AI and the ways this technology will transform industries from early detection of cancer to helping fight cyber attacks in smarter and bolder ways. While many are talking about the exciting possibilities, AI applications are also having important discussions around the ethics of it all, but today I want to focus on all the infrastructure technologies that are being created, updated and re-imagined to help fuel this generational technology shift.

As a VC, I invest in and see how infrastructure companies accelerate new waves of applications and rise to become some of the most valuable technology businesses around. 

I see technology infrastructure as akin to the Gold Rush. While some struck gold, other big winners during this era were those who supplied the miners with equipment. In the age of Generative AI, if developers are the miners, it's the tools like LLMs and vector databases that help them strike gold in place of pickaxes. 

It’s Still Early Days in This Infrastructure Race  

While there is a tremendous buzz and companies are being built around generative AI, we’re still in the very early days of adoption. While a few companies like Microsoft, Google, Meta are innovating and adopting generative AI in production, most companies are still in the phase of exploring foundational models to enhance existing products and workflows. Content summarization, email generation and conversational interfaces are some areas that have been the focus of these early efforts. 

Some companies, like Quizlet, Aisera and Notion, have begun efforts to embed Large Language Models in their core products for end users, but for the majority of the companies, this is still a futuristic scenario. The requirements of enterprises are complex and even though there is excitement around generative AI, there is also skepticism and apprehension around using it in production workloads. There are questions and concerns around its reliability, compliance and safety. 

The Enterprise Will Shape This Market Like All The Ones Before                                                                      

Enterprise stakeholders have strong requirements around SLAs - they need reliability, accuracy and robustness. In order to see adoption of LLMs in production workloads, more clarity is required from its outputs. There is a need for effective guardrails and usage framework to close these gaps. 

As the stack for deploying LLMs rapidly evolves, there will be a transformation in the infrastructure for software development and for data due to LLMs.  It remains unclear whether LLMs will augment the classical data science and development workflows or replace them. Foundational models have massive compute and storage requirements. Add to it other requirements like robust security controls, super low latency and operational simplicity - and we have a set of requirements that has the potential to disrupt multiple layers of the stack.

An example of this are Vector databases. Since the release of ChatGPT, Vector databases have come into focus as the database of choice to support AI applications. Traditional methods of managing unstructured data are time-consuming and rely on manual tagging with keywords and labels. Vector databases use a concept called embedding vectors - where, rather than matching on specific keywords, one can search for concepts and get data that is most similar to the concept. 

Similar to this, we will see specialized solutions emerge, optimized for various aspects of AI applications, creating opportunities for new players to provide “native” solutions for these new use cases. All of this creates opportunities for entrepreneurs to build strong and impactful companies as these use cases surface and harden up. Like other major technology shifts, it’s often the infrastructure front runners who become the leaders just like Levi’s jeans during the gold rush days.