As the macro environment remains sluggish, only two sectors continue to thrive in the primary market: generative AI and cyber security. Shortly after our research was published, Wiz, a cloud security company, concluded another round of financing, pushing its market valuation to $10 billion.

"AI security" represents the intersection of these two sectors, but discussing it can be very painful at present. The safety of LLMs, the security of models and their usage, and the implications of LLMs' development for "traditional" cybersecurity are often conflated. Therefore, we put forward a framework in this article to differentiate these three factors.

In addition, individuals with limited knowledge of cybersecurity (and LLMs) may be concerned that LLMs will directly phase out existing security companies. However, the security capabilities of Microsoft Secure Copilot are still contingent on existing security products. The recent downtime experienced by OpenAI due to the Redis bug serves as a reminder that even the most proficient LLM-focused company still needs a comprehensive cybersecurity stack to protect itself.

In the body of this article, I will primarily focus on the short-term opportunities that LLMs bring to "traditional" cybersecurity companies.

Take a very straightforward opportunity for example. Most people may have come across the news about a Samsung semiconductor engineer feeding confidential data to ChatGPT. While ChatGPT did not directly train on this data, it is inevitable that the data leaked onto OpenAI's servers. This is by no means an isolated incident, as every company faces significant risks of data breaches. Some large enterprises have opted to rely on SSE/CASB vendors such as Netskope and Palo Alto Networks to mitigate risks. The rise of CASBs was driven by the transition of Office 365 to cloud servers. The emergence of ChatGPT and LLMs presents a short-term windfall for CASB vendors, but they need to innovate rapidly to meet customer demands for LLMs. Otherwise, a new version of Netskope may emerge in the new era, or even OpenAI itself could step in to address the issue of sensitive data leakage.

Other opportunities include the transformation of API security, email security, and application security in the era of LLMs, as well as the assistance of LLMs in enabling intelligent SecOps automation.

Given these trends, I have great confidence in Netskope and Palo Alto Networks, and you can find the reasons in the main body of this article.

01 Discuss "AI Security" More Clearly

02 Most Enterprise Security Stacks Will Not Disappear

03 A Deep Dive into Five Major Security Opportunities Amidst the Dramatic Changes in LLMs

04 Conclusion

1. Discuss "AI Security" More Clearly

The discussion on security/safety has been particularly frustrating amid the recent LLM frenzy in the past few months. Both "safety" and "security" are translated as "安全" in Chinese, making it difficult to differentiate between the safety of LLMs and their impact on traditional cybersecurity. This lack of clarity greatly contributes to inconsistency in AI security discussions.

In this article, we aim to present a clearer and straightforward framework to facilitate discussions on "AI security":

Within this framework, we delineate three key concepts:

AI Safety

This is an established term that primarily focuses on the internal aspects of AI systems. It encompasses concepts such as model alignment with human intent, interpretability, and robustness. In the era of LLMs when we are close to achieving AGI, the notion of safety becomes more deeply intertwined with concepts such as alignment and RLHF. It puts an emphasis on making AI systems work for humans and minimizing the potential for harm to humans. In OpenAI's recent publication "Our Approach to AI Safety", it once again underscored its commitment to this definition: After completing the training of GPT-4, OpenAI dedicated six months to ensure the model's greater safety and alignment with human intent.

I believe that companies involved in model development, such as OpenAI, Anthropic, and DeepMind, will play a leading role in the field of AI safety, both in terms of the topics and the ecosystem. Labelling/annotation companies like Scale AI and Surge AI, which we have followed, have the opportunity to gain from the demand for human feedback labelling. On top of commercial companies, AI safety will also be an important issue of concern for academia, cutting across multiple disciplines from computer science to social policy and beyond.

Security for AI

AI safety focuses on the internal aspects of AI systems, while security for AI is more about protecting these systems from external threats and attacks, ensuring compliance with real-world compliance regulations. Compared to cybersecurity in a broader sense, security for AI is more targeted at protecting AI/ML models per se and less at perimeter defense layer such as firewalls and gateways.

Many well-known "AI for security" companies have been established prior to the LLM era, including Arize, Fiddler, and Credo that specialize in model monitoring. Conventionally, they assist companies in ensuring that their machine learning models meet data security requirements, protect user privacy, mitigate biases, and provide a certain level of interpretability. There are also similar tools, such as Calypso AI. Meanwhile, LLMs have introduced new ways of attack and defense demands. For instance, the picture below illustrates the most straightforward "prompt injection attack."

An example of malicious prompt injection attack demonstrated by Riley Goodside on Twitter.

The Impact of AI/LLMs on Existing Cybersecurity Stacks

From the perspective of classic cybersecurity stacks, the demands for endpoint, network, and identity security have always existed, but the underlying technologies have been evolving. In the 1990s, companies like McAfee relied on hard-coded signature databases for antivirus purposes. Then in the 2010s, CrowdStrike, among others, began leveraging cloud infrastructure and supervised shallow machine learning for security solutions. In addition to endpoint security, we have seen that unsupervised learning has gained traction and human feedback has been introduced. Therefore, cybersecurity has been evolving in parallel with the development of AI.

After the emergence of LLMs, I believe that their impact on existing cybersecurity stacks can be categorized in the following ways:

• Interaction with LLMs presents new security demands, such as the imperative for enterprises to prevent the potential leakage of sensitive information when employees use ChatGPT. Additionally, as advanced models like GPT-4 and PaLM are providing capabilities through APIs, there will be an escalated need to protect these APIs as an emerging attack surface.

• Attackers do not necessarily target the models themselves, but instead exploit LLMs to launch other types of attacks. LLMs are naturally suited for social engineering, enabling mass production of phishing emails and the forgery of certificates, credentials, and identities. Generating malicious code is also a significant attack technique. While these fall within the scope of protection provided by current cybersecurity vendors, they still require a certain level of innovation to mitigate such threats.

• LLMs have the potential to bring disruptive changes to certain aspects of security. Two natural scenarios include code generation and the SecOps process. We will delve into these scenarios in detail in section 3 of this article.

2. Most Enterprise Security Stacks Will Not Disappear.

There is good reason to be excited about the role that LLMs can play in the field of security. Frank Wang, a security engineer whom I greatly admire, has provided a high-level summary for various security companies:

The vast majority of security use cases can essentially be boiled down to two things:

1. Access Management;

2. Visibility.

Every security company collects all sorts of data in systems or devices. Some specific data, when provided with additional context, can greatly enhance our capacity to detect suspicious and malicious activities. These data are the core of security companies at the platform level, making it easier for them to build vertical apps as products for clients.

In “The Age of AI: Thought on Investing in LLMs and OpenAI", we discussed that the structure and scale of collectible data are poised for change. In addition, as computers become better at comprehension, the imaginative space for contextual stitching will also be unlocked, potentially leading to changes in the interaction forms of traditional vertical security applications. All of these provide an imaginative space for the role of LLMs. One concrete manifestation of this is Palo Alto Networks' Cortex XIASM, which I will discuss in more detail in section 3.

Beyond these imaginative spaces, one fact remains true: most security stacks will not disappear in the era of LLMs. The logic behind this is quite simple. Many crucial components of security capabilities are not closely tied to language models. Furthermore, the consequences of making mistakes in security actions are severe. Therefore, in the short to medium term, it would be more ideal for LLMs to act as Copilot, rather than full-service agents, in appropriate scenarios.

Microsoft Security Copilot, which was unveiled last month at the Microsoft Secure conference, is the one to affirm this point

It is worth noting that Microsoft is one of the most competitive companies dabbling in the intersection of security and LLMs. Its revenue of security, as a crucial pillar of the company's overall strategy, broke the $20 billion mark in the last quarter. The keynote address at the 2022 Microsoft Ignite conference also put significant emphasis on security, with nearly half of the time spent discussing security products. Additionally, Microsoft has restructured its security product line, aggregating them under the Microsoft brand and putting an end to the days when security capabilities and products were scattered across different departments, such as Windows, Office, and Azure.

The launch of Security Copilot does not diminish the value of Microsoft's existing security products. Rather, it positions Copilot as an interactive entity that can assist clients in improving their SecOps practices using natural language interactions, such as investigating suspicious activities. The underlying security capabilities are still provided by products like Defender and Sentinel.

A similar case in point is OpenAI, which is the world's foremost expert in LLMs. That said, it still relies on traditional enterprise security stacks to protect itself, which comes as no surprise. For instance, OpenAI employs Cloudflare and Auth0 to manage traffic and user identities. It would be difficult to envision how an LLM could replace Cloudflare's global routing network. Just last month, ChatGPT experienced downtime due to a new bug in Redis. The effective resolution of such issues still requires the use of APM and observability providers like Datadog and Sumo Logic.

3. A Deep Dive into Five Major Security Opportunities Amidst Dramatic Changes in LLMs

In the following discussion, I will delve into the opportunities that LLMs bring to existing network security stacks. Please note that the list of five major opportunities here is not intended to be exhaustive. Rather, we aim to present the latest developments to give you a sense of these transformations.

Data Leakage in the Era of LLMs Security Segment: SSE

Opportunity Type: Increased penetration rate and product purchase

Who Will Seize the Opportunities:

• SSE players: Netskope, Palo Alto Networks, Zscaler, Microsoft, Cloudflare, etc.

• LLM players: OpenAI, etc.

• Emerging startups.

Data leakage prevention (DLP) in ChatGPT is currently the top security need for enterprise clients.

According to a survey conducted by CYBERHAVEN on March 21, 8.2% of employees have used ChatGPT in the workplace. 6.5% of employees have pasted company data into ChatGPT, and 3.1% have fed sensitive company data to ChatGPT, which accounts for 11% of the total data. Jisheng, VP of Traceable, also mentioned in a recent presentation that the key banking clients of Traceable have an urgent need for ChatGPT DLP solutions.

A similar scenario unfolded during the transition of Office 365 to cloud servers, which gave rise to CASBs. Microsoft CASB currently contributes over $2 billion in revenue, accounting for nearly half of Microsoft's cloud security revenues. Independent vendors like Netskope have also achieved an ARR of over $270 million. These out-of-band products of CASB monitor user activity within SaaS through APIs, while the inline products intercept user traffic, redirect it to the vendor's PoP for inspection, and then forward it to the SaaS provider. This inline approach has significant transferability for DLP in the current LLM landscape. Therefore, I believe that top-tier CASB players have the opportunity to benefit from this trend.

For readers unfamiliar with DLP, it is important to note that traditional and basic DLP solutions are typically used to protect personally identifiable information (PII) data, such as social security numbers or credit card numbers. These types of data are relatively "identifiable", and blocking employees from copying and pasting the corresponding numerical digits can serve as an effective measure. However, when it comes to ChatGPT, users input a wide range of text, and information sensitivity is based on the business context of the client's company. As a result, these basic DLP solutions are almost ineffective for ChatGPT.

In this context, the simplest DLP solution for ChatGPT is to block its URL, which completely undermines employees' flexibility. Vendors like Netskope can offer more sophisticated control. For example, they can issue alerts to users and only allow them to send prompts to ChatGPT with double confirmation:

Clients can also use Netskope to create more advanced rules, such as intercepting specific strings or keywords.

Currently, Netskope's DLP capabilities for ChatGPT are scored around 60 points, which indicates a reasonable level of effectiveness. As a result, we have observed a significant increase in client inquiries regarding Netskope and Palo Alto Networks for LLM DLP. However, with more and more LLM-based web applications emerging, the capabilities designed solely for ChatGPT may quickly lose their effects. Therefore, more users are urging SSE vendors to provide separate category controls for AI applications. There will be a spike in similar demands, making it important to keep an eye on the product capabilities of major SSE/CASB vendors. Companies that are able to develop LLM DLP products with a score of 80-90 within a short period of time have the opportunity to secure more orders and expand their client base.

It is important to note that OpenAI itself has the opportunity to become a disruptor in this field. Given that governments (such as Italy, which has taken significant action) and companies (ranging from Amazon to JP Morgan) in various countries and regions have imposed restrictions on the use of ChatGPT due to sensitive data concerns, OpenAI may need to develop more original DLP strategies for its applications on its own or through collaboration with partners.

Automated SecOps Acceleration
Security Segment: SecOps

Opportunity Type: New product forms, increased penetration rate of existing innovative products

Who Will Seize the Opportunities: Microsoft, Palo Alto Networks, etc.

This presents an opportunity that may result in some analysts within the SOC being laid off. However, the efficiency improvements in SecOps and overall cost reduction for the entire SOC are crucial operational goals for large enterprises given the current economic landscape. This has enabled Palo Alto Networks to receive excellent market feedback for its Cortex XSIAM, which they started beta testing with major clients at the end of last year. In the past few months alone, it has generated an ARR of over $100 million, with each order valued at over $1 million.

Palo Alto's CEO, Nikesh Arora, provides a clear explanation of the background and details of this product. Here is a quote from him:

"The problem in security has been most enterprises have had what is traditionally said about AI, garbage and garbage out. In most secured -- most enterprises have about 30 to 40 security vendors. They collect data from 40 different security vendors and then try and cross co-related. It's like trying to get 40 different people who speak 40 languages, trying to get them to communicate. It doesn't work like that. You need a single source of truth. So what we did internally is we took our business where we had 67,000 alerts across 200 vendors. We've replaced all of that with one endpoint from ourselves. We cross correlate the data, and we took our mean time to respond from 27 days to under one minute by using AI, that's the product we put in the market four months ago called XSIAM.

I think that will -- that concept, I'm not suggesting to only be us. That concept will be the disruptive event in security in the next five years. The ability to cross correlate data make it consistent, run a normalized data lake in any company and be able to stop threats from happening. There's a ransomware attack that's going on in some company out there where the entire from the point in time they attack the company until the time they've extracted petabytes of data was 14 hours. So you have to be able to stop the threat in less than 14 hours. Today, is the mean time to respond for most companies in days, which means everybody is susceptible. So to me, this is a big deal. This is what needs to happen. And hopefully, we'll be one of the players in the market who can make it happen."

XSIAM Product Interface (The green number and the pie chart represent automated incidents)

Cortex XSIAM was not directly integrated with OpenAI before the hype around ChatGPT. However, it is evident that Cortex XSIAM is a perfect product to add LLM capabilities.

An ideal SecOps automation product is likely to be a combination of Cortex XSIAM and Microsoft Secure Copilot, with strong security capabilities, automated processing, and a user-friendly natural language UI. The key advantage for Palo Alto Networks in launching such a product lies in its comprehensive security stack portfolio, which includes both firewall and endpoint solutions that typically encompass over 85% of a client's security data. Microsoft appears to be the only rival with a similar breadth of product offerings.

APIs Continue To Be an Emerging Attack Surface.

Security Segment: API Security

Opportunity Type: Increased penetration rate

Who Will Seize the Opportunities: Cequence, Traceable, Noname Security, Salt Security, etc.

Besides cloud security, API security has been the focus of VC investments in the security field over the past three years. The core thesis is to identify the next major attack surface. It used to be endpoints, which evolved into networks and identities as endpoints multiplied. Now, with APIs becoming the central link for communication between software, they have a growing potential to become the next significant attack surface, giving birth to important companies in this field.

This is a new field that is rising rapidly. According to the VP in charge of security in the Open Banking department of US Bancorp, there was almost a void of API security vendors in 2018. It was not until 2019 that the OWASP API Top 10 was unveiled. According to a survey by Future Market Insights, the API security market was valued at approximately $1 billion in 2022 and is expected to maintain a CAGR of around 25% in the next 10 years.

The current state of LLMs validates this thesis once again and adds some upsides to API security. From OpenAI to Anthropic, Cohere, and even Google's PaLM, the most powerful LLMs provide capabilities via APIs.

OpenAI's Mira, during an interview at Greylock's event, has already considered this approach as the default best practice in security for continuous and controlled deployment of LLMs externally. For example, GPT-3 was initially deployed to a small group of users through APIs and then gradually expanded its access, throughout which measures were taken to govern illicit activities, misuse, and abuse.

Specifically, when it comes to LLMs and API security:

• For owners of LLMs and APIs such as OpenAI, Anthropic, and Google, they are in urgent need of the bot defense capabilities provided by API security companies. Their APIs are not just accessed by regular users. There are also numerous maliciously programmed bots and compromised accounts that launch large-scale automated attacks against the APIs. Leading API security vendors like Cequence can leverage the capabilities of AI/ML to detect malicious behavior that deviates from the baseline. This helps clients establish governance and defense mechanisms against bots.

• For users of LLM APIs, typical API security vendors like Traceable, Noname, and Salt can provide visibility to enterprise managers regarding the internal usage of APIs. For instance, imagine you are the CEO of a small unicorn company, and you want to know how many employees are using OpenAI's API key. Governance of such shadow IT usage is often challenging. These API security vendors can help maintain visibility and mitigate threats posed by authenticated yet unpreventable API usage.

Malicious Usage Creates Ripples for Email Security.

Security Segment: Email Security

Opportunity Type: Next-Generation Inter-generational Products

Who Will Seize the Opportunities:

• Next-generation Email Security GatewayPlayers: Cloudflare (Area 1), Abnormal Security, etc.

• SSE Players: Netskope, Palo Alto Networks, etc.

• Emerging Startups.

Email security hasn't seen much innovation for over 20 years, with Proofpoint almost dominating the entire market.

The situation has seen some change in the wake of the pandemic, for the surging volume of phishing emails during the pandemic exposed Proofpoint's outdated technology to vulnerabilities, prompting companies that had not adopted Proofpoint or next-generation firewalls to urgently seek email security vendors. This has resulted in the rise of companies like Area 1 and Abnormal Security in the next-generation email security field. These companies are easily deployable and offer better crawler technology and AI capabilities, accommodating substantial user demands during the pandemic.

Thanks to the immense power of ChatGPT and other text-generating LLMs, we can anticipate that phishing emails and various forms of social engineering will continue to show up. This ongoing trend will drive innovation in email security that has been made since the onset of the pandemic.

Abnormal Security Demo

Requesting ChatGPT to create an email that entices recipients to click on a link.

While there are certain barriers to entry in this field, the essence of next-generation email security solutions still lies in connecting via APIs to popular email platforms such as Gmail and Outlook, and then analyzing abnormal behavior and data. This approach bears a striking resemblance to out-of-band CASB solutions. In theory, both Netskope and Palo Alto Networks have the potential to expand their influence in this domain at any given time.

Transformation Might Take Place for Application Security

Security Segment: Application Security

Opportunity Type: New Product Forms

Who Will Seize the Opportunities: GitHub, SourceGraph, Snyk, GitLab, etc.

GitLab was originally on the right track in the DevSecOps realm, while Snyk was rising rapidly in this blue ocean. However, GitHub Copilot suddenly emerged as a disruptor. If the integration of "code generation" and the process of programmers writing code continues, the OpenAI Codex-empowered GitHub will become a pivotal platform for programmers and code and expand to security.

The security SVP of StrongDM represents a typical type of clients:

StrongDM utilizes Snyk to scan for code vulnerabilities. GitHub has been unable to analyze Infrastructure as Code (IaC) or other third-party dependencies. Once GitHub enhances its security capabilities, there will be strong motivation to switch from Snyk to GitHub.

Confidence in GitHub's foray stems from multiple aspects:

1. The majority of source code in the world is hosted on GitHub, which allows it to have access to and understand the most extensive range of vulnerabilities;

2. Microsoft possesses the complete security capabilities that Snyk delivers internally;

3. Copilot is a strong signal. The first step is using AI to generate code. Then the next step is to leverage AI to read code and scan for vulnerabilities. This approach enables vulnerability scanning to go beyond rule-based methods and be driven by LLMs.

Currently, the company pays $150 per month for each engineer for Snyk, while GitHub only pays each engineer $250 per year. From a cost and vendor reduction perspective, there is a strong motivation to migrate to GitHub.

In his article "How AI is Disrupting Software: Can You Work for AI?", Botaijin made a very bullish assessment of GitHub.

In the past, code repository products mainly focused on categorization, storage, code and security capabilities, as well as integration with other CI/CD products, all of which are likely to become less important in the future.

Under the influence of AI:

• Code repositories will soon evolve into "coding tools", and GitHub Copilot, as the most exciting application-level product amid the AI boom so far, will continue to maintain its leading advantage over other code repository companies in the industry.

• Code repositories that become "coding tools" will become utmost important in the entire development process, occupying a central position in future project management tools. AI will have the "authority to allocate tasks" in the development process.

From a practical perspective, there is some possibility for GitHub to enter the security segment. Microsoft architect Jorge demonstrated the capability of using Azure OpenAI Service and the davinci-003 model for code vulnerability scanning.

Davinci-003 has the ability to detect SQL injection vulnerabilities.

Davinci-003 can identify CSRF.

Davinci-003 can identify authentication and session management failures.

However, before we see Codex 2.0, GitHub does not have an absolute dominant advantage over players like GitLab and Snyk. Through OpenAI and LangChain, clients can independently create automated code detection and remediation capabilities within GitLab's CI pipeline, which can deliver experience that is close to that of Copilot. Snyk has also made some AI advancements, acquiring DeepCode in 2020 and recently investing in CodiumAI, which utilizes AI for automated code testing. As the capabilities of Copilot continue to expand, it will be worth following how Snyk and GitLab evolve their products in response.

4. Conclusion

From the perspective of global growth investors, the question we spend a lot of time contemplating is: Will companies that have already entered the growth stage be disrupted by earlier-stage companies that are more LLM-native and more agile?

The answer to this question ultimately boils down to one key judgment and five key dimensions. One key judgment revolves around whether the dominance of OpenAI will continue. If the current landscape with LLMs persists, OpenAI will have strong motivation to enter various natural vertical scenarios, such as DLP during ChatGPT application or code security scanning capabilities provided by Codex and ChatGPT. As security companies may not have the ability to intervene in OpenAI's market share, the following five key dimensions become even more important:

• The more important in client budgets and security stacks, the more secure companies become. In the case of StrongDM, an application security company, we can see that GitHub and Snyk hold varying degrees of significance for clients. Security companies with a more solid position are typically SSE, EDR, and CSPM companies. It is challenging for LLM-native applications to find enough value propositions to march into these domains;

• The more extensive the product line, the more secure companies become. Companies like Microsoft and Palo Alto Networks fall into this category. An extensive product line signifies strong client relationships, making it not only difficult to be disrupted but also more likely to benefit from the trends of SecOps and SOC automation;

• Companies that have no technical debt and can innovate quickly are more secure. Netskope is the best example in this regard. It has gained a competitive edge in the CASB and SSE space due to its continuous innovation and minimal technical debt. This is equally important in the era of LLMs;

• Security companies with "hard assets" are more secure than those with only "soft assets". This is a common-sense observation. Companies like Cloudflare and Netskope have established a sufficient number of PoPs worldwide to provide low-latency and high-performance services. Merely being LLM-native is not enough to narrow the development gap at the physical infrastructure level.

• Companies with richer experience in unsupervised learning are more secure. Some investors who lack the know-how of due diligence and security may assume that companies like CrowdStrike, which also have AI labels, are beneficiaries of the LLM boom. However, the reality is that they often use supervised and shallow deep learning techniques, which are not in the same direction as the current unsupervised deep learning. Examining the specific year a security company begins using machine learning and unsupervised learning should become an important topic during due diligence.