Realism Reigns on AI at Cybersecurity Events: Black Hat and DEF CON

The Evolution of Generative AI in Cybersecurity

Rapid Turn from Hype to Realism

Recent IT industry conferences such as Black Hat and RSAC have witnessed a rapid evolution in the discourse surrounding generative artificial intelligence (AI) in the cybersecurity field. While CISOs at Black Hat 2022 expressed a reluctance to embrace AI, the narrative shifted dramatically at RSAC 2023, where discussions about generative AI dominated, with speculation about the significant changes it would bring to the security industry. By Black Hat USA 2023, the conversation had matured, focusing on managing generative AI as a tool to augment human operators and acknowledging the limitations of AI engines.

Amplifying the Effectiveness of Cybersecurity Professionals

The realism surrounding generative AI is driven by the recognition that it will become an essential feature of cybersecurity products, services, and operations in the coming years. One of the key reasons for this is the persistent shortage of cybersecurity professionals. Rather than replacing human workers, generative AI is seen as a means to amplify their effectiveness. The goal is to make each cybersecurity professional more productive, particularly in enabling Tier 1 analysts to provide context, certainty, and prescriptive options to higher-tier analysts as they handle alerts.

The Limitations of Generative AI

As the conversation shifted towards the use of generative AI, the limitations of the technology became apparent. Two significant limitations were discussed: the quality of training data and the issue of trust in AI-generated results.

Training Data Quality Concerns

Participants generally agreed that the effectiveness of any AI deployment is directly linked to the quality of the data on which it is trained. However, the pursuit of larger datasets can clash with concerns about privacy, data security, and intellectual property protection. To address this, companies are increasingly emphasizing “domain expertise” in generative AI deployments. By narrowing the scope of an AI instance to a specific topic or area of interest, organizations can optimize training for prompts on that subject, ensuring accurate and effective results.

The “Black Box” Limitation and Trust

One of the primary challenges in adopting generative AI is the lack of trust in its outcomes. Executives and employees tend to view AI engines as mysterious and difficult to comprehend. To cultivate trust in AI-generated results, both security and IT departments must enhance transparency in how models are trained, generated, and utilized. Generative AI‘s primary role is to assist human workers, but if these workers do not trust the responses they receive, the potential benefits of AI will be severely limited.

Defining AI and the Need for Specificity

The conferences also highlighted the need for precision in discussions about AI. It became evident that considerable confusion stemmed from the lack of clarity surrounding the term “AI.” While many were referring to generative AI or Large Language Model (LLM) AI, others mistakenly assumed that AI had been present in their products and services for years. It is crucial to define terms accurately and specify the type of AI under consideration to avoid misunderstandings.

For instance, the AI used in security products for several years utilizes smaller models compared to generative AI. It delivers faster responses and proves valuable for automation, quickly answering repeated, specific questions. In contrast, generative AI can address a broader range of questions using models created from vast datasets. However, it may not consistently generate responses quickly enough to be fully leveraged for automation.

Looking Ahead

Generative AI, particularly in the form of LLM AI, has secured its place as a prominent topic in cybersecurity. As the technology continues to develop, there will be numerous conversations and articles that explore its potential applications and limitations. It is essential for industry professionals to familiarize themselves with generative AI and prepare for the discussions and advancements that lie ahead.