Introducing CheckAI: Enhancing Security by Detecting and Preventing Attacks on ChatGPT-Generated Code

New Plugin by Checkmarx Protects Against Attacks on GenAI-Generated Code

Introduction

Checkmarx, a global leader in application security solutions, has recently introduced the CheckAI Plugin for ChatGPT, becoming the first in the industry to offer a solution to detect and prevent potential attacks against ChatGPT-generated code. This plugin aims to protect developers and security teams from malicious open-source packages and dependencies while working within the ChatGPT interface. With the rise of open source and GenAI-generated code, the tension between significant opportunities and emerging vulnerabilities has become a crucial decision-making concern for CISOs.

The Role of GenAI in Application Development

GenAI, short for Generative Artificial Intelligence, is disrupting the way software is developed. It offers time-saving tools, such as ChatGPT, that can significantly enhance the developer experience and accelerate the delivery of applications. However, the increasing reliance on GenAI also brings new types of security risks that need to be addressed.

Emerging Attacks and the Need for Protection

Attackers have already started targeting GenAI solutions with techniques like “AI hallucinations” and prompt injections. In response to these threats, the OWASP Foundation has introduced the first draft of the OWASP Top 10 list for LLMs (Language Models), highlighting the urgency for security measures.

The CheckAI Plugin for ChatGPT

Checkmarx’s CheckAI Plugin for ChatGPT provides developers and security teams with a powerful tool to scan GPT-generated code for potential vulnerabilities within the ChatGPT interface. The plugin offers instant feedback on possible vulnerabilities and helps validate open-source packages, thus ensuring applications meet AppSec standards. It also employs protection against malicious open-source packages, safeguarding against emerging attack vectors.

The Importance of AppSec Standards and Governance

As GenAI becomes a fundamental part of the development process, organizations must provide the right tools and governance to ensure the secure utilization of these technologies. AppSec (Application Security) standards play a critical role in maintaining the integrity and security of the software being developed. The CheckAI Plugin for ChatGPT enables development teams to continue leveraging the benefits of GenAI while adhering to AppSec standards and staying aligned with compliance requirements.

Collaboration and Exposing New Attacks

Checkmarx encourages GenAI solution providers to partner with them in identifying and addressing new types of attacks. By working together, the industry can stay ahead of emerging threats and develop comprehensive security solutions that protect businesses and their customers.

Addressing the OpenAI Vulnerability

In December 2022, Checkmarx’s AppSec security researchers discovered a vulnerability in the OpenAI signup process that could have allowed unlimited credit on new accounts. The team promptly reported this vulnerability to OpenAI, who quickly resolved the issue. This incident highlights the importance of robust security measures in AI systems and the need for proactive vulnerability identification.

Conclusion and Recommendations

The advent of GenAI brings immense potential for innovation and efficiency in software development. However, it also introduces new security challenges that organizations must address. Solutions like the CheckAI Plugin for ChatGPT by Checkmarx provide developers and security teams with the necessary tools to detect and prevent attacks in GenAI-generated code.

To maximize security when utilizing GenAI technologies, organizations should consider the following recommendations:

1. Adopt Comprehensive Application Security Platforms

Implement an application security platform, such as Checkmarx One, that integrates seamlessly with development tools and environments. These platforms provide holistic security solutions and enable organizations to identify vulnerabilities throughout the entire development process.

2. Stay Abreast of Emerging Attacks

Stay informed about the latest attack techniques targeting GenAI solutions and language models. Organizations should closely monitor industry developments and collaborate with security vendors and researchers to address new attack vectors proactively.

3. Educate and Train Development Teams

Invest in regular security training and education for development teams. This will help them understand the risks associated with GenAI-generated code and equip them with the knowledge to implement secure coding practices.

4. Implement Secure Coding Guidelines

Develop and enforce secure coding guidelines specific to GenAI technologies. These guidelines should cover best practices for utilizing GenAI tools, handling open-source packages, and validating dependencies.

By following these recommendations, organizations can take full advantage of the benefits offered by GenAI while maintaining a strong security posture. The collaboration between security vendors, researchers, and GenAI solution providers is crucial to mitigating emerging risks and ensuring the long-term safety of AI-powered applications.