Detecting and Preventing Prompt Injection Attacks in AI-Integrated Applications

---
description: Enforce secure coding practices to prevent prompt injection vulnerabilities in AI-integrated applications.
globs: ['**/*.py', '**/*.js', '**/*.ts']
tags: [security, ai, prompt-injection]
priority: 1
version: 1.0.0
---

# Detecting and Preventing Prompt Injection Attacks in AI-Integrated Applications

## Context
- Applicable to all code handling user inputs for AI models.
- Essential for applications utilizing generative AI to ensure security against prompt injection attacks.

## Requirements
- **Input Validation and Sanitization**: Implement strict validation and sanitization of all user inputs to prevent malicious content.
  - Use regular expressions to filter out special characters and patterns that could be used in injection attacks.
  - Enforce input length restrictions to mitigate buffer overflow risks.
- **Role-Based Access Control (RBAC)**: Restrict access to AI functionalities based on user roles to minimize exposure to potential attacks.
  - Define clear roles and permissions for all users interacting with AI systems.
  - Regularly review and update these permissions to reflect changes in roles or responsibilities.
- **Secure Prompt Engineering**: Design prompts and AI interactions with security in mind to reduce the risk of injection attacks.
  - Incorporate security considerations into the design phase of AI development.
  - Use techniques such as prompt partitioning, where user input is strictly separated from the control logic of prompts.
- **Continuous Monitoring and Anomaly Detection**: Implement real-time monitoring of AI system interactions to detect and respond to potential prompt injection attacks.
  - Deploy monitoring solutions that can track and analyze user interactions with AI systems at a granular level.
  - Use machine learning-based anomaly detection to identify unusual patterns that may indicate an attack.
- **Regular Security Audits and Testing**: Conduct regular security audits and penetration testing to identify and mitigate vulnerabilities.
  - Simulate various attack scenarios to see how the model responds to malicious input.
  - Update and retrain models regularly to improve their resistance to new and evolving attack techniques.

## Examples

<example>
# Good Example: Input validation and sanitization
import re

def sanitize_input(user_input):
    # Remove special characters
    sanitized = re.sub(r'[^a-zA-Z0-9 ]', '', user_input)
    # Enforce length restriction
    if len(sanitized) > 100:
        raise ValueError("Input too long")
    return sanitized
</example>

<example type="invalid">
# Bad Example: Lack of input validation
def process_input(user_input):
    # Directly using user input without validation
    result = ai_model.generate(user_input)
    return result
</example>