Security Implications of Large Language Model Code Assistants: A User Study

Who Should Read

Engineering managers and security teams considering or already using AI coding assistants in their development workflow.

Core Mechanics

• Controlled study: developers using GitHub Copilot produced code with significantly more security vulnerabilities than those without
• Participants using Copilot were also more likely to rate their insecure code as secure — overconfidence effect
• Vulnerabilities span common categories: SQL injection, XSS, insecure deserialization, hardcoded secrets
• Effect persisted even among experienced developers, not just juniors
• The speed gains from Copilot may be offset by increased security review burden

Evidence

• Randomized controlled experiment with developers assigned to Copilot vs. no-AI condition
• Security audits of produced code by independent security researchers
• Statistically significant difference in vulnerability rates (p < 0.05) between conditions

How to Apply

• Treat AI-generated code as untrusted and route all suggestions through your existing security review pipeline.
• Add automated SAST (Static Application Security Testing) as a CI gate specifically for AI-assisted code changes.
• Train developers to be skeptical of AI suggestions in security-sensitive code paths (auth, input handling, cryptography).

Code Example

# Security-enhanced prompt examples when using AI code assistants

# ❌ Vulnerable prompt (requesting functionality only)
'''
Write a function that retrieves user info from the DB using a user ID
'''

# ✅ Security-conscious prompt
'''
Write a Python function that retrieves user info from the DB using a user ID.
Make sure to include:
- Parameter binding to prevent SQL injection (absolutely no string formatting)
- Input value type and range validation
- Safe exception handling that does not expose internal information on DB errors
- Remove sensitive fields (e.g., password_hash) before returning
'''

# Example of adding Bandit SAST to CI/CD (GitHub Actions)
# .github/workflows/security.yml
'''
steps:
  - name: Run Bandit Security Scan
    run: |
      pip install bandit
      bandit -r ./src -ll -ii -f json -o bandit-report.json
  - name: Upload Security Report
    uses: actions/upload-artifact@v3
    with:
      name: bandit-security-report
      path: bandit-report.json
'''

Terminology