Mastering Semgrep Custom Rules for Security Testing: A Deep Dive

Generic security scanners are great for catching the ‘low-hanging fruit’—things like hardcoded secrets or outdated libraries. But in my experience, the most dangerous vulnerabilities are often logic-based and specific to how your application is architected. This is where semgrep custom rules for security testing become a superpower.

I’ve spent the last few months integrating Semgrep into various CI/CD pipelines, and the biggest shift in my security posture didn’t come from the community rules, but from the custom ones I wrote to block specific patterns in our internal API wrappers. If you’re tired of false positives and want a tool that actually understands your codebase, you’re in the right place.

The Challenge: Why Generic Rules Aren’t Enough

Most SAST (Static Application Security Testing) tools rely on massive databases of known vulnerable patterns. While useful, they suffer from two main problems: noise and blind spots.

• Noise: A rule that flags every instance of innerHTML is useless if your team uses a sanitized wrapper for everything.
• Blind Spots: A generic scanner doesn’t know that in your specific project, calling db.executeRaw() without a prior auth.checkRole() call is a critical privilege escalation bug.

To move beyond these limitations, we need to treat security policies as code. By writing custom rules, we can tell the scanner exactly what ‘bad’ looks like in our specific context. For those managing larger infrastructures, this complements best practices for container security scanning by securing the application layer while the container scanners handle the OS layer.

Solution Overview: How Semgrep Works

Unlike traditional grep, which searches for strings, Semgrep searches for code patterns. It parses your code into an Abstract Syntax Tree (AST), meaning it understands that user.name and user['name'] are often the same thing.

A custom rule consists of three main parts: the pattern (what to find), the message (why it’s a problem), and the severity (how bad it is). Because it’s YAML-based, these rules can be version-controlled and peer-reviewed just like your application code.

Techniques for Writing High-Impact Rules

1. Using Metavariables for Data Flow

The most powerful feature of Semgrep is the metavariable (represented by $VARIABLE). This allows you to capture a value in one part of the code and ensure it’s used (or not used) elsewhere.

rules:
  - id: unsafe-api-call
    patterns:
      - pattern: $USER.executeQuery($QUERY)
      - pattern-not: $USER.validateSession()
    message: "Executing a query without session validation is prohibited."
    severity: ERROR
    languages: [javascript]

In this example, I’m telling Semgrep: “Find any instance where some object calls executeQuery, but ensure that the same object hasn’t called validateSession first.” This is far more precise than a simple string search.

2. Ellipsis for Flexible Matching

The ... operator is your best friend. It allows you to ignore irrelevant code between the parts you actually care about. This is essential for catching vulnerabilities that span multiple lines of a function.

rules:
  - id: missing-audit-log
    patterns:
      - pattern: |
          $FUNC(...) {
            ...
            $DB.update(...);
            ...
          }
      - pattern-not: |
          $FUNC(...) {
            ...
            $LOG.audit(...);
            ...
            $DB.update(...);
            ...
          }
    message: "Database updates must be preceded by an audit log entry."
    severity: WARNING
    languages: [python]

As shown in the logic above, we aren’t caring about exactly what happens between the function start and the DB update; we only care that the audit log is present somewhere before the update occurs.

Implementation: Integrating into the Workflow

Writing the rule is only half the battle. To make it effective, it must be automated. I recommend a three-tier approach:

1. Local Pre-commit: Run a lightweight set of custom rules locally using semgrep scan --config my-rules.yaml.
2. CI Pipeline: Integrate Semgrep into your GitHub Actions or GitLab CI. If a rule with severity: ERROR is triggered, fail the build.
3. Security Dashboard: For teams comparing different toolings, like those looking at Snyk vs SonarQube for security testing, Semgrep fits in as the ‘fast’ layer that catches logic bugs before the heavier scanners even start.

Here is how I typically structure my CI step in a GitHub Action:

- name: Semgrep Security Scan
  run: |
    docker run --rm -v $(pwd):/src semgrep/semgrep 
    semgrep scan --config /src/.semgrep/custom-rules.yaml 
    --error

Case Study: Catching an IDOR Vulnerability

Last year, I worked on a project where developers were frequently forgetting to check if the userId in the URL matched the session.userId. This is a classic Insecure Direct Object Reference (IDOR) bug.

Instead of relying on manual code reviews, I wrote a custom rule that flagged any controller method that accessed a User entity by ID without calling our AccessControl.verifyOwnership() method. Within one hour, Semgrep found four existing vulnerabilities that had slipped through three previous PR reviews. The ROI on that one rule was massive.

Pitfalls to Avoid

• Over-constraining: Don’t make your patterns too specific. If you include every single argument in a function call, the rule will break the moment a developer adds an optional parameter. Use (...) for arguments.
• The “Alert Fatigue” Trap: Avoid severity: WARNING for everything. If everything is a warning, developers will ignore them all. Only use ERROR for things that must be fixed before merge.
• Ignoring False Positives: When a rule flags something incorrectly, don’t just ignore it. Use pattern-not to refine the rule. A high-quality rule set is one that developers trust.

If you’re scaling your security efforts, remember that tooling is only as good as the process around it. Whether you’re using custom Semgrep rules or a full-scale platform, the goal is to empower developers, not block them.