Skip to main content

Semgrep - Discovering Secrets in JavaScript

The process can be slow for larger applications.

Katana GitHub

SemGrep GitHub

Step 1: Gather URLs​

  1. Crawl with katana using "-sr" to save the responses.

    ~/go/bin/katana -u [url] -fs fqdn [-headless] -xhr -jc -jsl -sr -em js

Step 2: Rename all the files with the .js extension so semgrep applies the correct rules​

Note: The responses will also include the headers and some may not actually be javascript files.

  1. Use the following one liner to rename all the .txt files

    find . -type f -name "*.txt" -exec bash -c 'mv "$0" "${0%.txt}.js"' {} \;

Step 3: Scan with Semgrep​

  1. Setup semgrep (Make an account to access pro rules)

    SemGrep Website

  2. Ensure there is an empty .semgrepignore file in the directory

    .semgrepignore

    # .semgrepignore

  3. Run a semgrep scan with pro rules

    semgrep scan --dryrun --config "p/secrets" --config "p/gitleaks" --pro --text --text-output=semgrep.txt --timeout 0 --timeout-threshold 0 --max-target-bytes=-1

Parsing Semgrep output​

#!/bin/bash

# Check if the correct number of arguments is provided
if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <file_path>"
    exit 1
fi

file_path="$1"

# Check if the file exists
if [ ! -f "$file_path" ]; then
    echo "File not found: $file_path"
    exit 1
fi

# Define ANSI color code for green
GREEN='\033[0;32m'
NC='\033[0m' # No Color

# Function to extract fields for each occurrence
extract_fields() {
    local data=$1

    local check_id=$(echo "$data" | jq -r '.check_id')
    local start=$(echo "$data" | jq -r '.start | @json')
    local end=$(echo "$data" | jq -r '.end | @json')
    local location_path=$(echo "$data" | jq -r '.path')
    local lines=$(echo "$data" | jq -r '.extra.lines')
    local message=$(echo "$data" | jq -r '.extra.message')
    local impact=$(echo "$data" | jq -r '.extra.metadata.impact')
    local likelihood=$(echo "$data" | jq -r '.extra.metadata.likelihood')
    local owasp=$(echo "$data" | jq -r '.extra.metadata.owasp | join(", ")')

    if [ "$impact" == "HIGH" ] && ([[ "$likelihood" == "MEDIUM" ]] || [[ "$likelihood" == "HIGH" ]]); then
        echo -e "${GREEN}Check ID:${NC} $check_id"
        echo -e "${GREEN}Start:${NC} $start"
        echo -e "${GREEN}End:${NC} $end"
        echo -e "${GREEN}Location Path:${NC} $location_path"
        echo -e "${GREEN}Lines:${NC} $lines"
        echo -e "${GREEN}Message:${NC} $message"
        echo -e "${GREEN}Impact:${NC} $impact"
        echo -e "${GREEN}Likelihood:${NC} $likelihood"
        echo -e "${GREEN}OWASP:${NC} $owasp"
        echo
    fi
}

# Loop through each item in the results array and extract fields
jq -c '.results[]' "$file_path" | while IFS= read -r item; do
    extract_fields "$item"
done