Analysis_Tools

Vulnerability Analysis and Enrichment Tools

Tools for processing CVE records and generating CPE Applicability Statements. Processes CVE data from MITRE and NVD APIs to create interactive HTML reports for CPE matching and configuration generation.

Overview

Understanding the Problem Space: For comprehensive insight into the cybersecurity challenges this tool addresses, see CPE Automation Challenges.

CPE Applicability Generator

Processes CVE records to generate CPE Applicability Statements:

• Ingests CVE information from CVE List and NVD APIs
• Extracts CPE attribute values from affected product data
• Queries NVD /cpes/ API for matching CPE Names
• Processes results to identify relevant CPE Base String values
• Generates HTML reports for user review and selection
• Produces CPE Applicability Statements (configurations) from selected CPE Base Strings

Features

• Dashboards for monitoring processing progress, performance and source data concerns
• Caching system reduces API calls by caching responses locally
• Rules engine for automated CPE Applicability Statement JSON generation
• Interactive Modal System for display of contextual data to assist with user review
• Unified runs structure maintains all output in timestamped run directories
• Comprehensive test suites for validating functionality

All generated datasets are tracked in run-specific directories under runs/[timestamp]_[context]/logs/ with metadata for future differential updates.

Documentation

• CPE Automation Challenges - Problem domains, solutions, and codebase architecture
• Badge and Modal System Reference - Complete badge/modal system documentation
• CPE Caching System - Cache configuration and management
• Logging System - Structured logging patterns and configuration
• Dashboard Usage - Dashboard system and usage
• Dataset Generation - Dataset generation methodology and capabilities

Test Documentation

• Test Suites Overview - Comprehensive test documentation

Examples

The complete collection of generated pages is maintained at Hashmire/cpeApplicabilityGeneratorPages.

To access a specific CVE analysis page, use the following URL pattern:

https://hashmire.github.io/cpeApplicabilityGeneratorPages/generated_pages/[CVE-ID].html

Note: Not all CVEs are currently present in the dataset.

Usage

Single CVE Commands

# Single CVE analysis
python run_tools.py --cve CVE-2024-20515

# Multiple CVEs from file
python run_tools.py --file testExamples.txt

# Test file processing
python run_tools.py --test-file test_files/testModularRulesEnhanced.json

# Disable cache for testing
python run_tools.py --cve CVE-2024-20515 --no-cache

Dataset Generation

# Traditional status-based generation
python generate_dataset.py --statuses "Received" "Awaiting Analysis"

# Generate dataset for CVEs modified in the last 30 days
python generate_dataset.py --last-days 30

# Generate dataset for specific date range
python generate_dataset.py --start-date 2024-01-01 --end-date 2024-01-31

All dataset outputs are isolated in run-specific directories under runs/[timestamp]_[context]/logs/.

Dashboards

The system includes two monitoring dashboards:

• Dataset Generation Dashboard (dashboards/generateDatasetDashboard.html) - Monitors dataset generation progress with processing statistics and ETA calculations
• Source Data Concern Dashboard (dashboards/sourceDataConcernDashboard.html) - Tracks data quality concerns