Lesson 12: Alerting and Monitoring

Incident Response

NIST Phases:

1. Preparation: Before incidents - setting up tools, policies, training, IR plans
2. Detection: Identifying/noticing potential security issues - alerts, unusual activity, IDS/IPS triggers
3. Analysis: Investigating to understand scope, impact, cause - review logs, assess damage
4. Containment: Stopping the spread - isolate systems, block IPs, disable accounts
5. Eradication: Removing threat completely - delete malware, terminate sessions, patch systems
6. Recovery: Restoring to normal operations - restore from backup, test systems, monitor
7. Lessons Learned: After resolution - post-incident review, update policies, generate reports

Forensics:

• Chain of custody: Document each step from collection to court
• Disk imaging: dd tool for creating forensic copies

Digital Forensics

Due Process & Legal Hold:

• Forensics: Preserve evidence for legal use
• Due Process: Follow legal procedures and fairness
• Legal Hold: Right to retain and seize digital assets

Acquisition Order of Volatility (most volatile first):

1. CPU registers, cache
2. RAM
3. Network data
4. Disk
5. Backups

Memory Acquisition:

• Live memory capture tools (Volatility Framework)
• Collects RAM content, session keys, temporary files

Disk Imaging:

• Live Acquisition: While system is running
• Static Acquisition: When system is shut down
• Tools: dd, dcfldd (adds hashing and logging)

Preservation:

• Write Blockers: Prevent data modification
• Hashing: SHA-256 for integrity (compare source vs image)
• Chain of Custody: Document each step
• Tamper Evidence: Secure storage and bags

Reporting:

• Summarize evidence and conclusions
• Must be objective, repeatable, non-tampered
• E-discovery: Process digital data (email, docs) for legal cases

Data Sources

Common Data Sources:

• Memory & Disk: RAM, file systems, metadata
• Host Logs: Security, system, app logs (Windows Event Viewer, syslog, journald)
• Application Logs: App-level activities, endpoint alerts
• Network Logs: Firewall, IDS/IPS (Suricata)
• Packet Captures: Full traffic visibility (Wireshark)
• Metadata: File timestamps, email headers, web request/response headers

Dashboards:

• Analyst View: Alerts for triage
• Manager View: Status and summaries
• Automated Reports: For compliance and executive decision-making

SIEM (Security Information and Event Management)

Log Collection:

• Agent-Based: Installed on endpoints
• Listener/Collector: Listens to syslog/NetFlow
• Aggregation: Combine logs from various formats
• Normalization: Standardize fields
• Time Sync: Align timestamps across sources

Examples: Splunk, Wazuh, Security Onion

Alerting and Correlation:

• Static Rules: Trigger alerts on patterns or thresholds
• Correlation: Connect related events across sources
• Threat Feeds: Enrich alert context

Alert Tuning:

• False Positive: Alert triggered without actual threat
• False Negative: Threat occurred but no alert triggered
• True Positive: Correctly detected real threat

Tuning Techniques:

• Suppress noisy or redundant alerts
• Adjust sensitivity
• Redirect flood alerts
• Use machine learning for pattern detection

Monitoring Infrastructure

Tools/Methods:

• SNMP Traps: Alert on hardware/software events
• NetFlow/IPFIX: Monitor traffic flow statistics (who talks to whom, port/protocol)
• Heartbeat Monitoring: Confirm system availability
• Network Monitors: Track appliance and link state
• System Logs: Detect uptime, usage, errors
• Cloud Monitors: Detect cloud service outages
• Vulnerability Scanners: Check for missing patches/config issues
• DLP Tools: Detect and prevent sensitive data exfiltration

Benchmarks and SCAP

SCAP (Security Content Automation Protocol): NIST standard for automated compliance scanning

Used With:

• OVAL (Open Vulnerability and Assessment Language)
• XCCDF (Extensible Configuration Checklist Description Format)

Purpose: Detect misconfigurations, missing controls, deviations from standards

Testing Types

• Tabletop Exercise: Walkthrough, no live systems
• Simulation: Red team emulates attacker
• Walkthrough: Step-by-step response drill

Threat Hunting: Proactive detection of unknown threats using logs and intelligence, different from reactive incident response