ELK HOME

ELK SIEM Setup Guide - Ubuntu Server with Windows Endpoint

Purpose & Scope
System Requirements
Architecture Overview
Part 1: Ubuntu Server Setup
Part 2: Install Elasticsearch
Part 3: Install Kibana
Part 4: Install Logstash
Part 5: Install Zeek for Network Monitoring
Part 6: Windows 11 Endpoint Configuration
Part 7: Verify Data Flow
Part 8: Create Data Views and View Logs
Part 9: Maintenance and Monitoring
Troubleshooting
Security Hardening
Performance Tuning
Backup and Recovery
Managing Daily Indices
Additional Resources

Purpose & Scope

This guide provides comprehensive instructions for building a Security Information and Event Management (SIEM) system using the ELK Stack (Elasticsearch, Logstash, Kibana) with network traffic analysis and Windows endpoint monitoring.

Primary Objectives:

Deploy centralized log management and analysis platform
Implement network traffic monitoring with Zeek
Configure Windows endpoint detection and response (EDR)
Establish real-time security event monitoring
Create unified security operations dashboard

Technical Scope:

Elasticsearch for data storage and search
Logstash for log processing and filtering
Kibana for visualization and dashboards
Zeek for network traffic analysis
Winlogbeat and Sysmon for Windows event collection
Centralized security monitoring and alerting

System Requirements

Ubuntu Server (SIEM Server)

Specifications:

OS: Ubuntu 22.04.5 LTS
RAM: 6GB
CPU: 3 cores
Storage: 120GB
Network: Bridge Adapter
IP: 192.168.1.74

Download:

Ubuntu 22.04.5 LTS Server ISO

Windows 11 (Endpoint/Agent)

Specifications:

OS: Windows 11
IP: 192.168.1.91
Role: Agent sending logs to ELK

Network Requirements:

SSH access (port 22)
Logstash Beats input (port 5044)
Elasticsearch HTTP (port 9200)
Kibana web interface (port 5601)

Architecture Overview

Data Flow:

Windows events (Sysmon, Security, Application) captured by Winlogbeat
Network traffic analyzed by Zeek and forwarded by Filebeat
All logs sent to Logstash for filtering and processing
Processed logs stored in Elasticsearch indices
Kibana provides visualization and security operations interface

Part 1: Ubuntu Server Setup

1.1 Initial System Configuration

sudo apt update && sudo apt upgrade -y
sudo hostnamectl set-hostname elk-siem
sudo reboot

1.2 Install Prerequisites

sudo apt install -y wget curl apt-transport-https gnupg2 software-properties-common

1.3 Increase Virtual Memory for Elasticsearch

sudo sysctl -w vm.max_map_count=262144
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf

1.4 Configure Firewall

sudo ufw allow 22/tcp
sudo ufw allow 5044/tcp
sudo ufw allow 9200/tcp
sudo ufw allow 5601/tcp
sudo ufw enable

Part 2: Install Elasticsearch

2.1 Import Elasticsearch GPG Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

2.2 Add Elasticsearch Repository

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

2.3 Install Elasticsearch

sudo apt update
sudo apt install -y elasticsearch

2.4 Configure Elasticsearch

sudo nano /etc/elasticsearch/elasticsearch.yml

Replace the entire file with:

# ======================== Elasticsearch Configuration =========================

cluster.name: elk-siem-cluster
node.name: elk-node-1

# -- Paths 
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

# - Network --
network.host: 192.168.1.74
http.port: 9200
http.host: 0.0.0.0

#  Discovery -
discovery.type: single-node

#  Security --
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl.enabled: false
xpack.security.transport.ssl.enabled: false

2.5 Start and Enable Elasticsearch

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Note: Elasticsearch restart takes 30-60 seconds. Wait for the service to fully start before proceeding.

2.6 Set Elasticsearch Password

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

Save the generated password securely.

Example output (do not share real passwords in production):

New value: tMOpyF+me_vrD6wsM=2m

Important: This is the ELASTIC SUPERUSER account. You will use this credential to login to Kibana web interface.

2.7 Set Kibana System User Password

Elasticsearch 8.x requires a dedicated service account for Kibana instead of using the elastic superuser:

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system

Save this password as well. You will use it in Kibana configuration.

Example output (do not share real passwords in production):

New value: cMn0FVIDe4N4GpyhcLlv

Important: This is the KIBANA SERVICE ACCOUNT. Kibana application uses this internally to communicate with Elasticsearch. You will NOT use this to login to Kibana.

Understanding the two accounts:

elastic (tMOpyF+me_vrD6wsM=2m): Human login for Kibana web interface, creating dashboards, managing SIEM
kibana_system (cMn0FVIDe4N4GpyhcLlv): Service account for Kibana application to talk to Elasticsearch backend

This separation follows security best practices where human administrators and applications use different credentials.

2.8 Verify Elasticsearch

curl -u elastic:tMOpyF+me_vrD6wsM=2m http://192.168.1.74:9200

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

Part 3: Install Kibana

3.1 Install Kibana

sudo apt install -y kibana

3.2 Configure Kibana

sudo nano /etc/kibana/kibana.yml

Replace the entire file with:

# =================== System: Kibana Server ===================
server.port: 5601
server.host: "192.168.1.74"

# =================== System: Elasticsearch ===================
elasticsearch.hosts: ["http://192.168.1.74:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "cMn0FVIDe4N4GpyhcLlv"

# =================== System: Logging ===================
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

# =================== System: Other ===================
pid.file: /run/kibana/kibana.pid

Note: Replace cMn0FVIDe4N4GpyhcLlv with your actual kibana_system password from step 2.7.

3.3 Set Proper Permissions

sudo chown -R kibana:kibana /var/log/kibana
sudo chown -R kibana:kibana /var/lib/kibana
sudo chown -R kibana:kibana /etc/kibana

3.4 Start and Enable Kibana

sudo systemctl daemon-reload
sudo systemctl enable kibana
sudo systemctl start kibana
sudo systemctl status kibana

3.5 Access Kibana

Open browser and navigate to: http://192.168.1.74:5601

Wait 1-3 minutes for Kibana to fully initialize on first startup. You will see "Kibana server is not ready yet" until initialization completes.

Check Kibana is ready:

sudo journalctl -u kibana | grep "http server running"

Once ready, login with the ELASTIC SUPERUSER credentials from step 2.6:

Username: elastic
Password: tMOpyF+me_vrD6wsM=2m

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

Important: You login to Kibana with the elastic user (your admin account), NOT the kibana_system user. The kibana_system account is only used by Kibana application internally to communicate with Elasticsearch.

Part 4: Install Logstash

4.1 Install Logstash

sudo apt install -y logstash

4.2 Create Logstash Configuration for Beats Input

sudo nano /etc/logstash/conf.d/02-beats-input.conf

Add the following:

input {
  beats {
    port => 5044
  }
}

4.3 Create Logstash Filter Configuration

sudo nano /etc/logstash/conf.d/10-filter.conf

Add the following:

filter {
  if [agent][type] == "winlogbeat" {
    mutate {
      add_field => { "log_type" => "windows_event" }
    }
  }

  if [event][module] == "sysmon" {
    mutate {
      add_field => { "log_type" => "sysmon" }
    }
  }

  if [agent][type] == "filebeat" and [tags] and "zeek" in [tags] {
    mutate {
      add_field => { "log_type" => "zeek_network" }
    }
  }

  mutate {
    remove_field => [ "[agent][ephemeral_id]", "[agent][id]" ]
  }
}

4.4 Create Logstash Output Configuration

sudo nano /etc/logstash/conf.d/30-elasticsearch-output.conf

Add the following:

output {
  elasticsearch {
    hosts => ["http://192.168.1.74:9200"]
    user => "elastic"
    password => "tMOpyF+me_vrD6wsM=2m"
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

4.5 Start and Enable Logstash

sudo systemctl enable logstash
sudo systemctl start logstash

4.6 Verify Logstash

sudo systemctl status logstash
sudo tail -f /var/log/logstash/logstash-plain.log

Part 5: Install Zeek for Network Monitoring

5.1 Install Zeek

Option 1: Compile from Source (takes 30-40 minutes with 3 CPUs)

sudo apt install -y cmake make gcc g++ flex bison libpcap-dev libssl-dev python3 python3-dev swig zlib1g-dev
cd /tmp
wget https://download.zeek.org/zeek-6.0.0.tar.gz
tar -xzf zeek-6.0.0.tar.gz
cd zeek-6.0.0
./configure
make -j3
sudo make install

Option 2: Install from Pre-built Packages (recommended, takes seconds)

echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/ /' | sudo tee /etc/apt/sources.list.d/security:zeek.list
curl -fsSL https://download.opensuse.org/repositories/security:zeek/xUbuntu_22.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null
sudo apt update
sudo apt install -y zeek

5.2 Configure Zeek

export PATH=/usr/local/zeek/bin:$PATH
echo 'export PATH=/usr/local/zeek/bin:$PATH' >> ~/.bashrc

sudo nano /usr/local/zeek/etc/node.cfg

Modify the interface:

[zeek]
type=standalone
host=localhost
interface=<your_network_interface>

Find your network interface:

ip a

5.3 Configure Zeek Networks

sudo nano /usr/local/zeek/etc/networks.cfg

Add:

192.168.1.0/24    Private Network

5.4 Start Zeek

sudo /usr/local/zeek/bin/zeekctl deploy

5.5 Install Filebeat for Zeek Logs

sudo apt install -y filebeat

5.6 Configure Filebeat for Zeek

sudo nano /etc/filebeat/filebeat.yml

Replace the entire file with:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/local/zeek/logs/current/*.log
  tags: ["zeek"]
  fields:
    log_source: zeek
  fields_under_root: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

output.logstash:
  hosts: ["192.168.1.74:5044"]

setup.kibana:
  host: "192.168.1.74:5601"

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

5.7 Start Filebeat

sudo systemctl enable filebeat
sudo systemctl start filebeat

Part 6: Windows 11 Endpoint Configuration

6.1 Install Sysmon on Windows

Download Sysmon from: https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

Download Sysmon configuration from: https://github.com/SwiftOnSecurity/sysmon-config

Open PowerShell as Administrator:

cd C:\path\to\sysmon
.\Sysmon64.exe -accepteula -i sysmonconfig-export.xml

6.2 Install Winlogbeat on Windows

Download Winlogbeat from: https://www.elastic.co/downloads/beats/winlogbeat

Extract to C:\Program Files\Winlogbeat

6.3 Configure Winlogbeat

Edit C:\Program Files\Winlogbeat\winlogbeat.yml:

Replace the entire file with:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: System
  - name: Security
  - name: Microsoft-Windows-Sysmon/Operational
  - name: Windows PowerShell
    event_id: 400, 403, 600, 800
  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106
  - name: ForwardedEvents
    tags: [forwarded]
  - name: Microsoft-Windows-Windows Defender/Operational
  - name: Microsoft-Windows-AppLocker/EXE and DLL
  - name: Microsoft-Windows-AppLocker/MSI and Script
  - name: Microsoft-Windows-TaskScheduler/Operational
  - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
  - name: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
  - name: Microsoft-Windows-DNS-Client/Operational
  - name: Microsoft-Windows-WMI-Activity/Operational

setup.template.settings:
  index.number_of_shards: 1

output.logstash:
  hosts: ["192.168.1.74:5044"]

setup.kibana:
  host: "192.168.1.74:5601"

logging.level: info
logging.to_files: true
logging.files:
  path: C:\ProgramData\winlogbeat\Logs

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

6.4 Install Winlogbeat as Service

Open PowerShell as Administrator:

cd "C:\Program Files\Winlogbeat"
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
PowerShell.exe -ExecutionPolicy Bypass -File .\install-service-winlogbeat.ps1

6.5 Start Winlogbeat Service

Start-Service winlogbeat

6.6 Verify Winlogbeat is Running

Get-Service winlogbeat

6.7 Test Connectivity

Test-NetConnection -ComputerName 192.168.1.74 -Port 5044

6.8 Test Winlogbeat Configuration

cd "C:\Program Files\Winlogbeat"
.\winlogbeat.exe test config
.\winlogbeat.exe test output

The output should show connection to Logstash on port 5044.

Part 7: Verify Data Flow

7.1 Check Elasticsearch Indices

curl -u elastic:tMOpyF+me_vrD6wsM=2m http://192.168.1.74:9200/_cat/indices?v

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

You should see indices like:

winlogbeat-8.x.x-YYYY.MM.DD
filebeat-8.x.x-YYYY.MM.DD

7.2 Check Kibana Discover

Navigate to Kibana: http://192.168.1.74:5601

Go to: Menu > Discover

Create index patterns:

winlogbeat-*
filebeat-*

7.3 View Windows Logs

In Discover, select winlogbeat-* index pattern and you should see Windows events.

Filter for Sysmon events:

event.provider: "Microsoft-Windows-Sysmon"

7.4 View Network Traffic

In Discover, select filebeat-* index pattern and filter:

tags: "zeek"

Part 8: Create Data Views and View Logs

8.1 Create Zeek Network Traffic Data View

Go to Menu > Management > Stack Management > Kibana > Data Views
Click Create data view
Name: Zeek Network Traffic
Index pattern: filebeat-*
Timestamp field: @timestamp
Click Save data view to Kibana

8.2 Create Windows Events Data View

Click Create data view again
Name: Windows Events
Index pattern: winlogbeat-*
Timestamp field: @timestamp
Click Save data view to Kibana

8.3 View Logs in Discover

Go to Menu > Discover
Switch between data views using the dropdown
For Zeek logs: Select Zeek Network Traffic
For Windows logs: Select Windows Events

8.4 Filter for Sysmon Events

To view only Sysmon events in Windows data view:

Add filter: Field = event.provider, Operator = is, Value = Microsoft-Windows-Sysmon

8.5 Filter for Zeek Events

To view only Zeek network logs in filebeat data view:

Add filter: Field = tags, Operator = is, Value = zeek

Windows Events

Zeek Network Traffic

Part 9: Maintenance and Monitoring

9.1 Monitor Service Status

sudo systemctl status elasticsearch
sudo systemctl status kibana
sudo systemctl status logstash
sudo systemctl status filebeat

9.2 Check Logs

sudo tail -f /var/log/elasticsearch/elk-siem-cluster.log
sudo tail -f /var/log/kibana/kibana.log
sudo tail -f /var/log/logstash/logstash-plain.log
sudo tail -f /var/log/filebeat/filebeat

9.3 Zeek Maintenance

sudo /usr/local/zeek/bin/zeekctl status
sudo /usr/local/zeek/bin/zeekctl restart

9.4 Disk Space Management

df -h

Configure index lifecycle management in Kibana to manage old indices.

9.5 Update Components

sudo apt update
sudo apt upgrade elasticsearch kibana logstash filebeat

Troubleshooting

Issue: Repository errors during apt update (if you have old Elastic repos)

Remove old repository files:

sudo rm -f /etc/apt/sources.list.d/elastic-*.list
sudo apt update

Then proceed with Part 2 installation.

Issue: Elasticsearch won't start

Check memory settings:

sudo nano /etc/elasticsearch/jvm.options

Set heap size (50% of RAM):

-Xms3g
-Xmx3g

Issue: No data in Kibana

Check Logstash is receiving data:

sudo tail -f /var/log/logstash/logstash-plain.log

Check Winlogbeat on Windows:

Get-Content "C:\ProgramData\winlogbeat\Logs\winlogbeat"

Issue: Winlogbeat can't connect

Check firewall on Ubuntu:

sudo ufw status
sudo netstat -tulpn | grep 5044

Test connectivity from Windows:

Test-NetConnection -ComputerName 192.168.1.74 -Port 5044

Issue: Zeek not capturing traffic

Check network interface:

sudo /usr/local/zeek/bin/zeekctl status
ip a

Verify interface in node.cfg matches active interface.

Security Hardening

Enable HTTPS for Kibana

sudo nano /etc/kibana/kibana.yml

Add:

server.ssl.enabled: true
server.ssl.certificate: /path/to/cert.crt
server.ssl.key: /path/to/cert.key

Enable Elasticsearch Authentication

Already configured in Part 2.6

Restrict Network Access

sudo ufw delete allow 9200/tcp
sudo ufw allow from 192.168.1.0/24 to any port 9200

Performance Tuning

Elasticsearch

sudo nano /etc/elasticsearch/elasticsearch.yml

indices.memory.index_buffer_size: 30%
thread_pool.write.queue_size: 1000

Logstash

sudo nano /etc/logstash/logstash.yml

pipeline.workers: 2
pipeline.batch.size: 125
pipeline.batch.delay: 50

Backup and Recovery

Configure Backup Repository

Create backup directory:

sudo mkdir -p /backup/elasticsearch
sudo chown -R elasticsearch:elasticsearch /backup/elasticsearch

Configure Elasticsearch to allow backup path:

sudo nano /etc/elasticsearch/elasticsearch.yml

Add this line at the end of the file:

path.repo: ["/backup/elasticsearch"]

Restart Elasticsearch:

sudo systemctl restart elasticsearch
sudo systemctl status elasticsearch

Wait 30 seconds for Elasticsearch to fully start.

Backup Elasticsearch Data

Create the backup repository:

curl -X PUT "http://192.168.1.74:9200/_snapshot/backup_repo" -H 'Content-Type: application/json' -u elastic:tMOpyF+me_vrD6wsM=2m -d'
{
  "type": "fs",
  "settings": {
    "location": "/backup/elasticsearch"
  }
}'

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

Create Snapshot

curl -X PUT "http://192.168.1.74:9200/_snapshot/backup_repo/snapshot_1?wait_for_completion=true" -u elastic:tMOpyF+me_vrD6wsM=2m

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6.

Managing Daily Indices

ELK creates a new index daily (e.g., winlogbeat-9.1.5-2025.01.26, winlogbeat-9.1.5-2025.01.27). This is normal and beneficial for:

Performance: Smaller daily indices are faster to search
Management: Easy to delete old data by date
Backup: Can backup/restore specific days

Delete Old Indices

Via Kibana UI (Recommended):

Go to Menu > Management > Stack Management > Index Management
Select old indices by checking boxes
Click Manage indices > Delete indices
Confirm deletion

Via Command Line:

curl -X DELETE -u elastic:tMOpyF+me_vrD6wsM=2m "http://192.168.1.74:9200/winlogbeat-9.1.5-2024.03.23"

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password from step 2.6, and replace the index name with the actual index you want to delete.

Best Practice: Keep 7-30 days of recent data for a home lab SIEM.

Additional Resources

Official Documentation

Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
Kibana: https://www.elastic.co/guide/en/kibana/current/index.html
Logstash: https://www.elastic.co/guide/en/logstash/current/index.html
Winlogbeat: https://www.elastic.co/guide/en/beats/winlogbeat/current/index.html
Zeek: https://docs.zeek.org/en/master/

Sysmon Configuration

SwiftOnSecurity Config: https://github.com/SwiftOnSecurity/sysmon-config
Olaf Hartong Config: https://github.com/olafhartong/sysmon-modular

Implementation Summary

Installed Components

Ubuntu Server (192.168.1.74):

Elasticsearch 8.19.5 - Data storage and search engine
Logstash 8.19.5 - Log processing and filtering
Kibana 8.19.5 - Visualization and dashboards
Zeek 6.0.0 - Network traffic analysis
Filebeat 8.19.5 - Zeek log forwarding

Windows 11 Endpoint (192.168.1.91):

Sysmon - Advanced Windows event logging
Winlogbeat 9.1.5 - Windows event log forwarding

Authentication Credentials

Elastic Superuser (for Kibana login):

Username: elastic
Password: tMOpyF+me_vrD6wsM=2m

Kibana System Account (internal use only):

Username: kibana_system
Password: cMn0FVIDe4N4GpyhcLlv

Note: Replace these example passwords with your actual passwords generated during installation.

Key Configuration Files

Ubuntu Server:

Elasticsearch: /etc/elasticsearch/elasticsearch.yml
Kibana: /etc/kibana/kibana.yml
Logstash Input: /etc/logstash/conf.d/02-beats-input.conf
Logstash Filter: /etc/logstash/conf.d/10-filter.conf
Logstash Output: /etc/logstash/conf.d/30-elasticsearch-output.conf
Filebeat: /etc/filebeat/filebeat.yml
Zeek Node: /usr/local/zeek/etc/node.cfg
Zeek Networks: /usr/local/zeek/etc/networks.cfg

Windows Endpoint:

Winlogbeat: C:\Program Files\Winlogbeat\winlogbeat.yml
Sysmon Config: C:\path\to\sysmon\sysmonconfig-export.xml

Log Locations

Ubuntu Server:

Elasticsearch: /var/log/elasticsearch/elk-siem-cluster.log
Kibana: /var/log/kibana/kibana.log
Logstash: /var/log/logstash/logstash-plain.log
Filebeat: /var/log/filebeat/filebeat
Zeek: /usr/local/zeek/logs/current/

Windows Endpoint:

Winlogbeat: C:\ProgramData\winlogbeat\Logs\winlogbeat

Network Ports

| Port | Service | Purpose | |||| | 22 | SSH | Server management | | 5044 | Logstash | Beats input | | 9200 | Elasticsearch | HTTP API | | 5601 | Kibana | Web interface |

Essential Commands

Check Service Status:

sudo systemctl status elasticsearch
sudo systemctl status kibana
sudo systemctl status logstash
sudo systemctl status filebeat

View Logs:

sudo tail -f /var/log/logstash/logstash-plain.log
sudo journalctl -u kibana -f

Zeek Management:

sudo /usr/local/zeek/bin/zeekctl status
sudo /usr/local/zeek/bin/zeekctl restart

Check Indices:

curl -u elastic:tMOpyF+me_vrD6wsM=2m http://192.168.1.74:9200/_cat/indices?v

Note: Replace tMOpyF+me_vrD6wsM=2m with your actual elastic password.

Technical Architecture Summary

Multi-Layer Security Monitoring

Layer 1: Network Traffic Analysis

Zeek captures and analyzes all network traffic
Protocol detection and behavioral analysis
Connection logging and metadata extraction
Filebeat forwards logs to Logstash

Layer 2: Windows Endpoint Monitoring

Sysmon provides advanced process and network monitoring
Windows Security events capture authentication and access
PowerShell logging detects malicious script execution
Winlogbeat forwards all events to Logstash

Layer 3: Log Processing and Enrichment

Logstash receives data from all Beats agents
Filters add context and normalize data
Grok patterns parse unstructured logs
Processed data indexed in Elasticsearch

Layer 4: Storage and Search

Elasticsearch stores all security events
Daily indices provide optimal performance
Full-text search across all collected data
Retention policies manage storage usage

Layer 5: Visualization and Analysis

Kibana provides web-based interface
Data views separate Windows and network logs
Discover allows ad-hoc log analysis
Dashboards visualize security trends

Data Flow Architecture

Windows Events → Winlogbeat → Logstash → Elasticsearch → Kibana
Network Traffic → Zeek → Filebeat → Logstash → Elasticsearch → Kibana

Performance Characteristics

Response Time:

Event ingestion: < 1 second
Search queries: < 2 seconds
Dashboard loading: 2-5 seconds

Capacity:

Events per second: 1,000+
Storage per day: 1-5 GB
Retention: 7-30 days recommended

Resource Utilization:

RAM usage: ~4-5 GB
CPU usage: 20-40% average
Network bandwidth: 10-50 Mbps

Security Operations Use Cases

Threat Detection:

Detect brute force authentication attempts
Identify lateral movement patterns
Monitor for privilege escalation
Track suspicious PowerShell execution
Analyze network anomalies

Incident Response:

Timeline reconstruction from multiple sources
Correlation of network and endpoint events
Fast search across historical data
Evidence collection and preservation

Compliance and Auditing:

Centralized log retention
User activity monitoring
Access control auditing
Change tracking and reporting

Next Steps

Recommended Enhancements

Create Custom Dashboards
- Failed login attempts visualization
- Process creation timeline
- Network connection maps
- Top talkers and protocols
Configure Alerting
- Set up watchers for critical events
- Email notifications for security alerts
- Threshold-based anomaly detection
- Integration with ticketing systems
Implement Detection Rules
- MITRE ATT&CK framework mapping
- Sigma rule conversion
- Custom use-case specific rules
- Threat intelligence integration
Optimize Performance
- Index lifecycle management policies
- Hot-warm-cold architecture
- Curator for index management
- Query optimization and caching
Enhance Security
- Enable HTTPS for all communication
- Implement role-based access control
- Network segmentation for SIEM traffic
- Regular security updates and patching

Training Resources

Elastic Certified Engineer: https://www.elastic.co/training/certification
Free Elastic Webinars: https://www.elastic.co/webinars/
Zeek Training: https://zeek.org/training/
Windows Event Log Analysis: Microsoft documentation

Community Support

Elastic Community Forum: https://discuss.elastic.co/
Zeek Community: https://community.zeek.org/
Reddit r/SIEM: https://reddit.com/r/SIEM
GitHub Issues: Report bugs and request features

Congratulations! You have successfully built a fully functional ELK SIEM with network traffic analysis and Windows endpoint monitoring. Your SIEM is now collecting, processing, and visualizing security events from multiple sources in real-time.

All Content

VPS Security Blueprint

This blueprint provides comprehensive VPS security hardening through automated deployment of coordinated defense mechanisms. The implementation focuses on creating multiple security layers that communicate and synchronize threat intelligence across all services.

Blueprint Ubutnu Debian Suricata Wazuh Bash

ELK HOME

ELK SIEM Setup Guide - Ubuntu Server with Windows Endpoint

Table of Contents

Purpose & Scope

System Requirements

Ubuntu Server (SIEM Server)

Windows 11 (Endpoint/Agent)

Architecture Overview

Part 1: Ubuntu Server Setup

1.1 Initial System Configuration

1.2 Install Prerequisites

1.3 Increase Virtual Memory for Elasticsearch

1.4 Configure Firewall

Part 2: Install Elasticsearch

2.1 Import Elasticsearch GPG Key

2.2 Add Elasticsearch Repository

2.3 Install Elasticsearch

2.4 Configure Elasticsearch

2.5 Start and Enable Elasticsearch

2.6 Set Elasticsearch Password

2.7 Set Kibana System User Password

2.8 Verify Elasticsearch

Part 3: Install Kibana

3.1 Install Kibana

3.2 Configure Kibana

3.3 Set Proper Permissions

3.4 Start and Enable Kibana

3.5 Access Kibana

Part 4: Install Logstash

4.1 Install Logstash

4.2 Create Logstash Configuration for Beats Input

4.3 Create Logstash Filter Configuration

4.4 Create Logstash Output Configuration

4.5 Start and Enable Logstash

4.6 Verify Logstash

Part 5: Install Zeek for Network Monitoring

5.1 Install Zeek

5.2 Configure Zeek

5.3 Configure Zeek Networks

5.4 Start Zeek

5.5 Install Filebeat for Zeek Logs

5.6 Configure Filebeat for Zeek

5.7 Start Filebeat

Part 6: Windows 11 Endpoint Configuration

6.1 Install Sysmon on Windows

6.2 Install Winlogbeat on Windows

6.3 Configure Winlogbeat

6.4 Install Winlogbeat as Service

6.5 Start Winlogbeat Service

6.6 Verify Winlogbeat is Running

6.7 Test Connectivity

6.8 Test Winlogbeat Configuration

Part 7: Verify Data Flow

7.1 Check Elasticsearch Indices

7.2 Check Kibana Discover

7.3 View Windows Logs

7.4 View Network Traffic

Part 8: Create Data Views and View Logs

8.1 Create Zeek Network Traffic Data View

8.2 Create Windows Events Data View

8.3 View Logs in Discover

8.4 Filter for Sysmon Events

8.5 Filter for Zeek Events

Windows Events

Zeek Network Traffic

Part 9: Maintenance and Monitoring

9.1 Monitor Service Status

9.2 Check Logs

9.3 Zeek Maintenance

9.4 Disk Space Management

9.5 Update Components

Troubleshooting

Issue: Repository errors during apt update (if you have old Elastic repos)

Issue: Elasticsearch won't start

Issue: No data in Kibana

Issue: Winlogbeat can't connect

Issue: Zeek not capturing traffic

Security Hardening

Enable HTTPS for Kibana

Enable Elasticsearch Authentication