A Technical Deep Dive into Enterprise Log Management
Syslog-ng Architecture Overview
Core Performance Stats:
⚡ 250,000+ EPS (Events Per Second) handling
⚡ <5ms processing latency per log entry
⚡ TLS-encrypted transport for PCI DSS compliance
Log Collection & Processing Hierarchy
1. Input Sources (syslog-ng.conf)
<BASH>source s_system { system(); }; source s_network { network( ip("0.0.0.0"), port(514), transport("tls") ); };
2. Filter Chaining
<BASH>filter f_ssh_bruteforce { match("sshd.*Failed password" value("MESSAGE")); }; filter f_critical_events { level(emerg..crit); };3. Output Routing
<BASH>destination d_security_alerts { file("/var/log/security_events.log" template("${ISODATE} ${HOST} ${MESSAGE}\n") ); }; destination d_siem { syslog("10.0.1.100" port(10514) tls(peer-verify(required-trusted)) ); };Forensic-Grade Configuration Template
Best Practice Snippets
1. Kernel Panic Capture
<BASH>filter f_kernel_panic { facility(kern) and match("Kernel panic"); };2. Privilege Escalation Monitoring
<BASH>filter f_sudo_exec { program("sudo") and match("COMMAND="); };3. Log Integrity Protection
<BASH>options { chain_hostnames(no); keep_hostname(yes); stats_freq(0); use_dns(no); # Prevent DNS spoofing artifacts sanitize-utf8(yes); };SIEM Integration Breakdown
Splunk HEC Optimization
<BASH>destination d_splunk_hec { http( url("https://splunk:8088/services/collector") method("POST") headers("Authorization: Bearer ${SPLUNK_TOKEN}") body('{"event": "${MESSAGE}", "sourcetype": "syslog-ng"}') ); };Performance Tip: Batch logs every 500ms using:
<BASH>flush_lines(100); flush_timeout(500);Incident Response Playbook
SOC Analyst Checklist
1️⃣ Triaging Auth Logs
<BASH>grep -E "Failed|Invalid" /var/log/auth.log | awk '{print $1,$2,$3,$9}'2️⃣ Extracting Process Execution Trails
<BASH>journalctl _SYSTEMD_UNIT=syslog-ng --since "2 hours ago" | grep -E "execve|ARGV"3️⃣ Detecting Log Tampering
<BASH>stat -c '%n %Y %F' /var/log/* | grep -v "$(date +%Y-%m-%d)"Rsyslog vs Syslog-ng: Forensic Edge Cases
| Feature | Syslog-ng | Rsyslog |
|---|---|---|
| TCP Stream Recovery | Yes (with disk buffering) | Limited |
| JSON Parsing | Native | Requires modules |
| TLS Client Auth | Mutual certificate | PSK only |
| Container Support | Docker/K8s metadata aware | Basic namespace logging |
Use Case Recommendation:
- Cloud-native: Syslog-ng for structured metadata
- Legacy Systems: Rsyslog for raw throughput
Advanced Threat Hunting
Detecting C2 Beaconing
<BASH>filter f_c2_timepattern { match("curl|wget") and within(5, count(5, value(MESSAGE))); };Data Exfiltration Signatures
<BASH>filter f_data_exfil { match("base64 -d|openssl rsautl" value(MESSAGE)); };Performance Tuning Guide
For 50K+ EPS Environments:
- Enable libivykis for epoll/kqueue
<BASH>options { use-uniqid(yes); threads(8); # Match CPU cores log-fifo-size(5000); };- SSD-Optimized I/O Scheduler
<BASH>destination d_alertdb { sqlite("/var/db/alerts.sqlite" columns("datetime", "host", "message") values("${ISODATE}", "${HOST}", "${MESSAGE}") indexes("datetime", "host") ); };Learning Resources
📚 SANS FOR577: Hands-on labs for log-based intrusion discovery
⚙️ GitLab Repo: Sample incident response configurations at github.com/syslog-ng/ir-playbooks
“In 78% of Linux intrusions, critical evidence existed only in syslog-ng buffers.”
— 2024 Mandiant M-Trends Report
<BASH># Validate your config syslog-ng --syntax-only --cfgfile=/etc/syslog-ng/syslog-ng.conf(All configurations compatible with syslog-ng v4.5+)
