How to Audit System Logs for Suspicious Activity in Debian 12 Bookworm

System logs serve as a crucial resource for system administrators in maintaining security, diagnosing issues, and ensuring system stability. Auditing logs helps in detecting unauthorized access, malicious activities, or system misconfigurations. In this guide, we will discuss how to audit system logs for suspicious activity in Debian 12 Bookworm effectively.

Understanding System Logs in Debian 12

Debian 12 uses systemd as its init system, meaning most logs are managed by journald. However, traditional log files still exist, and understanding where to find them is the first step in log auditing.

Common Log Locations

1. Journal logs (managed by systemd-journald):

   • Stored in binary format and accessed using journalctl
   • Location: /var/log/journal/ (if persistent logging is enabled)

2. Syslog logs (traditional log files managed by rsyslog):

   • General system logs: /var/log/syslog
   • Authentication logs: /var/log/auth.log
   • Kernel logs: /var/log/kern.log
   • Boot logs: /var/log/boot.log
   • Cron logs: /var/log/cron.log

Enabling Persistent Journal Logging

By default, journal logs may be stored only in memory. To ensure logs are available after reboots, enable persistent logging:

sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

Analyzing System Logs for Suspicious Activity

1. Checking General System Logs

Use the journalctl command to review system logs:

journalctl -xe

To filter logs by a specific date:

journalctl --since "2024-04-01" --until "2024-04-02"

For real-time log monitoring:

journalctl -f

2. Analyzing Authentication Logs

Authentication logs (/var/log/auth.log) provide insights into login attempts and user authentication activities. To check failed login attempts:

grep "Failed password" /var/log/auth.log

To check successful root logins:

grep "session opened for user root" /var/log/auth.log

Suspicious entries may include repeated failed login attempts or logins from unusual IP addresses.

3. Identifying SSH Attacks

SSH attacks are common on public-facing servers. Check logs for brute-force attempts:

grep "Failed password" /var/log/auth.log | wc -l

Check the source IP addresses of failed logins:

grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -nr | head

4. Examining Kernel and System Errors

Kernel logs contain important system messages:

dmesg | less

To find critical errors:

dmesg | grep -i "error"

5. Checking Boot Logs

To check for boot-related issues:

journalctl -b -1

This command shows logs from the previous boot session.

6. Monitoring Cron Jobs

Attackers may install malicious cron jobs. Review scheduled jobs:

cat /var/spool/cron/crontabs/*

Check cron logs:

grep CRON /var/log/syslog

Using Log Analysis Tools

Logwatch

Logwatch provides summaries of system logs and highlights unusual activities. Install it using:

sudo apt install logwatch

Generate a report:

sudo logwatch --detail High --mailto your-email@example.com

Fail2Ban

Fail2Ban monitors log files and bans IPs with repeated failed login attempts.

Install Fail2Ban:

sudo apt install fail2ban

Start the service:

sudo systemctl enable --now fail2ban

Check banned IPs:

sudo fail2ban-client status sshd

Rsyslog and Logrotate

Ensure logs are properly rotated to prevent storage issues:

sudo systemctl status rsyslog
cat /etc/logrotate.conf

Automating Log Auditing

Automate log analysis using cron jobs or shell scripts. Example script to email failed SSH attempts:

#!/bin/bash
LOG=/var/log/auth.log
EMAIL=your-email@example.com
grep "Failed password" $LOG | mail -s "SSH Login Attempts" $EMAIL

Schedule this script using cron:

crontab -e

Add:

0 * * * * /path/to/script.sh

Conclusion

Regular auditing of system logs in Debian 12 is essential for security and stability. By using tools like journalctl, logwatch, and fail2ban, administrators can proactively detect and mitigate threats. Automating log analysis ensures ongoing monitoring and helps prevent security breaches.