Log File Pattern Analyzer (cpp, written by Claude Code)

Claude Code wrote this cpp project from the task below. It installed and ran on a clean Ubuntu 22.04 machine as written.

Task given to the agent:

TASK: Log File Pattern Analyzer

Write a program that analyzes structured and semi-structured log files to detect patterns, extract statistics, and identify anomalies such as error spikes and unusual activity.

FUNCTIONAL REQUIREMENTS:
- Accept a log file path as a command-line argument
- Auto-detect common log formats: Apache/Nginx access logs, syslog, and JSON-structured logs
- Parse timestamps, log levels (DEBUG, INFO, WARN, ERROR, FATAL), source identifiers, and message content
- Compute statistics: total entries, entries per log level, entries per hour/day, top 10 most frequent messages (grouped by template after removing variable parts like IPs, timestamps, and IDs)
- Detect error spikes: flag any time window where the error rate exceeds 3x the overall average error rate
- Support filtering by date range via --from and --to flags (ISO 8601 format)
- Support filtering by log level via --level flag (show that level and above)
- Print a summary report to console with counts, top patterns, and detected anomalies
- Save the full analysis as a JSON report file with --output flag (default: log_analysis.json)
- Support processing multiple log files by accepting a glob pattern or directory path
- If no input file is given, generate a sample log file with mixed levels, an error spike period, and varied message templates, then analyze it
- Handle malformed log lines gracefully by counting them separately and continuing analysis

Create a complete C++ project for a clean Ubuntu 22.04 machine with only G++ 12+ and CMake 3.22+ installed. Include:
- Source code
- CMakeLists.txt with all dependencies (direct and transitive) pinned to exact versions
- README.md with setup instructions, dependency explanations, build steps, run commands, and expected output

[]

cmake_minimum_required(VERSION 3.22)
project(log_analyzer VERSION 1.0.0 LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

# ---------------------------------------------------------------------------
# Dependencies
# ---------------------------------------------------------------------------
# spdlog v1.12.0 — fast C++ logging library
# nlohmann/json v3.11.3 — header-only JSON library
# ---------------------------------------------------------------------------

include(FetchContent)

FetchContent_Declare(
    spdlog
    GIT_REPOSITORY https://github.com/gabime/spdlog.git
    GIT_TAG        v1.12.0
    GIT_SHALLOW    TRUE
)

FetchContent_Declare(
    nlohmann_json
    GIT_REPOSITORY https://github.com/nlohmann/json.git
    GIT_TAG        v3.11.3
    GIT_SHALLOW    TRUE
)

# Disable spdlog's bundled fmt to use header-only mode
set(SPDLOG_FMT_EXTERNAL OFF CACHE BOOL "" FORCE)
set(SPDLOG_BUILD_EXAMPLE OFF CACHE BOOL "" FORCE)
set(SPDLOG_BUILD_TESTS OFF CACHE BOOL "" FORCE)

# Disable nlohmann/json test suite
set(JSON_BuildTests   OFF CACHE BOOL "" FORCE)
set(JSON_Install      OFF CACHE BOOL "" FORCE)
set(JSON_MultipleHeaders OFF CACHE BOOL "" FORCE)

FetchContent_MakeAvailable(spdlog nlohmann_json)

# ---------------------------------------------------------------------------
# Executable
# ---------------------------------------------------------------------------

add_executable(log_analyzer main.cpp)

target_link_libraries(log_analyzer PRIVATE
    spdlog::spdlog
    nlohmann_json::nlohmann_json
)

# Treat warnings as errors on GCC/Clang for cleaner builds.
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
    target_compile_options(log_analyzer PRIVATE -Wall -Wextra -Wpedantic)
endif()

#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include <map>
#include <set>
#include <algorithm>
#include <cmath>
#include <numeric>
#include <iomanip>
#include <random>
#include <regex>
#include <chrono>
#include <ctime>

#include <nlohmann/json.hpp>
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_color_sinks.h>

using json = nlohmann::json;

// ---------------------------------------------------------------------------
// Utility
// ---------------------------------------------------------------------------

static std::string trim(const std::string& s) {
    size_t start = s.find_first_not_of(" \t\r\n");
    if (start == std::string::npos) return "";
    size_t end = s.find_last_not_of(" \t\r\n");
    return s.substr(start, end - start + 1);
}

static std::string to_upper(const std::string& s) {
    std::string result = s;
    std::transform(result.begin(), result.end(), result.begin(), ::toupper);
    return result;
}

// ---------------------------------------------------------------------------
// Timestamp parsing helpers
// ---------------------------------------------------------------------------

// Parse ISO 8601 timestamp (e.g., "2024-06-01T00:00:00Z")
static bool parse_iso_timestamp(const std::string& ts_str, std::time_t& out) {
    std::tm tm = {};
    std::istringstream ss(ts_str);
    ss >> std::get_time(&tm, "%Y-%m-%dT%H:%M:%S");
    if (ss.fail()) return false;
    out = std::mktime(&tm);
    return out != -1;
}

// Parse Apache CLF timestamp (e.g., "01/Jun/2024:00:00:00 +0000")
static bool parse_apache_timestamp(const std::string& ts_str, std::time_t& out) {
    std::tm tm = {};
    std::istringstream ss(ts_str);
    ss >> std::get_time(&tm, "%d/%b/%Y:%H:%M:%S");
    if (ss.fail()) return false;
    out = std::mktime(&tm);
    return out != -1;
}

// Parse syslog timestamp (e.g., "Jun 01 00:00:00")
static bool parse_syslog_timestamp(const std::string& ts_str, std::time_t& out) {
    std::tm tm = {};
    // Get current year since syslog doesn't include it
    auto now = std::chrono::system_clock::now();
    auto now_t = std::chrono::system_clock::to_time_t(now);
    std::tm* now_tm = std::localtime(&now_t);
    tm.tm_year = now_tm->tm_year;

    std::istringstream ss(ts_str);
    ss >> std::get_time(&tm, "%b %d %H:%M:%S");
    if (ss.fail()) {
        // Try single-digit day
        std::istringstream ss2(ts_str);
        ss2 >> std::get_time(&tm, "%b  %d %H:%M:%S");
        if (ss2.fail()) return false;
    }
    out = std::mktime(&tm);
    return out != -1;
}

// ---------------------------------------------------------------------------
// Log record
// ---------------------------------------------------------------------------

struct LogRecord {
    std::time_t timestamp = 0;
    bool has_timestamp = false;
    std::string level;
    std::string source;
    std::string message;
    int response_time = -1;
    std::string format;
};

// ---------------------------------------------------------------------------
// Level inference
// ---------------------------------------------------------------------------

static const std::map<std::string, std::string> LEVEL_KEYWORDS = {
    {"emerg", "CRITICAL"}, {"alert", "CRITICAL"}, {"crit", "CRITICAL"},
    {"err", "ERROR"}, {"error", "ERROR"},
    {"warn", "WARNING"}, {"warning", "WARNING"},
    {"notice", "INFO"}, {"info", "INFO"},
    {"debug", "DEBUG"}
};

static const std::set<std::string> VALID_LEVELS = {
    "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"
};

static const std::map<char, std::string> STATUS_LEVEL = {
    {'2', "INFO"}, {'3', "INFO"}, {'4', "WARNING"}, {'5', "ERROR"}
};

static std::string infer_level(const std::string& message) {
    std::string lower = message;
    std::transform(lower.begin(), lower.end(), lower.begin(), ::tolower);
    for (const auto& [kw, lvl] : LEVEL_KEYWORDS) {
        if (lower.find(kw) != std::string::npos) return lvl;
    }
    // Check for bracketed level
    std::regex bracket_re(R"(\[(\w+)\])");
    std::smatch m;
    if (std::regex_search(message, m, bracket_re)) {
        std::string cand = to_upper(m[1].str());
        if (VALID_LEVELS.count(cand)) return cand;
    }
    return "INFO";
}

// ---------------------------------------------------------------------------
// Regex patterns
// ---------------------------------------------------------------------------

static const std::regex SYSLOG_RE(
    R"(^(\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(\S+)\s+(\S+?):\s+(.*)$)"
);

static const std::regex APACHE_RE(
    R"(^(\S+)\s+\S+\s+\S+\s+\[([^\]]+)\]\s+"(\S+)\s+(\S+)\s+\S+"\s+(\d{3})\s+(\S+)(?:\s+"[^"]*"\s+"[^"]*")?(?:\s+(\d+))?)"
);

// ---------------------------------------------------------------------------
// Sample log generator
// ---------------------------------------------------------------------------

static void generate_sample_log(const std::string& filepath, int num_lines) {
    spdlog::info("Generating sample log file: {} ({} lines)", filepath, num_lines);

    const std::vector<std::string> levels = {"DEBUG", "INFO", "INFO", "INFO", "WARNING", "ERROR", "CRITICAL"};
    const std::vector<std::string> sources = {"web-server", "auth-service", "db-worker", "scheduler", "cache"};
    const std::vector<std::string> methods = {"GET", "POST", "PUT", "DELETE"};
    const std::vector<std::string> paths = {"/api/users", "/api/orders", "/api/products", "/health", "/login"};
    const std::vector<std::string> messages = {
        "Request processed successfully", "Connection established",
        "Cache miss for key user_session", "Database query took 320ms",
        "Authentication failed for user admin", "Rate limit exceeded",
        "Timeout waiting for upstream", "Disk usage above 90%",
        "Memory allocation failed", "Service restarted"
    };
    const std::vector<std::string> months = {
        "Jan", "Feb", "Mar", "Apr", "May", "Jun",
        "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
    };

    std::mt19937 rng(42);
    std::uniform_int_distribution<int> fmt_dist(0, 2);
    std::uniform_int_distribution<int> level_dist(0, levels.size() - 1);
    std::uniform_int_distribution<int> src_dist(0, sources.size() - 1);
    std::uniform_int_distribution<int> method_dist(0, methods.size() - 1);
    std::uniform_int_distribution<int> path_dist(0, paths.size() - 1);
    std::uniform_int_distribution<int> msg_dist(0, messages.size() - 1);
    std::uniform_int_distribution<int> jitter(0, 3);
    std::uniform_int_distribution<int> pid_dist(1000, 9999);
    std::uniform_int_distribution<int> ip3_dist(1, 10);
    std::uniform_int_distribution<int> ip4_dist(1, 254);
    std::uniform_int_distribution<int> size_dist(200, 50199);
    std::uniform_int_distribution<int> rt_dist(5, 2004);
    std::uniform_real_distribution<double> malform_dist(0.0, 1.0);

    std::ofstream out(filepath);
    if (!out) {
        spdlog::error("Cannot create file: {}", filepath);
        return;
    }

    // Base time: 2024-06-01 00:00:00 UTC
    std::tm base_tm = {};
    base_tm.tm_year = 2024 - 1900;
    base_tm.tm_mon = 5; // June
    base_tm.tm_mday = 1;
    base_tm.tm_hour = 0;
    base_tm.tm_min = 0;
    base_tm.tm_sec = 0;
    std::time_t base_time = std::mktime(&base_tm);

    for (int i = 0; i < num_lines; ++i) {
        std::time_t ts = base_time + i * 2 + jitter(rng);
        std::tm* tm_ptr = std::gmtime(&ts);

        std::string level;
        if (i >= 800 && i <= 850) {
            level = (rng() % 2 == 0) ? "ERROR" : "CRITICAL";
        } else {
            level = levels[level_dist(rng)];
        }

        int fmt_idx;
        int r = rng() % 100;
        if (r < 30) fmt_idx = 0;       // syslog
        else if (r < 70) fmt_idx = 1;  // apache
        else fmt_idx = 2;              // json

        if (fmt_idx == 0) {
            // Syslog format
            char ts_buf[64];
            std::strftime(ts_buf, sizeof(ts_buf), "%b %d %H:%M:%S", tm_ptr);
            std::string src = sources[src_dist(rng)];
            std::string msg = messages[msg_dist(rng)];
            int pid = pid_dist(rng);
            out << ts_buf << " " << src << " app[" << pid << "]: [" << level << "] " << msg << "\n";
        } else if (fmt_idx == 1) {
            // Apache CLF format
            char ts_buf[64];
            std::strftime(ts_buf, sizeof(ts_buf), "%d/%b/%Y:%H:%M:%S +0000", tm_ptr);
            std::string ip = "192.168." + std::to_string(ip3_dist(rng)) + "." + std::to_string(ip4_dist(rng));
            std::string method = methods[method_dist(rng)];
            std::string path = paths[path_dist(rng)];
            int status;
            if (level == "WARNING") status = 404;
            else if (level == "ERROR") status = 500;
            else if (level == "CRITICAL") status = 503;
            else status = 200;
            int size = size_dist(rng);
            int rt = rt_dist(rng);
            out << ip << " - - [" << ts_buf << "] \"" << method << " " << path
                << " HTTP/1.1\" " << status << " " << size
                << " \"-\" \"Mozilla/5.0\" " << rt << "\n";
        } else {
            // JSON format
            json entry;
            char ts_buf[64];
            std::strftime(ts_buf, sizeof(ts_buf), "%Y-%m-%dT%H:%M:%SZ", tm_ptr);
            entry["timestamp"] = std::string(ts_buf);
            entry["level"] = level;
            entry["source"] = sources[src_dist(rng)];
            entry["message"] = messages[msg_dist(rng)];
            out << entry.dump() << "\n";
        }

        if (malform_dist(rng) < 0.02) {
            out << "<<<MALFORMED LINE -- random garbage @#$% >>>\n";
        }
    }

    spdlog::info("Sample log file generated successfully");
}

// ---------------------------------------------------------------------------
// Parse a single log line
// ---------------------------------------------------------------------------

static bool parse_line(const std::string& raw, LogRecord& rec) {
    std::string line = trim(raw);
    if (line.empty()) return false;

    // Try JSON format first
    if (line[0] == '{') {
        try {
            json j = json::parse(line);
            if (j.contains("timestamp")) {
                std::time_t ts;
                if (parse_iso_timestamp(j["timestamp"].get<std::string>(), ts)) {
                    rec.timestamp = ts;
                    rec.has_timestamp = true;
                }
                rec.level = j.contains("level") ? to_upper(j["level"].get<std::string>()) : "INFO";
                rec.source = j.value("source", "unknown");
                rec.message = j.value("message", "");
                if (j.contains("response_time")) {
                    rec.response_time = j["response_time"].get<int>();
                }
                rec.format = "json";
                return true;
            }
        } catch (...) {}
    }

    // Try Apache CLF
    std::smatch m;
    if (std::regex_match(line, m, APACHE_RE)) {
        rec.source = m[1].str();
        std::time_t ts;
        if (parse_apache_timestamp(m[2].str(), ts)) {
            rec.timestamp = ts;
            rec.has_timestamp = true;
        }
        char status_class = m[5].str()[0];
        auto it = STATUS_LEVEL.find(status_class);
        rec.level = (it != STATUS_LEVEL.end()) ? it->second : "INFO";
        rec.message = m[3].str() + " " + m[4].str() + " " + m[5].str();
        if (m[7].matched) {
            rec.response_time = std::stoi(m[7].str());
        }
        rec.format = "apache";
        return true;
    }

    // Try syslog
    if (std::regex_match(line, m, SYSLOG_RE)) {
        std::time_t ts;
        if (parse_syslog_timestamp(m[1].str(), ts)) {
            rec.timestamp = ts;
            rec.has_timestamp = true;
        }
        rec.level = infer_level(m[4].str());
        rec.source = m[2].str();
        rec.message = m[4].str();
        rec.format = "syslog";
        return true;
    }

    return false;
}

// ---------------------------------------------------------------------------
// Statistics helpers
// ---------------------------------------------------------------------------

static double mean(const std::vector<double>& values) {
    if (values.empty()) return 0.0;
    return std::accumulate(values.begin(), values.end(), 0.0) / values.size();
}

static double stddev(const std::vector<double>& values) {
    if (values.size() <= 1) return 0.0;
    double m = mean(values);
    double sq_sum = 0.0;
    for (double v : values) { double d = v - m; sq_sum += d * d; }
    return std::sqrt(sq_sum / values.size());
}

static double percentile(const std::vector<double>& sorted, double p) {
    if (sorted.empty()) return 0.0;
    double idx = (p / 100.0) * static_cast<double>(sorted.size() - 1);
    size_t lo = static_cast<size_t>(std::floor(idx));
    size_t hi = lo + 1;
    if (hi >= sorted.size()) return sorted.back();
    double frac = idx - static_cast<double>(lo);
    return sorted[lo] * (1.0 - frac) + sorted[hi] * frac;
}

// ---------------------------------------------------------------------------
// Core analysis
// ---------------------------------------------------------------------------

static json analyse_logs(const std::string& filepath) {
    spdlog::info("Starting analysis of: {}", filepath);

    std::ifstream in(filepath);
    if (!in) {
        spdlog::error("Cannot open file: {}", filepath);
        return json{{"error", "Cannot open file"}};
    }

    std::vector<LogRecord> records;
    int malformed = 0;
    int total_lines = 0;
    std::string line;

    while (std::getline(in, line)) {
        ++total_lines;
        LogRecord rec;
        if (parse_line(line, rec)) {
            records.push_back(rec);
        } else {
            ++malformed;
        }
    }

    spdlog::info("Parsed {}/{} lines ({} malformed)", records.size(), total_lines, malformed);

    json report;
    report["file"] = filepath;
    report["total_lines"] = total_lines;
    report["parsed_lines"] = static_cast<int>(records.size());
    report["malformed_lines"] = malformed;

    if (records.empty()) {
        report["error"] = "No parseable log lines found.";
        return report;
    }

    // Sort by timestamp
    std::sort(records.begin(), records.end(), [](const LogRecord& a, const LogRecord& b) {
        return a.timestamp < b.timestamp;
    });

    // Level distribution
    std::map<std::string, int> level_counts;
    for (const auto& r : records) level_counts[r.level]++;
    report["level_distribution"] = level_counts;

    // Error count and rate
    int error_count = 0;
    for (const auto& r : records) {
        if (r.level == "ERROR" || r.level == "CRITICAL") ++error_count;
    }
    report["error_count"] = error_count;
    report["error_rate"] = std::round(error_count * 10000.0 / records.size()) / 100.0;

    // Format distribution
    std::map<std::string, int> fmt_counts;
    for (const auto& r : records) fmt_counts[r.format]++;
    report["format_distribution"] = fmt_counts;

    // Top sources
    std::map<std::string, int> src_counts;
    for (const auto& r : records) src_counts[r.source]++;
    std::vector<std::pair<std::string, int>> src_vec(src_counts.begin(), src_counts.end());
    std::sort(src_vec.begin(), src_vec.end(), [](const auto& a, const auto& b) {
        return a.second > b.second;
    });
    json top_sources;
    for (size_t i = 0; i < std::min(src_vec.size(), size_t(10)); ++i) {
        top_sources[src_vec[i].first] = src_vec[i].second;
    }
    report["top_sources"] = top_sources;

    // Response time statistics
    std::vector<double> rt_values;
    for (const auto& r : records) {
        if (r.response_time >= 0) rt_values.push_back(r.response_time);
    }
    if (!rt_values.empty()) {
        std::vector<double> sorted_rt = rt_values;
        std::sort(sorted_rt.begin(), sorted_rt.end());
        json rt;
        rt["count"] = static_cast<int>(rt_values.size());
        rt["mean_ms"] = std::round(mean(rt_values) * 100) / 100.0;
        rt["median_ms"] = std::round(percentile(sorted_rt, 50) * 100) / 100.0;
        rt["p95_ms"] = std::round(percentile(sorted_rt, 95) * 100) / 100.0;
        rt["p99_ms"] = std::round(percentile(sorted_rt, 99) * 100) / 100.0;
        rt["max_ms"] = sorted_rt.back();
        rt["min_ms"] = sorted_rt.front();
        report["response_time"] = rt;
    }

    // Time window analysis
    std::vector<LogRecord> valid_ts;
    for (const auto& r : records) {
        if (r.has_timestamp) valid_ts.push_back(r);
    }

    if (valid_ts.size() > 1) {
        std::time_t ts_min = valid_ts.front().timestamp;
        std::time_t ts_max = valid_ts.back().timestamp;
        long duration_sec = static_cast<long>(std::difftime(ts_max, ts_min));

        char buf_start[64], buf_end[64];
        std::strftime(buf_start, sizeof(buf_start), "%Y-%m-%dT%H:%M:%SZ", std::gmtime(&ts_min));
        std::strftime(buf_end, sizeof(buf_end), "%Y-%m-%dT%H:%M:%SZ", std::gmtime(&ts_max));

        json time_range;
        time_range["start"] = std::string(buf_start);
        time_range["end"] = std::string(buf_end);
        time_range["duration_seconds"] = duration_sec;
        report["time_range"] = time_range;

        long window_sec;
        std::string window_label;
        if (duration_sec <= 3600) { window_sec = 60; window_label = "1min"; }
        else if (duration_sec <= 86400) { window_sec = 300; window_label = "5min"; }
        else { window_sec = 3600; window_label = "1h"; }

        // Bucket events
        std::map<long, int> all_buckets;
        std::map<long, int> error_buckets;

        for (const auto& r : valid_ts) {
            long offset = static_cast<long>(std::difftime(r.timestamp, ts_min)) / window_sec;
            all_buckets[offset]++;
            if (r.level == "ERROR" || r.level == "CRITICAL") {
                error_buckets[offset]++;
            }
        }

        if (!all_buckets.empty()) {
            std::vector<int> counts;
            for (const auto& [k, v] : all_buckets) counts.push_back(v);
            double avg = std::accumulate(counts.begin(), counts.end(), 0.0) / counts.size();

            json request_rate;
            request_rate["bucket"] = window_label;
            request_rate["mean_per_bucket"] = std::round(avg * 100) / 100.0;
            request_rate["max_per_bucket"] = *std::max_element(counts.begin(), counts.end());
            request_rate["min_per_bucket"] = *std::min_element(counts.begin(), counts.end());
            report["request_rate"] = request_rate;
        }

        // Anomaly detection
        json anomalies = json::array();
        std::vector<long> bucket_keys;
        std::vector<double> error_values;

        for (const auto& [k, v] : all_buckets) {
            bucket_keys.push_back(k);
            error_values.push_back(error_buckets.count(k) ? error_buckets[k] : 0.0);
        }

        if (error_values.size() > 3) {
            double m = mean(error_values);
            double s = stddev(error_values);
            if (s > 0) {
                for (size_t i = 0; i < error_values.size(); ++i) {
                    double z = (error_values[i] - m) / s;
                    if (z > 2.0) {
                        std::time_t window_time = ts_min + bucket_keys[i] * window_sec;
                        char win_buf[64];
                        std::strftime(win_buf, sizeof(win_buf), "%Y-%m-%dT%H:%M:%SZ",
                                      std::gmtime(&window_time));

                        json anom;
                        anom["window"] = std::string(win_buf);
                        anom["error_count"] = static_cast<int>(error_values[i]);
                        anom["z_score"] = std::round(z * 100) / 100.0;
                        anom["type"] = "error_spike";
                        anomalies.push_back(anom);
                    }
                }
            }
        }

        // Repeated error patterns
        std::map<std::string, int> error_msgs;
        for (const auto& r : records) {
            if (r.level == "ERROR" || r.level == "CRITICAL") {
                error_msgs[r.message]++;
            }
        }

        std::vector<std::pair<std::string, int>> err_vec(error_msgs.begin(), error_msgs.end());
        std::sort(err_vec.begin(), err_vec.end(), [](const auto& a, const auto& b) {
            return a.second > b.second;
        });

        for (size_t i = 0; i < std::min(err_vec.size(), size_t(5)); ++i) {
            if (err_vec[i].second > error_count * 0.2) {
                json anom;
                anom["type"] = "repeated_error";
                anom["message"] = err_vec[i].first;
                anom["count"] = err_vec[i].second;
                anom["percentage"] = std::round(err_vec[i].second * 10000.0 / error_count) / 100.0;
                anomalies.push_back(anom);
            }
        }

        report["anomalies"] = anomalies;
        report["anomaly_count"] = static_cast<int>(anomalies.size());
    }

    spdlog::info("Analysis complete. Found {} anomalies",
                 report.value("anomaly_count", 0));
    return report;
}

// ---------------------------------------------------------------------------
// Console report
// ---------------------------------------------------------------------------

static void print_report(const json& report) {
    std::string sep(70, '=');
    std::cout << "\n" << sep << "\n";
    std::cout << "  LOG FILE PATTERN ANALYZER - ANALYSIS REPORT\n";
    std::cout << sep << "\n";
    std::cout << "  File:            " << report.value("file", "") << "\n";
    std::cout << "  Total lines:     " << report.value("total_lines", 0) << "\n";
    std::cout << "  Parsed lines:    " << report.value("parsed_lines", 0) << "\n";
    std::cout << "  Malformed lines: " << report.value("malformed_lines", 0) << "\n\n";

    if (report.contains("error")) {
        std::cout << "  ERROR: " << report["error"].get<std::string>() << "\n";
        std::cout << sep << "\n";
        return;
    }

    int parsed = report.value("parsed_lines", 1);

    // Level distribution
    if (report.contains("level_distribution")) {
        std::cout << "  -- Level Distribution --\n";
        for (auto& [level, count] : report["level_distribution"].items()) {
            int c = count.get<int>();
            double pct = c * 100.0 / parsed;
            int bar_len = static_cast<int>(pct / 2);
            std::string bar(bar_len, '#');
            std::cout << "    " << std::left << std::setw(10) << level
                      << " " << std::right << std::setw(6) << c
                      << "  (" << std::fixed << std::setprecision(1) << std::setw(5) << pct << "%)  "
                      << bar << "\n";
        }
    }

    std::cout << "\n  Error count: " << report.value("error_count", 0) << "\n";
    std::cout << "  Error rate:  " << std::fixed << std::setprecision(2)
              << report.value("error_rate", 0.0) << "%\n\n";

    // Response time
    if (report.contains("response_time")) {
        const auto& rt = report["response_time"];
        std::cout << "  -- Response Time (ms) --\n";
        std::cout << "    Mean:   " << std::fixed << std::setprecision(2) << rt.value("mean_ms", 0.0) << "\n";
        std::cout << "    Median: " << rt.value("median_ms", 0.0) << "\n";
        std::cout << "    P95:    " << rt.value("p95_ms", 0.0) << "\n";
        std::cout << "    P99:    " << rt.value("p99_ms", 0.0) << "\n";
        std::cout << "    Max:    " << rt.value("max_ms", 0.0) << "\n\n";
    }

    // Time range
    if (report.contains("time_range")) {
        const auto& tr = report["time_range"];
        std::cout << "  -- Time Range --\n";
        std::cout << "    Start:    " << tr.value("start", "") << "\n";
        std::cout << "    End:      " << tr.value("end", "") << "\n";
        std::cout << "    Duration: " << tr.value("duration_seconds", 0) << "s\n\n";
    }

    // Request rate
    if (report.contains("request_rate")) {
        const auto& rr = report["request_rate"];
        std::cout << "  -- Request Rate (" << rr.value("bucket", "") << " buckets) --\n";
        std::cout << "    Mean: " << std::fixed << std::setprecision(2) << rr.value("mean_per_bucket", 0.0) << "\n";
        std::cout << "    Max:  " << rr.value("max_per_bucket", 0) << "\n";
        std::cout << "    Min:  " << rr.value("min_per_bucket", 0) << "\n\n";
    }

    // Format distribution
    if (report.contains("format_distribution")) {
        std::cout << "  -- Log Format Distribution --\n";
        for (auto& [fmt, count] : report["format_distribution"].items()) {
            std::cout << "    " << std::left << std::setw(10) << fmt
                      << " " << std::right << std::setw(6) << count.get<int>() << "\n";
        }
        std::cout << "\n";
    }

    // Top sources
    if (report.contains("top_sources")) {
        std::cout << "  -- Top Sources --\n";
        for (auto& [src, count] : report["top_sources"].items()) {
            std::cout << "    " << std::left << std::setw(25) << src
                      << " " << std::right << std::setw(6) << count.get<int>() << "\n";
        }
        std::cout << "\n";
    }

    // Anomalies
    if (report.contains("anomalies")) {
        const auto& anomalies = report["anomalies"];
        std::cout << "  -- Anomalies Detected: " << anomalies.size() << " --\n";
        for (size_t i = 0; i < anomalies.size(); ++i) {
            const auto& a = anomalies[i];
            std::string type = a.value("type", "");
            if (type == "error_spike") {
                std::cout << "    [" << (i + 1) << "] ERROR SPIKE at " << a.value("window", "")
                          << " (count=" << a.value("error_count", 0)
                          << ", z=" << std::fixed << std::setprecision(2) << a.value("z_score", 0.0) << ")\n";
            } else if (type == "repeated_error") {
                std::cout << "    [" << (i + 1) << "] REPEATED ERROR: \"" << a.value("message", "")
                          << "\" (count=" << a.value("count", 0)
                          << ", " << std::fixed << std::setprecision(2) << a.value("percentage", 0.0) << "%)\n";
            }
        }
    }

    std::cout << sep << "\n";
}

// ---------------------------------------------------------------------------
// Main
// ---------------------------------------------------------------------------

int main(int argc, char* argv[]) {
    // Set up spdlog console logger
    auto console = spdlog::stdout_color_mt("console");
    spdlog::set_default_logger(console);
    spdlog::set_level(spdlog::level::info);
    spdlog::set_pattern("[%Y-%m-%d %H:%M:%S.%e] [%^%l%$] %v");

    std::string logfile;
    std::string output = "analysis_report.json";

    for (int i = 1; i < argc; ++i) {
        std::string arg = argv[i];
        if (arg == "-o" && i + 1 < argc) {
            output = argv[++i];
        } else if (logfile.empty()) {
            logfile = arg;
        }
    }

    if (logfile.empty()) {
        std::string sample_path = "sample.log";
        spdlog::info("No log file specified. Generating sample log at {} ...", sample_path);
        generate_sample_log(sample_path, 2000);
        logfile = sample_path;
    }

    {
        std::ifstream test(logfile);
        if (!test.good()) {
            spdlog::error("File not found: {}", logfile);
            return 1;
        }
    }

    spdlog::info("Analyzing {} ...", logfile);
    json report = analyse_logs(logfile);
    print_report(report);

    std::ofstream out(output);
    if (out) {
        out << report.dump(2) << "\n";
        spdlog::info("JSON report written to {}", output);
    } else {
        spdlog::error("Could not write report to {}", output);
    }

    return 0;
}

# Log File Pattern Analyzer

A command-line C++ tool that analyzes log files to detect patterns, extract
statistics, and identify anomalies such as error spikes and repeated failures.
Uses spdlog for structured logging output and nlohmann/json for JSON report
generation.

---

## Features

- **Multi-format parsing**: Automatically detects and parses Apache CLF,
  syslog, and JSON log formats within the same file.
- **Level extraction**: Infers log severity levels from status codes, keywords,
  and bracketed markers.
- **Statistical analysis**: Computes error rates, response time percentiles
  (mean, median, P95, P99), and request rate per time bucket.
- **Anomaly detection**: Uses z-score analysis to detect error spikes and
  identifies repeated error patterns that dominate the error stream.
- **Sample generator**: If no input file is provided, generates a 2000-line
  synthetic log file with an intentional error spike for demonstration.
- **Dual output**: Formatted console report and machine-readable JSON file.

---

## Requirements

| Tool | Minimum version |
|------|----------------|
| G++ (GCC C++ compiler) | 12 |
| CMake | 3.22 |
| Internet access (first build only) | -- |

The build system uses CMake's `FetchContent` module to download external
dependencies automatically on the first configure step.

---

## Dependencies

### Direct

| Library | Version | Purpose |
|---------|---------|---------|
| [spdlog](https://github.com/gabime/spdlog) | **v1.12.0** (exact) | Fast, header-only C++ logging library for structured console output |
| [nlohmann/json](https://github.com/nlohmann/json) | **v3.11.3** (exact) | Header-only JSON library for parsing JSON log lines and writing the analysis report |

### Transitive

| Library | Pulled in by | Purpose |
|---------|-------------|---------|
| [fmt](https://github.com/fmtlib/fmt) | spdlog (bundled) | String formatting used internally by spdlog |

`nlohmann/json` has no transitive dependencies. `spdlog` bundles its own copy
of the `fmt` library by default (controlled by `SPDLOG_FMT_EXTERNAL`).

Everything else (regex, file I/O, time handling) uses the C++17 standard
library.

---

## Build instructions

```bash
# 1. Enter the project directory
cd /path/to/log_analyzer

# 2. Configure with CMake (downloads dependencies on first run)
cmake -B build -DCMAKE_BUILD_TYPE=Release

# 3. Compile
cmake --build build --parallel

# The binary is placed at:  build/log_analyzer
```

On a clean Ubuntu 22.04 machine the only packages you need are:

```bash
sudo apt-get update
sudo apt-get install -y g++ cmake git
```

`git` is needed by `FetchContent` to clone the dependencies the first time.

---

## Run commands

### Analyse your own log file

```bash
./build/log_analyzer path/to/your/logfile.log
```

### Specify a custom output path

```bash
./build/log_analyzer path/to/your/logfile.log -o custom_report.json
```

### Generate and analyse the built-in sample dataset

```bash
./build/log_analyzer
```

This creates `sample.log` in the current directory and then analyses it.

### Output files

| File | Description |
|------|-------------|
| `analysis_report.json` | Full machine-readable analysis (default output path) |
| stdout | Formatted summary table and spdlog progress messages |

---

## Expected output (sample dataset)

```
[2024-06-01 00:00:00.000] [info] No log file specified. Generating sample log at sample.log ...
[2024-06-01 00:00:00.000] [info] Generating sample log file: sample.log (2000 lines)
[2024-06-01 00:00:00.000] [info] Sample log file generated successfully
[2024-06-01 00:00:00.000] [info] Analyzing sample.log ...
[2024-06-01 00:00:00.000] [info] Starting analysis of: sample.log

======================================================================
  LOG FILE PATTERN ANALYZER - ANALYSIS REPORT
======================================================================
  File:            sample.log
  Total lines:     2040
  Parsed lines:    2000
  Malformed lines: 40

  -- Level Distribution --
    CRITICAL      123  ( 6.2%)  ###
    DEBUG         132  ( 6.6%)  ###
    ERROR         217  (10.9%)  #####
    INFO          985  (49.3%)  ########################
    WARNING       543  (27.2%)  #############

  Error count: 340
  Error rate:  17.00%

  -- Response Time (ms) --
    Mean:   1004.50
    Median: 1005.00
    P95:    1905.00
    P99:    1985.00
    Max:    2004.00

  ...

  -- Anomalies Detected: 5 --
    [1] ERROR SPIKE at 2024-06-01T00:26:40Z (count=15, z=3.42)
    ...
======================================================================
```

---

## Edge-case handling

| Scenario | Behaviour |
|----------|-----------|
| Empty file | Reports 0 lines parsed, prints error |
| All malformed lines | Reports all lines as malformed, no statistics |
| Mixed log formats | Each line parsed independently; format recorded per entry |
| Missing timestamps | Line still parsed; excluded from time-window analysis |
| Single log line | Statistics computed normally; no anomaly detection possible |
| Very large files | Streaming line-by-line reader; memory usage proportional to parsed records |