HMAC File Integrity Checker (java, written by Codex)

Codex wrote this java project from the task below. It installed and ran on a clean Ubuntu 22.04 machine as written.

Task given to the agent:

TASK: HMAC File Integrity Checker

Write a program that computes and verifies HMAC-based integrity checksums for files and directories, creating a manifest of checksums that can later be used to detect unauthorized modifications.

FUNCTIONAL REQUIREMENTS:
- Accept a file or directory path, a secret key, and a mode (generate or verify) as command-line arguments
- Support multiple hash algorithms selectable via --algorithm flag: SHA-256 (default), SHA-384, SHA-512, SHA3-256
- generate mode: Compute HMAC for each file and save a manifest file containing file paths, HMAC values, file sizes, and timestamps
- verify mode: Read the manifest, recompute HMACs, and report which files are unchanged, modified, added (present but not in manifest), or missing (in manifest but deleted)
- For directories, recursively process all files and support --exclude flag with glob patterns to skip files (e.g., --exclude "*.log,*.tmp")
- Support a --output flag for the manifest file path (default: integrity_manifest.json)
- Print a colored summary to console showing verification results: passed files in green, modified in red, missing in yellow, new files in blue
- Compute a master HMAC over the entire manifest to detect tampering of the manifest file itself
- Support incremental updates: when generating with an existing manifest, only recompute HMACs for files whose modification time has changed via --incremental flag
- If no arguments are given, generate sample files in a temporary directory, create an integrity manifest, modify one file, delete another, add a new file, then run verification to demonstrate all detection capabilities
- Handle binary and text files correctly, permission errors gracefully

Create a complete Java project for a clean Ubuntu 22.04 machine with only JDK 17+ installed. Include:
- Source code
- pom.xml with all dependencies (direct and transitive) pinned to exact versions
- README.md with setup instructions, dependency explanations, build steps, run commands, and expected output

[]

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.tmlr</groupId>
    <artifactId>hmac-file-integrity-checker</artifactId>
    <version>1.0.0</version>
    <name>hmac-file-integrity-checker</name>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20240303</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.13.0</version>
                <configuration>
                    <release>17</release>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>3.4.2</version>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>HmacFileIntegrityChecker</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

# HMAC File Integrity Checker (Java)

Computes and verifies HMAC manifests for files/directories to detect unauthorized changes.

## Requirements

- Ubuntu 22.04
- JDK 17+
- Maven 3.8+

## Dependencies

- Direct:
  - `org.json:json:20240303` (pinned in `pom.xml`)
- Transitive:
  - none

Build plugins are also pinned in `pom.xml`:
- `maven-compiler-plugin:3.13.0`
- `maven-jar-plugin:3.4.2`

## Build

```bash
mvn clean package
```

## Run

```bash
java -jar target/hmac-file-integrity-checker-1.0.0.jar <generate|verify> <path> <secret-key> [--algorithm SHA-256|SHA-384|SHA-512|SHA3-256] [--output integrity_manifest.json] [--exclude "*.log,*.tmp"] [--incremental]
```

Examples:

```bash
java -jar target/hmac-file-integrity-checker-1.0.0.jar generate ./data my-secret --algorithm SHA-512 --output ./integrity_manifest.json
java -jar target/hmac-file-integrity-checker-1.0.0.jar verify ./data my-secret --output ./integrity_manifest.json
java -jar target/hmac-file-integrity-checker-1.0.0.jar generate ./data my-secret --exclude "*.log,*.tmp" --incremental
```

## Behavior

- `generate` creates/updates a JSON manifest containing:
  - relative file path
  - HMAC digest
  - file size
  - mtime (ms)
  - timestamp
- Computes a `masterHmac` over canonical manifest content to detect manifest tampering.
- `verify` reports `UNCHANGED`, `MODIFIED`, `MISSING`, and `ADDED` in color.
- Supports recursive directories, exclusion globs, binary/text files, and incremental updates.
- No arguments runs a demo workflow.

import org.json.JSONArray;
import org.json.JSONObject;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileVisitOption;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileTime;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.regex.Pattern;
import java.util.stream.Stream;

public final class HmacFileIntegrityChecker {
    private static final String RESET = "\u001B[0m";
    private static final String GREEN = "\u001B[32m";
    private static final String RED = "\u001B[31m";
    private static final String YELLOW = "\u001B[33m";
    private static final String BLUE = "\u001B[34m";

    private static final class ParsedArgs {
        Map<String, String> options = new HashMap<>();
        List<String> positional = new ArrayList<>();
    }

    private static final class AlgorithmSpec {
        final String label;
        final String jceName;

        AlgorithmSpec(String label, String jceName) {
            this.label = label;
            this.jceName = jceName;
        }
    }

    private static final class Entry {
        String path;
        String hmac;
        long size;
        long mtimeMs;
        String timestamp;
    }

    private static final class CollectResult {
        List<Path> files;
        boolean rootIsDirectory;
    }

    private HmacFileIntegrityChecker() {}

    public static void main(String[] args) {
        try {
            ParsedArgs parsed = parseArgs(args);
            if (parsed.positional.isEmpty()) {
                runDemo();
                return;
            }
            if (parsed.positional.size() < 3) {
                throw new IllegalArgumentException(usage());
            }

            String mode = parsed.positional.get(0).toLowerCase();
            Path targetPath = Paths.get(parsed.positional.get(1)).toAbsolutePath();
            String secret = parsed.positional.get(2);
            String output = parsed.options.getOrDefault("output", "integrity_manifest.json");
            Path manifestPath = Paths.get(output).toAbsolutePath();
            String algorithmOpt = parsed.options.getOrDefault("algorithm", "sha-256");
            List<Pattern> excludes = parseExclude(parsed.options.getOrDefault("exclude", ""));
            boolean incremental = parsed.options.containsKey("incremental");

            switch (mode) {
                case "generate" -> generateMode(targetPath, secret, algorithmOpt, manifestPath, excludes, incremental);
                case "verify" -> {
                    boolean ok = verifyMode(targetPath, secret, algorithmOpt, manifestPath, excludes);
                    if (!ok) System.exit(2);
                }
                default -> throw new IllegalArgumentException("Unknown mode. Use generate or verify.");
            }
        } catch (Exception e) {
            System.err.println("Error: " + e.getMessage());
            System.exit(1);
        }
    }

    private static String usage() {
        return String.join("\n",
                "Usage:",
                "  java -jar target/hmac-file-integrity-checker-1.0.0.jar <generate|verify> <path> <secret-key>",
                "    [--algorithm SHA-256|SHA-384|SHA-512|SHA3-256]",
                "    [--output integrity_manifest.json] [--exclude \"*.log,*.tmp\"] [--incremental]",
                "",
                "No arguments runs a demo.");
    }

    private static ParsedArgs parseArgs(String[] args) {
        ParsedArgs out = new ParsedArgs();
        for (int i = 0; i < args.length; i++) {
            String token = args[i];
            if (token.startsWith("--")) {
                String key = token.substring(2);
                if (i + 1 < args.length && !args[i + 1].startsWith("--")) out.options.put(key, args[++i]);
                else out.options.put(key, "true");
            } else {
                out.positional.add(token);
            }
        }
        return out;
    }

    private static AlgorithmSpec normalizeAlgorithm(String value) {
        String v = value.toLowerCase().replace("_", "-");
        return switch (v) {
            case "sha-256", "sha256" -> new AlgorithmSpec("sha-256", "HmacSHA256");
            case "sha-384", "sha384" -> new AlgorithmSpec("sha-384", "HmacSHA384");
            case "sha-512", "sha512" -> new AlgorithmSpec("sha-512", "HmacSHA512");
            case "sha3-256" -> new AlgorithmSpec("sha3-256", "HmacSHA3-256");
            default -> throw new IllegalArgumentException("Unsupported algorithm. Use SHA-256, SHA-384, SHA-512, or SHA3-256.");
        };
    }

    private static List<Pattern> parseExclude(String raw) {
        List<Pattern> patterns = new ArrayList<>();
        if (raw == null || raw.isBlank()) return patterns;
        String[] parts = raw.split(",");
        for (String p : parts) {
            String trimmed = p.trim();
            if (trimmed.isEmpty()) continue;
            patterns.add(globToRegex(trimmed));
        }
        return patterns;
    }

    private static Pattern globToRegex(String glob) {
        StringBuilder out = new StringBuilder("^");
        for (int i = 0; i < glob.length(); i++) {
            char c = glob.charAt(i);
            if (c == '*') out.append(".*");
            else if (c == '?') out.append(".");
            else if ("\\.^$|()[]{}+".indexOf(c) >= 0) out.append("\\").append(c);
            else out.append(c);
        }
        out.append("$");
        return Pattern.compile(out.toString(), Pattern.CASE_INSENSITIVE);
    }

    private static String toPortable(Path p) {
        return p.toString().replace('\\', '/');
    }

    private static boolean shouldExclude(String relPath, String baseName, List<Pattern> patterns) {
        for (Pattern pattern : patterns) {
            if (pattern.matcher(relPath).matches() || pattern.matcher(baseName).matches()) return true;
        }
        return false;
    }

    private static CollectResult collectFiles(Path target, List<Pattern> excludes) throws IOException {
        if (Files.isRegularFile(target)) {
            CollectResult result = new CollectResult();
            result.files = List.of(target);
            result.rootIsDirectory = false;
            return result;
        }
        if (!Files.isDirectory(target)) throw new IllegalArgumentException("Target path must be a file or directory.");

        List<Path> files = new ArrayList<>();
        try (Stream<Path> stream = Files.walk(target, FileVisitOption.FOLLOW_LINKS)) {
            stream.filter(Files::isRegularFile).forEach(path -> {
                String rel = toPortable(target.relativize(path));
                if (!shouldExclude(rel, path.getFileName().toString(), excludes)) files.add(path);
            });
        }
        files.sort(Comparator.comparing(Path::toString));
        CollectResult result = new CollectResult();
        result.files = files;
        result.rootIsDirectory = true;
        return result;
    }

    private static String relativeKey(Path root, Path file, boolean rootIsDirectory) {
        if (rootIsDirectory) return toPortable(root.relativize(file));
        return file.getFileName().toString();
    }

    private static String toHex(byte[] bytes) {
        StringBuilder sb = new StringBuilder(bytes.length * 2);
        for (byte b : bytes) sb.append(String.format("%02x", b));
        return sb.toString();
    }

    private static String hmacBytes(byte[] bytes, String secret, String jceAlgorithm) throws Exception {
        Mac mac = Mac.getInstance(jceAlgorithm);
        mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), jceAlgorithm));
        mac.update(bytes);
        return toHex(mac.doFinal());
    }

    private static String hmacFile(Path file, String secret, String jceAlgorithm) throws Exception {
        Mac mac = Mac.getInstance(jceAlgorithm);
        mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), jceAlgorithm));
        try (InputStream in = Files.newInputStream(file)) {
            byte[] buffer = new byte[1024 * 1024];
            int read;
            while ((read = in.read(buffer)) >= 0) {
                if (read > 0) mac.update(buffer, 0, read);
            }
        }
        return toHex(mac.doFinal());
    }

    private static String isoFromMs(long ms) {
        return Instant.ofEpochMilli(ms).toString();
    }

    private static String buildMasterPayload(JSONObject manifestNoMaster) {
        List<Entry> entries = readEntries(manifestNoMaster.getJSONArray("entries"));
        entries.sort(Comparator.comparing(e -> e.path));
        StringBuilder payload = new StringBuilder();
        payload.append(manifestNoMaster.optInt("version", 1)).append('\n');
        payload.append(manifestNoMaster.optString("algorithm", "sha-256")).append('\n');
        payload.append(manifestNoMaster.optString("rootPath", "")).append('\n');
        payload.append(manifestNoMaster.optString("generatedAt", "")).append('\n');
        for (Entry e : entries) {
            payload.append(e.path).append('|')
                    .append(e.size).append('|')
                    .append(e.mtimeMs).append('|')
                    .append(e.timestamp).append('|')
                    .append(e.hmac).append('\n');
        }
        return payload.toString();
    }

    private static String computeMasterHmac(JSONObject manifestNoMaster, String secret, String jceAlgorithm) throws Exception {
        return hmacBytes(buildMasterPayload(manifestNoMaster).getBytes(StandardCharsets.UTF_8), secret, jceAlgorithm);
    }

    private static JSONObject buildManifestNoMaster(String algorithmLabel, Path rootPath, List<Entry> entries) {
        JSONObject out = new JSONObject();
        out.put("version", 1);
        out.put("generatedAt", Instant.now().toString());
        out.put("rootPath", rootPath.toString());
        out.put("algorithm", algorithmLabel);
        JSONArray array = new JSONArray();
        for (Entry e : entries) {
            JSONObject item = new JSONObject();
            item.put("path", e.path);
            item.put("hmac", e.hmac);
            item.put("size", e.size);
            item.put("mtimeMs", e.mtimeMs);
            item.put("timestamp", e.timestamp);
            array.put(item);
        }
        out.put("entries", array);
        return out;
    }

    private static List<Entry> readEntries(JSONArray array) {
        List<Entry> out = new ArrayList<>();
        for (int i = 0; i < array.length(); i++) {
            JSONObject item = array.getJSONObject(i);
            Entry e = new Entry();
            e.path = item.optString("path", "");
            e.hmac = item.optString("hmac", "");
            e.size = item.optLong("size", 0L);
            e.mtimeMs = item.optLong("mtimeMs", 0L);
            e.timestamp = item.optString("timestamp", "");
            out.add(e);
        }
        return out;
    }

    private static void generateMode(
            Path target,
            String secret,
            String algorithmInput,
            Path manifestPath,
            List<Pattern> excludes,
            boolean incremental
    ) throws Exception {
        AlgorithmSpec algorithm = normalizeAlgorithm(algorithmInput);
        CollectResult collect = collectFiles(target, excludes);

        Map<String, Entry> previous = new HashMap<>();
        if (incremental && Files.exists(manifestPath)) {
            String text = Files.readString(manifestPath, StandardCharsets.UTF_8);
            JSONObject prior = new JSONObject(text);
            for (Entry e : readEntries(prior.optJSONArray("entries") == null ? new JSONArray() : prior.getJSONArray("entries"))) {
                previous.put(e.path, e);
            }
        }

        List<Entry> entries = new ArrayList<>();
        for (Path file : collect.files) {
            String rel = relativeKey(target, file, collect.rootIsDirectory);
            long size = Files.size(file);
            FileTime mtime = Files.getLastModifiedTime(file);
            long mtimeMs = mtime.toMillis();
            Entry prior = previous.get(rel);

            Entry current = new Entry();
            current.path = rel;
            current.size = size;
            current.mtimeMs = mtimeMs;
            current.timestamp = isoFromMs(mtimeMs);
            if (prior != null && prior.size == size && prior.mtimeMs == mtimeMs) current.hmac = prior.hmac;
            else current.hmac = hmacFile(file, secret, algorithm.jceName);
            entries.add(current);
        }
        entries.sort(Comparator.comparing(e -> e.path));

        JSONObject manifestNoMaster = buildManifestNoMaster(algorithm.label, target.toAbsolutePath(), entries);
        JSONObject manifest = new JSONObject(manifestNoMaster.toString());
        manifest.put("masterHmac", computeMasterHmac(manifestNoMaster, secret, algorithm.jceName));

        Files.createDirectories(manifestPath.getParent());
        Files.writeString(manifestPath, manifest.toString(2) + "\n", StandardCharsets.UTF_8);
        System.out.println("Manifest written: " + manifestPath);
        System.out.println("Processed files: " + entries.size());
        System.out.println("Algorithm: " + algorithm.label);
        System.out.println("Incremental: " + (incremental ? "enabled" : "disabled"));
    }

    private static boolean verifyMode(
            Path target,
            String secret,
            String algorithmInput,
            Path manifestPath,
            List<Pattern> excludes
    ) throws Exception {
        if (!Files.exists(manifestPath)) throw new IllegalArgumentException("Manifest not found: " + manifestPath);
        JSONObject manifest = new JSONObject(Files.readString(manifestPath, StandardCharsets.UTF_8));
        JSONArray entriesJson = manifest.optJSONArray("entries");
        if (entriesJson == null) throw new IllegalArgumentException("Manifest malformed: entries missing.");
        List<Entry> manifestEntries = readEntries(entriesJson);

        AlgorithmSpec manifestAlgorithm = normalizeAlgorithm(manifest.optString("algorithm", "sha-256"));
        AlgorithmSpec effectiveAlgorithm = normalizeAlgorithm(algorithmInput);
        JSONObject manifestNoMaster = new JSONObject();
        manifestNoMaster.put("version", manifest.optInt("version", 1));
        manifestNoMaster.put("generatedAt", manifest.optString("generatedAt", ""));
        manifestNoMaster.put("rootPath", manifest.optString("rootPath", ""));
        manifestNoMaster.put("algorithm", manifestAlgorithm.label);
        manifestNoMaster.put("entries", entriesJson);

        String expectedMaster = computeMasterHmac(manifestNoMaster, secret, manifestAlgorithm.jceName);
        boolean masterValid = expectedMaster.equals(manifest.optString("masterHmac", ""));

        CollectResult collect = collectFiles(target, excludes);
        Map<String, Path> currentByRel = new HashMap<>();
        for (Path file : collect.files) currentByRel.put(relativeKey(target, file, collect.rootIsDirectory), file);

        List<String> unchanged = new ArrayList<>();
        List<String> modified = new ArrayList<>();
        List<String> missing = new ArrayList<>();
        Set<String> seen = new HashSet<>();
        for (Entry entry : manifestEntries) {
            Path fullPath = currentByRel.get(entry.path);
            if (fullPath == null) {
                missing.add(entry.path);
                continue;
            }
            seen.add(entry.path);
            String digest = hmacFile(fullPath, secret, effectiveAlgorithm.jceName);
            if (digest.equals(entry.hmac)) unchanged.add(entry.path);
            else modified.add(entry.path);
        }
        List<String> added = new ArrayList<>();
        for (String rel : currentByRel.keySet()) {
            if (!seen.contains(rel)) added.add(rel);
        }
        unchanged.sort(String::compareTo);
        modified.sort(String::compareTo);
        missing.sort(String::compareTo);
        added.sort(String::compareTo);

        if (!masterValid) System.out.println(colorize(RED, "Manifest master HMAC mismatch: manifest may be tampered."));
        if (!effectiveAlgorithm.label.equals(manifestAlgorithm.label)) {
            System.out.println(colorize(YELLOW, "Using --algorithm " + effectiveAlgorithm.label
                    + " instead of manifest algorithm " + manifestAlgorithm.label + "."));
        }
        for (String rel : unchanged) System.out.println(colorize(GREEN, "UNCHANGED " + rel));
        for (String rel : modified) System.out.println(colorize(RED, "MODIFIED  " + rel));
        for (String rel : missing) System.out.println(colorize(YELLOW, "MISSING   " + rel));
        for (String rel : added) System.out.println(colorize(BLUE, "ADDED     " + rel));

        System.out.println();
        System.out.println("Summary");
        System.out.println("  unchanged: " + unchanged.size());
        System.out.println("  modified : " + modified.size());
        System.out.println("  missing  : " + missing.size());
        System.out.println("  added    : " + added.size());
        System.out.println("  manifest : " + (masterValid ? "valid" : "tampered"));

        return masterValid && modified.isEmpty() && missing.isEmpty() && added.isEmpty();
    }

    private static String colorize(String color, String text) {
        return color + text + RESET;
    }

    private static void runDemo() throws Exception {
        Path root = Files.createTempDirectory("hmac-integrity-");
        Files.writeString(root.resolve("alpha.txt"), "alpha\n", StandardCharsets.UTF_8);
        Files.writeString(root.resolve("beta.txt"), "beta\n", StandardCharsets.UTF_8);
        Files.createDirectories(root.resolve("nested"));
        Files.write(root.resolve("nested").resolve("gamma.bin"), "binary-data".getBytes(StandardCharsets.UTF_8));

        String secret = "demo-secret";
        Path manifest = root.resolve("integrity_manifest.json");
        System.out.println("Demo directory: " + root);
        generateMode(root, secret, "sha-256", manifest, List.of(), false);

        Files.writeString(root.resolve("alpha.txt"), "alpha\ntampered\n", StandardCharsets.UTF_8);
        Files.deleteIfExists(root.resolve("beta.txt"));
        Files.writeString(root.resolve("new.txt"), "new file\n", StandardCharsets.UTF_8);

        boolean ok = verifyMode(root, secret, "sha-256", manifest, List.of());
        System.out.println("Demo verification result: " + (ok ? "PASS" : "FAIL (expected for demo)"));
    }
}