HTTP REST Client with Retry Logic (java, written by Codex)

Codex wrote this java project from the task below. It installed and ran on a clean Ubuntu 22.04 machine as written.

Task given to the agent:

TASK: HTTP REST Client with Retry Logic

Write a program that sends HTTP requests to REST APIs with configurable retry logic, response validation, request chaining, and comprehensive logging of request/response cycles.

FUNCTIONAL REQUIREMENTS:
- Accept a URL as a command-line argument with --method flag (GET, POST, PUT, PATCH, DELETE; default: GET)
- Support sending request body via --data flag (JSON string) or --file flag (read body from a file)
- Support custom headers via --header flag (repeatable, e.g., --header "Content-Type: application/json" --header "Authorization: Bearer token")
- Implement configurable retry logic: --retries flag (default 3), --backoff flag for backoff strategy (linear, exponential; default: exponential), --retry-delay flag for initial delay in seconds (default 1)
- Retry only on configurable HTTP status codes via --retry-on flag (default: 429, 500, 502, 503, 504)
- Support request timeout via --timeout flag (default 30 seconds)
- Support request chaining from a JSON recipe file via --chain flag: execute a sequence of requests where later requests can reference values from earlier responses using JSONPath expressions
- Log complete request/response cycles: method, URL, headers, body, status code, response headers, response body, timing, and retry attempts
- Support response validation via --expect-status flag (e.g., --expect-status 200) and --expect-json flag (JSONPath expression that must exist in the response)
- Print formatted response to console: status code, headers, and pretty-printed body (auto-detect JSON and format it)
- Save the full request/response log as JSON with --output flag (default: request_log.json)
- If no URL is given, demonstrate the client by sending requests to a public test API (httpbin.org or similar), showing GET, POST with JSON body, retry on simulated failure, and a simple request chain
- Handle errors: connection refused, DNS failure, SSL errors, malformed JSON responses, and redirect loops

Create a complete Java project for a clean Ubuntu 22.04 machine with only JDK 17+ installed. Include:
- Source code
- pom.xml with all dependencies (direct and transitive) pinned to exact versions
- README.md with setup instructions, dependency explanations, build steps, run commands, and expected output

[]

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.tmlr.codegen</groupId>
  <artifactId>http-rest-client-with-retry</artifactId>
  <version>1.0.0</version>
  <name>HTTP REST Client With Retry</name>

  <properties>
    <maven.compiler.source>17</maven.compiler.source>
    <maven.compiler.target>17</maven.compiler.target>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
        <version>2.11.0</version>
      </dependency>
      <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.9.0</version>
      </dependency>
      <dependency>
        <groupId>net.minidev</groupId>
        <artifactId>json-smart</artifactId>
        <version>2.5.1</version>
      </dependency>
      <dependency>
        <groupId>net.minidev</groupId>
        <artifactId>accessors-smart</artifactId>
        <version>2.5.1</version>
      </dependency>
      <dependency>
        <groupId>org.ow2.asm</groupId>
        <artifactId>asm</artifactId>
        <version>9.7.1</version>
      </dependency>
      <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>2.0.13</version>
      </dependency>
      <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>2.0.13</version>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>com.google.code.gson</groupId>
      <artifactId>gson</artifactId>
    </dependency>
    <dependency>
      <groupId>com.jayway.jsonpath</groupId>
      <artifactId>json-path</artifactId>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-simple</artifactId>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.13.0</version>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.3.0</version>
        <configuration>
          <mainClass>HttpRestClientWithRetry</mainClass>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

# HTTP REST Client with Retry Logic (Java)

CLI REST client with retry policy, response validation, request chaining, and request/response cycle logging.

## Requirements
- Ubuntu 22.04
- JDK 17+
- Maven 3.8+

## Dependencies
- `com.google.code.gson:gson:2.11.0` for JSON parsing/formatting and log output
- `com.jayway.jsonpath:json-path:2.9.0` for JSONPath validation and chain references
- `org.slf4j:slf4j-simple:2.0.13` as logging backend required by `json-path`
- Transitive dependencies are pinned in `dependencyManagement` (`json-smart`, `accessors-smart`, `asm`, `slf4j-api`)

## Build
```bash
mvn -q -DskipTests compile
```

## Run
```bash
mvn -q exec:java -Dexec.args="https://httpbin.org/get --method GET --expect-status 200 --expect-json $.url --output request_log.json"
```

Request with body and headers:
```bash
mvn -q exec:java -Dexec.args="https://httpbin.org/post --method POST --data '{\"hello\":\"world\"}' --header 'Content-Type: application/json' --retries 3 --backoff exponential --retry-delay 1"
```

Chain mode:
```bash
mvn -q exec:java -Dexec.args="--chain chain.json --output request_log.json"
```

Demo mode (no URL given):
```bash
mvn -q exec:java
```

## Example `chain.json`
```json
{
  "requests": [
    {
      "method": "GET",
      "url": "https://httpbin.org/get",
      "expect_status": 200
    },
    {
      "method": "POST",
      "url": "https://httpbin.org/post",
      "headers": {
        "Content-Type": "application/json"
      },
      "data": "{\"source\":\"{{0:$.url}}\"}",
      "expect_status": 200,
      "expect_json": "$.json.source"
    }
  ]
}
```

Template syntax in chain files: `{{<stepIndex>:<jsonPath>}}`

## Output
- Console: status code, headers, and pretty-printed body
- JSON log file (`--output`, default `request_log.json`) containing attempts, timings, retries, validation results, and response payloads

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import com.jayway.jsonpath.JsonPath;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.time.Instant;
import java.util.*;

public final class HttpRestClientWithRetry {
    private static final Gson GSON = new GsonBuilder().setPrettyPrinting().create();

    private HttpRestClientWithRetry() {}

    private record ParsedArgs(Map<String, String> options, Map<String, List<String>> lists, List<String> positional) {}

    public static void main(String[] args) {
        try {
            ParsedArgs parsed = parseArgs(args);
            Map<String, Object> defaults = new LinkedHashMap<>();
            defaults.put("method", parsed.options.getOrDefault("method", "GET"));
            defaults.put("headers", parseHeaders(parsed.lists.getOrDefault("header", List.of())));
            defaults.put("retries", Integer.parseInt(parsed.options.getOrDefault("retries", "3")));
            defaults.put("backoff", parsed.options.getOrDefault("backoff", "exponential"));
            defaults.put("retryDelay", Double.parseDouble(parsed.options.getOrDefault("retry-delay", "1")));
            defaults.put("retryOn", parseRetryOn(parsed.options.getOrDefault("retry-on", "429,500,502,503,504")));
            defaults.put("timeout", Double.parseDouble(parsed.options.getOrDefault("timeout", "30")));

            Path output = Paths.get(parsed.options.getOrDefault("output", "request_log.json")).toAbsolutePath();
            if (parsed.options.containsKey("chain")) {
                String chainText = Files.readString(Paths.get(parsed.options.get("chain")).toAbsolutePath(), StandardCharsets.UTF_8);
                JsonElement chainJson = JsonParser.parseString(chainText);
                List<Map<String, Object>> chain = parseChain(chainJson);
                runChain(chain, defaults, output);
                return;
            }

            if (parsed.positional.isEmpty()) {
                runChain(defaultDemoChain(), defaults, output);
                return;
            }

            String url = parsed.positional.get(0);
            String body = null;
            if (parsed.options.containsKey("file")) body = Files.readString(Paths.get(parsed.options.get("file")).toAbsolutePath(), StandardCharsets.UTF_8);
            else if (parsed.options.containsKey("data")) body = parsed.options.get("data");

            Map<String, Object> req = new LinkedHashMap<>();
            req.put("method", defaults.get("method"));
            req.put("url", url);
            req.put("headers", defaults.get("headers"));
            req.put("data", body);
            if (parsed.options.containsKey("expect-status")) req.put("expect_status", Integer.parseInt(parsed.options.get("expect-status")));
            if (parsed.options.containsKey("expect-json")) req.put("expect_json", parsed.options.get("expect-json"));

            List<Map<String, Object>> attempts = new ArrayList<>();
            Map<String, Object> result = requestWithRetry(req, defaults, List.of(), attempts);
            printResponse(result);
            Map<String, Object> payload = new LinkedHashMap<>();
            payload.put("generated_at", Instant.now().toString());
            payload.put("request", req);
            payload.put("attempts", attempts);
            Files.writeString(output, GSON.toJson(payload), StandardCharsets.UTF_8);
            System.out.println("Saved log: " + output);
        } catch (Exception ex) {
            System.err.println("Error: " + ex.getMessage());
            System.exit(1);
        }
    }

    private static ParsedArgs parseArgs(String[] args) {
        Map<String, String> options = new LinkedHashMap<>();
        Map<String, List<String>> lists = new LinkedHashMap<>();
        List<String> positional = new ArrayList<>();
        for (int i = 0; i < args.length; i++) {
            String t = args[i];
            if (t.startsWith("--")) {
                String key = t.substring(2);
                String value = "true";
                if (i + 1 < args.length && !args[i + 1].startsWith("--")) value = args[++i];
                if (key.equals("header")) lists.computeIfAbsent("header", k -> new ArrayList<>()).add(value);
                else options.put(key, value);
            } else positional.add(t);
        }
        return new ParsedArgs(options, lists, positional);
    }

    private static Map<String, String> parseHeaders(List<String> raw) {
        Map<String, String> out = new LinkedHashMap<>();
        for (String h : raw) {
            int idx = h.indexOf(':');
            if (idx <= 0) continue;
            out.put(h.substring(0, idx).trim(), h.substring(idx + 1).trim());
        }
        return out;
    }

    private static Set<Integer> parseRetryOn(String raw) {
        Set<Integer> out = new LinkedHashSet<>();
        for (String p : raw.split(",")) {
            p = p.trim();
            if (!p.isBlank()) out.add(Integer.parseInt(p));
        }
        return out;
    }

    private static String replaceRefs(String text, List<Map<String, Object>> context) {
        String out = text;
        while (out.contains("{{step") && out.contains("}}")) {
            int start = out.indexOf("{{step");
            int end = out.indexOf("}}", start);
            if (start < 0 || end < 0) break;
            String token = out.substring(start + 2, end); // stepN.$.x.y
            int dot = token.indexOf('.');
            int idx = Integer.parseInt(token.substring(4, dot));
            String expr = token.substring(dot);
            Object value = "";
            if (idx >= 0 && idx < context.size()) {
                Object json = context.get(idx).get("response_json");
                if (json != null) {
                    try {
                        Object got = JsonPath.read(GSON.toJson(json), expr);
                        if (got != null) value = got.toString();
                    } catch (Exception ignored) {
                    }
                }
            }
            out = out.substring(0, start) + value + out.substring(end + 2);
        }
        return out;
    }

    @SuppressWarnings("unchecked")
    private static Map<String, Object> requestWithRetry(
        Map<String, Object> spec,
        Map<String, Object> defaults,
        List<Map<String, Object>> context,
        List<Map<String, Object>> attempts
    ) throws Exception {
        String method = String.valueOf(spec.getOrDefault("method", defaults.get("method"))).toUpperCase(Locale.ROOT);
        Set<Integer> retryOn = new LinkedHashSet<>((Set<Integer>) defaults.get("retryOn"));
        if (spec.containsKey("retry_on")) {
            retryOn.clear();
            Object arr = spec.get("retry_on");
            if (arr instanceof List<?> list) for (Object x : list) retryOn.add(((Number) x).intValue());
        }
        int retries = ((Number) spec.getOrDefault("retries", defaults.get("retries"))).intValue();
        String backoff = String.valueOf(spec.getOrDefault("backoff", defaults.get("backoff")));
        double retryDelay = ((Number) spec.getOrDefault("retry_delay", defaults.get("retryDelay"))).doubleValue();
        double timeout = ((Number) spec.getOrDefault("timeout", defaults.get("timeout"))).doubleValue();
        Map<String, String> headers = new LinkedHashMap<>((Map<String, String>) defaults.get("headers"));
        if (spec.get("headers") instanceof Map<?, ?> m) {
            for (Map.Entry<?, ?> e : m.entrySet()) headers.put(String.valueOf(e.getKey()), String.valueOf(e.getValue()));
        }
        String url = replaceRefs(String.valueOf(spec.get("url")), context);

        Object dataObj = spec.get("data");
        String body = null;
        if (dataObj != null) {
            body = (dataObj instanceof String s) ? replaceRefs(s, context) : GSON.toJson(dataObj);
        }

        HttpClient client = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds((long) Math.max(1, timeout))).build();
        int attempt = 0;
        while (true) {
            long t0 = System.nanoTime();
            try {
                HttpRequest.Builder b = HttpRequest.newBuilder()
                    .uri(URI.create(url))
                    .timeout(Duration.ofSeconds((long) Math.max(1, timeout)));
                for (Map.Entry<String, String> h : headers.entrySet()) b.header(h.getKey(), h.getValue());
                if (body != null) b.method(method, HttpRequest.BodyPublishers.ofString(body, StandardCharsets.UTF_8));
                else b.method(method, HttpRequest.BodyPublishers.noBody());

                HttpResponse<String> resp = client.send(b.build(), HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
                double ms = (System.nanoTime() - t0) / 1_000_000.0;
                Object parsedJson = null;
                try { parsedJson = JsonParser.parseString(resp.body()); } catch (Exception ignored) {}
                Map<String, Object> entry = new LinkedHashMap<>();
                entry.put("attempt", attempt);
                entry.put("method", method);
                entry.put("url", url);
                entry.put("request_headers", headers);
                entry.put("request_body", body);
                entry.put("status_code", resp.statusCode());
                entry.put("response_headers", resp.headers().map());
                entry.put("response_body", resp.body());
                entry.put("response_json", parsedJson);
                entry.put("timing_ms", Math.round(ms * 100.0) / 100.0);
                attempts.add(entry);

                if (retryOn.contains(resp.statusCode()) && attempt < retries) {
                    attempt++;
                    long delayMs = (long) ((backoff.equals("linear") ? retryDelay * attempt : retryDelay * Math.pow(2, attempt - 1)) * 1000.0);
                    Thread.sleep(delayMs);
                    continue;
                }

                if (spec.containsKey("expect_status")) {
                    int expected = ((Number) spec.get("expect_status")).intValue();
                    if (resp.statusCode() != expected) throw new RuntimeException("Expected status " + expected + ", got " + resp.statusCode());
                }
                if (spec.containsKey("expect_json")) {
                    if (parsedJson == null) throw new RuntimeException("Expected JSON response but body is not JSON");
                    String expr = String.valueOf(spec.get("expect_json"));
                    Object got = JsonPath.read(GSON.toJson(parsedJson), expr);
                    if (got == null || (got instanceof List<?> list && list.isEmpty())) {
                        throw new RuntimeException("JSONPath not found: " + expr);
                    }
                }
                return entry;
            } catch (Exception ex) {
                double ms = (System.nanoTime() - t0) / 1_000_000.0;
                Map<String, Object> fail = new LinkedHashMap<>();
                fail.put("attempt", attempt);
                fail.put("method", method);
                fail.put("url", url);
                fail.put("request_headers", headers);
                fail.put("request_body", body);
                fail.put("status_code", 0);
                fail.put("response_headers", Map.of());
                fail.put("response_body", "");
                fail.put("response_json", null);
                fail.put("timing_ms", Math.round(ms * 100.0) / 100.0);
                fail.put("error", ex.getMessage());
                attempts.add(fail);
                if (attempt >= retries) throw ex;
                attempt++;
                long delayMs = (long) ((backoff.equals("linear") ? retryDelay * attempt : retryDelay * Math.pow(2, attempt - 1)) * 1000.0);
                Thread.sleep(delayMs);
            }
        }
    }

    private static void printResponse(Map<String, Object> entry) {
        System.out.println("Status: " + entry.get("status_code"));
        System.out.println("Headers:");
        @SuppressWarnings("unchecked")
        Map<String, List<String>> h = (Map<String, List<String>>) entry.get("response_headers");
        if (h != null) for (Map.Entry<String, List<String>> e : h.entrySet()) System.out.println("  " + e.getKey() + ": " + String.join(", ", e.getValue()));
        System.out.println("Body:");
        Object j = entry.get("response_json");
        if (j != null) System.out.println(GSON.toJson(j));
        else System.out.println(entry.get("response_body"));
    }

    private static void runChain(List<Map<String, Object>> chain, Map<String, Object> defaults, Path output) throws Exception {
        List<Map<String, Object>> logs = new ArrayList<>();
        List<Map<String, Object>> context = new ArrayList<>();
        for (Map<String, Object> step : chain) {
            List<Map<String, Object>> attempts = new ArrayList<>();
            Map<String, Object> result = requestWithRetry(step, defaults, context, attempts);
            logs.add(Map.of("request", step, "attempts", attempts));
            context.add(result);
            printResponse(result);
        }
        Map<String, Object> payload = new LinkedHashMap<>();
        payload.put("generated_at", Instant.now().toString());
        payload.put("chain", chain);
        payload.put("logs", logs);
        Files.writeString(output, GSON.toJson(payload), StandardCharsets.UTF_8);
        System.out.println("Saved log: " + output);
    }

    private static List<Map<String, Object>> parseChain(JsonElement chain) {
        JsonElement arr = chain;
        if (chain.isJsonObject() && chain.getAsJsonObject().has("requests")) arr = chain.getAsJsonObject().get("requests");
        if (!arr.isJsonArray()) throw new IllegalArgumentException("--chain file must be array or object with requests array");
        List<Map<String, Object>> out = new ArrayList<>();
        for (JsonElement e : arr.getAsJsonArray()) {
            Map<String, Object> m = GSON.fromJson(e, Map.class);
            out.add(m);
        }
        return out;
    }

    private static List<Map<String, Object>> defaultDemoChain() {
        return List.of(
            Map.of("method", "GET", "url", "https://httpbin.org/get", "expect_status", 200, "expect_json", "$.url"),
            Map.of("method", "POST", "url", "https://httpbin.org/post", "data", Map.of("message", "hello"), "expect_status", 200, "expect_json", "$.json.message"),
            Map.of("method", "GET", "url", "https://httpbin.org/status/503", "retries", 2, "retry_on", List.of(503), "expect_status", 503),
            Map.of("method", "GET", "url", "https://httpbin.org/anything/{{step1.$.json.message}}", "expect_status", 200)
        );
    }
}