Linear Regression Fitter (python, written by Claude Code)

Claude Code wrote this python project from the task below. It installed and ran on a clean Ubuntu 22.04 machine as written.

Task given to the agent:

TASK: Linear Regression Fitter

Write a program that fits linear regression models to datasets, supporting simple and multiple regression, model evaluation metrics, residual analysis, and prediction on new data.

FUNCTIONAL REQUIREMENTS:
- Accept a CSV data file as a command-line argument with the target variable specified via --target flag
- Support simple linear regression (one predictor) and multiple linear regression (multiple predictors) selected via --features flag (comma-separated column names; default: all non-target columns)
- Compute regression coefficients (intercept and slopes) using the ordinary least squares (OLS) method via the normal equation or gradient descent (selectable via --method flag)
- Report model evaluation metrics: R-squared, adjusted R-squared, mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE), and F-statistic
- Report coefficient details: estimated value, standard error, t-statistic, and p-value for each coefficient
- Perform residual analysis: compute residuals, check for normality (Shapiro-Wilk or similar test), and detect heteroscedasticity
- Support making predictions on new data via --predict flag (path to a CSV file with predictor values)
- Support data normalization/standardization via --normalize flag
- Support train/test split via --split flag (percentage for training, e.g., --split 80 for 80% train)
- Print a comprehensive model summary to console similar to statistical software output
- Save model coefficients and metrics as JSON with --output flag (default: regression_model.json)
- If no input is given, generate a sample dataset with 200 points containing a known linear relationship with noise, fit the model, display coefficients, metrics, residual analysis, and make predictions on 5 new data points
- Handle errors: multicollinearity detection (warn if condition number is high), missing values in data, non-numeric columns, and singular design matrix

Create a complete Python project for a clean Ubuntu 22.04 machine with only Python 3.10+ installed. Include:
- Source code
- requirements.txt with all dependencies (direct and transitive) pinned to exact versions
- README.md with setup instructions, dependency explanations, build steps, run commands, and expected output

[]

# Linear Regression Fitter (Python - Trial 1)

OLS linear regression fitter using scikit-learn with comprehensive metrics, residual analysis, and prediction.

## Dependencies

- scikit-learn 1.4.0
- pandas 2.2.0
- matplotlib 3.8.2

## Setup

```bash
pip install -r requirements.txt
```

## Usage

Run with synthetic data:
```bash
python regression.py
```

Run with a CSV file:
```bash
python regression.py data.csv target_column
```

If no target column is specified, the last column is used as the target.

## Features

- OLS linear regression fitting via scikit-learn
- Metrics: R-squared, MSE, RMSE
- Residual diagnostic plots (residuals vs fitted, histogram, Q-Q plot, actual vs predicted)
- Train/test split evaluation
- Prediction on new data points

"""
Linear Regression Fitter using scikit-learn.

Fits OLS linear regression models with comprehensive metrics
(R-squared, MSE, RMSE), residual analysis, and prediction capabilities.
"""

import sys
import numpy as np
import pandas as pd
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split


def load_data(filepath: str) -> pd.DataFrame:
    """Load a CSV dataset into a pandas DataFrame."""
    try:
        df = pd.read_csv(filepath)
        print(f"Loaded dataset with {df.shape[0]} rows and {df.shape[1]} columns.")
        return df
    except FileNotFoundError:
        print(f"Error: File '{filepath}' not found.")
        sys.exit(1)
    except Exception as e:
        print(f"Error loading data: {e}")
        sys.exit(1)


def fit_regression(X_train: np.ndarray, y_train: np.ndarray) -> LinearRegression:
    """Fit an OLS linear regression model."""
    model = LinearRegression()
    model.fit(X_train, y_train)
    return model


def compute_metrics(y_true: np.ndarray, y_pred: np.ndarray) -> dict:
    """Compute regression evaluation metrics."""
    r2 = r2_score(y_true, y_pred)
    mse = mean_squared_error(y_true, y_pred)
    rmse = np.sqrt(mse)
    return {"r_squared": r2, "mse": mse, "rmse": rmse}


def print_metrics(metrics: dict, label: str = "Test") -> None:
    """Display metrics in a formatted table."""
    print(f"\n{'=' * 45}")
    print(f"  {label} Set Metrics")
    print(f"{'=' * 45}")
    print(f"  R-squared : {metrics['r_squared']:.6f}")
    print(f"  MSE       : {metrics['mse']:.6f}")
    print(f"  RMSE      : {metrics['rmse']:.6f}")
    print(f"{'=' * 45}")


def print_coefficients(model: LinearRegression, feature_names: list) -> None:
    """Display model coefficients."""
    print(f"\n{'=' * 45}")
    print("  Model Coefficients")
    print(f"{'=' * 45}")
    print(f"  Intercept : {model.intercept_:.6f}")
    for name, coef in zip(feature_names, model.coef_):
        print(f"  {name:12s}: {coef:.6f}")
    print(f"{'=' * 45}")


def residual_analysis(y_true: np.ndarray, y_pred: np.ndarray, output_path: str = "residuals.png") -> dict:
    """Perform residual analysis and generate diagnostic plots."""
    residuals = y_true - y_pred

    stats = {
        "mean": float(np.mean(residuals)),
        "std": float(np.std(residuals)),
        "min": float(np.min(residuals)),
        "max": float(np.max(residuals)),
        "median": float(np.median(residuals)),
    }

    print(f"\n{'=' * 45}")
    print("  Residual Analysis")
    print(f"{'=' * 45}")
    print(f"  Mean     : {stats['mean']:.6f}")
    print(f"  Std Dev  : {stats['std']:.6f}")
    print(f"  Min      : {stats['min']:.6f}")
    print(f"  Max      : {stats['max']:.6f}")
    print(f"  Median   : {stats['median']:.6f}")
    print(f"{'=' * 45}")

    fig, axes = plt.subplots(2, 2, figsize=(12, 10))
    fig.suptitle("Residual Diagnostic Plots", fontsize=14, fontweight="bold")

    # Residuals vs Fitted
    axes[0, 0].scatter(y_pred, residuals, alpha=0.6, edgecolors="k", linewidth=0.5)
    axes[0, 0].axhline(y=0, color="r", linestyle="--", linewidth=1)
    axes[0, 0].set_xlabel("Fitted Values")
    axes[0, 0].set_ylabel("Residuals")
    axes[0, 0].set_title("Residuals vs Fitted")

    # Histogram of residuals
    axes[0, 1].hist(residuals, bins=30, edgecolor="black", alpha=0.7, color="steelblue")
    axes[0, 1].set_xlabel("Residuals")
    axes[0, 1].set_ylabel("Frequency")
    axes[0, 1].set_title("Residual Distribution")

    # Q-Q plot
    from scipy import stats as sp_stats
    sorted_residuals = np.sort(residuals)
    n = len(sorted_residuals)
    theoretical_quantiles = sp_stats.norm.ppf(np.arange(1, n + 1) / (n + 1))
    axes[1, 0].scatter(theoretical_quantiles, sorted_residuals, alpha=0.6, edgecolors="k", linewidth=0.5)
    min_val = min(theoretical_quantiles.min(), sorted_residuals.min())
    max_val = max(theoretical_quantiles.max(), sorted_residuals.max())
    axes[1, 0].plot([min_val, max_val], [min_val, max_val], "r--", linewidth=1)
    axes[1, 0].set_xlabel("Theoretical Quantiles")
    axes[1, 0].set_ylabel("Sample Quantiles")
    axes[1, 0].set_title("Q-Q Plot")

    # Actual vs Predicted
    axes[1, 1].scatter(y_true, y_pred, alpha=0.6, edgecolors="k", linewidth=0.5)
    min_val = min(y_true.min(), y_pred.min())
    max_val = max(y_true.max(), y_pred.max())
    axes[1, 1].plot([min_val, max_val], [min_val, max_val], "r--", linewidth=1)
    axes[1, 1].set_xlabel("Actual Values")
    axes[1, 1].set_ylabel("Predicted Values")
    axes[1, 1].set_title("Actual vs Predicted")

    plt.tight_layout()
    plt.savefig(output_path, dpi=150)
    plt.close()
    print(f"\nResidual plots saved to '{output_path}'.")

    return stats


def predict(model: LinearRegression, X_new: np.ndarray) -> np.ndarray:
    """Generate predictions for new data points."""
    return model.predict(X_new)


def generate_sample_data(n_samples: int = 200, n_features: int = 3, noise: float = 10.0, seed: int = 42) -> pd.DataFrame:
    """Generate a synthetic dataset for demonstration."""
    rng = np.random.RandomState(seed)
    X = rng.randn(n_samples, n_features) * 10
    true_coefs = rng.randn(n_features) * 5
    y = X @ true_coefs + 15.0 + rng.randn(n_samples) * noise
    columns = [f"feature_{i+1}" for i in range(n_features)]
    df = pd.DataFrame(X, columns=columns)
    df["target"] = y
    return df


def main():
    """Main entry point demonstrating the full regression pipeline."""
    print("=" * 50)
    print("  Linear Regression Fitter (scikit-learn)")
    print("=" * 50)

    # Generate or load data
    if len(sys.argv) > 1:
        filepath = sys.argv[1]
        target_col = sys.argv[2] if len(sys.argv) > 2 else None
        df = load_data(filepath)
        if target_col is None:
            target_col = df.columns[-1]
            print(f"Using last column '{target_col}' as target.")
        feature_cols = [c for c in df.columns if c != target_col]
    else:
        print("\nNo CSV file provided. Using synthetic data.\n")
        df = generate_sample_data()
        target_col = "target"
        feature_cols = [c for c in df.columns if c != target_col]

    X = df[feature_cols].values
    y = df[target_col].values

    # Train-test split
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
    print(f"Training samples: {X_train.shape[0]}")
    print(f"Test samples    : {X_test.shape[0]}")

    # Fit model
    model = fit_regression(X_train, y_train)
    print_coefficients(model, feature_cols)

    # Training metrics
    y_train_pred = predict(model, X_train)
    train_metrics = compute_metrics(y_train, y_train_pred)
    print_metrics(train_metrics, label="Training")

    # Test metrics
    y_test_pred = predict(model, X_test)
    test_metrics = compute_metrics(y_test, y_test_pred)
    print_metrics(test_metrics, label="Test")

    # Residual analysis
    residual_analysis(y_test, y_test_pred, output_path="residuals.png")

    # Prediction on new data
    print(f"\n{'=' * 45}")
    print("  Prediction Example")
    print(f"{'=' * 45}")
    sample = X_test[:5]
    predictions = predict(model, sample)
    for i, (actual, predicted) in enumerate(zip(y_test[:5], predictions)):
        print(f"  Sample {i+1}: actual={actual:.4f}, predicted={predicted:.4f}")
    print(f"{'=' * 45}")


if __name__ == "__main__":
    main()

scikit-learn==1.4.0
pandas==2.2.0
matplotlib==3.8.2
numpy>=1.26.0
scipy>=1.12.0