Spell Checker (python, written by Claude Code)

Claude Code wrote this python project from the task below. It installed and ran on a clean Ubuntu 22.04 machine as written.

Task given to the agent:

TASK: Spell Checker

Write a program that checks spelling in text files using dictionary lookups and edit distance algorithms, supporting custom dictionaries, multiple languages, and suggestion ranking.

FUNCTIONAL REQUIREMENTS:
- Accept a text file path as a command-line argument
- Load a dictionary of valid words (ship a built-in English dictionary of at least 50,000 common words)
- Identify misspelled words by checking each word against the dictionary, ignoring case, numbers, URLs, email addresses, and common abbreviations
- Generate spelling suggestions for each misspelled word using edit distance (Levenshtein distance) with up to 2 edits, ranked by likelihood (frequency-weighted if possible)
- Support custom dictionaries via --dictionary flag (path to a text file with one word per line) that supplements the built-in dictionary
- Support an ignore list via --ignore flag (comma-separated words or path to file) for domain-specific terms that should not be flagged
- Support multiple output modes via --format flag: interactive (show each error with context and suggestions), report (summary with all errors), and json (machine-readable)
- Show each misspelled word with its line number, column number, and surrounding context (the line containing the error with the word highlighted)
- Compute document statistics: total words, unique words, misspelled words count, and spelling accuracy percentage
- Support checking multiple files via --batch flag
- Save the spell check report as JSON with --output flag (default: spelling_report.json) including all misspelled words, their locations, and suggested corrections
- If no input file is given, generate a sample text document with intentional spelling errors of various types (transpositions, missing letters, extra letters, wrong letters), check it, and display the results with suggestions
- Handle errors: unreadable files, binary file detection, encoding issues, and very large files

Create a complete Python project for a clean Ubuntu 22.04 machine with only Python 3.10+ installed. Include:
- Source code
- requirements.txt with all dependencies (direct and transitive) pinned to exact versions
- README.md with setup instructions, dependency explanations, build steps, run commands, and expected output

[]

# Spell Checker (Python - Trial 1)

A spell checking tool that uses dictionary lookups and Levenshtein distance to detect misspellings and provide correction suggestions. Supports custom dictionaries.

## Dependencies

- **pyspellchecker (0.8.1)**: Core spell checking engine using Levenshtein distance for word similarity matching and suggestion generation.
- **textblob (0.18.0)**: NLP library providing additional spelling correction capabilities based on statistical models.

## Setup

```bash
pip install -r requirements.txt
python -m textblob.download_corpora
```

## Usage

### Interactive Mode

```bash
python spellcheck.py
```

### With Custom Dictionary Path

```bash
python spellcheck.py my_dictionary.json
```

### Batch Mode (stdin)

```bash
echo "Ths is a tset sentance" | python spellcheck.py
```

### Interactive Commands

- `check <text>` - Check the spelling of the provided text
- `suggest <word>` - Get spelling suggestions for a word
- `add <word>` - Add a word to the custom dictionary
- `remove <word>` - Remove a word from the custom dictionary
- `list` - List all words in the custom dictionary
- `distance <word1> <word2>` - Calculate the Levenshtein distance between two words
- `quit` - Exit the spell checker

## Features

- Dictionary-based spell checking with Levenshtein distance ranking
- TextBlob-based statistical spelling correction
- Persistent custom dictionary stored as JSON
- Batch processing mode via stdin with JSON output
- Accuracy statistics for checked text

pyspellchecker==0.8.1
textblob==0.18.0

"""
Spell Checker - Checks spelling via dictionary lookups and Levenshtein distance.
Supports custom dictionaries and provides spelling suggestions.

Dependencies:
    - pyspellchecker (0.8.1): Core spell checking with Levenshtein distance
    - textblob (0.18.0): Additional NLP-based spelling correction
"""

import sys
import json
import os
from spellchecker import SpellChecker
from textblob import TextBlob


class CustomDictionary:
    """Manages a custom dictionary that persists to a JSON file."""

    def __init__(self, path=None):
        self.path = path or "custom_dictionary.json"
        self.words = set()
        self._load()

    def _load(self):
        """Load custom words from the JSON file."""
        if os.path.exists(self.path):
            try:
                with open(self.path, "r", encoding="utf-8") as f:
                    data = json.load(f)
                    self.words = set(data.get("words", []))
            except (json.JSONDecodeError, IOError):
                self.words = set()

    def save(self):
        """Save current custom words to the JSON file."""
        with open(self.path, "w", encoding="utf-8") as f:
            json.dump({"words": sorted(list(self.words))}, f, indent=2)

    def add_word(self, word):
        """Add a word to the custom dictionary."""
        self.words.add(word.lower())
        self.save()

    def remove_word(self, word):
        """Remove a word from the custom dictionary."""
        self.words.discard(word.lower())
        self.save()

    def contains(self, word):
        """Check if a word exists in the custom dictionary."""
        return word.lower() in self.words

    def list_words(self):
        """Return all words in the custom dictionary."""
        return sorted(list(self.words))


class SpellCheckEngine:
    """
    Spell checking engine combining pyspellchecker and TextBlob.
    Provides dictionary lookups, Levenshtein distance suggestions,
    and custom dictionary support.
    """

    def __init__(self, language="en", custom_dict_path=None):
        self.spell = SpellChecker(language=language)
        self.custom_dict = CustomDictionary(custom_dict_path)
        # Load custom words into pyspellchecker
        if self.custom_dict.words:
            self.spell.word_frequency.load_words(list(self.custom_dict.words))

    def add_to_dictionary(self, word):
        """Add a word to the custom dictionary and the spell checker."""
        self.custom_dict.add_word(word)
        self.spell.word_frequency.load_words([word.lower()])

    def remove_from_dictionary(self, word):
        """Remove a word from the custom dictionary."""
        self.custom_dict.remove_word(word)

    def check_word(self, word):
        """
        Check if a single word is spelled correctly.

        Returns:
            dict with 'word', 'correct', 'suggestions', and 'correction' keys.
        """
        clean_word = word.strip().lower()
        if not clean_word or not clean_word.isalpha():
            return {
                "word": word,
                "correct": True,
                "suggestions": [],
                "correction": word,
            }

        # Check custom dictionary first
        if self.custom_dict.contains(clean_word):
            return {
                "word": word,
                "correct": True,
                "suggestions": [],
                "correction": word,
            }

        # Check with pyspellchecker
        misspelled = self.spell.unknown([clean_word])
        if not misspelled:
            return {
                "word": word,
                "correct": True,
                "suggestions": [],
                "correction": word,
            }

        # Get suggestions from pyspellchecker (uses Levenshtein distance)
        candidates = self.spell.candidates(clean_word)
        suggestions = sorted(list(candidates)) if candidates else []

        # Get the most likely correction
        correction = self.spell.correction(clean_word)

        # Also try TextBlob correction for comparison
        blob_correction = str(TextBlob(clean_word).correct())
        if blob_correction != clean_word and blob_correction not in suggestions:
            suggestions.insert(0, blob_correction)

        return {
            "word": word,
            "correct": False,
            "suggestions": suggestions[:10],
            "correction": correction if correction else word,
        }

    def check_text(self, text):
        """
        Check all words in a text string.

        Returns:
            dict with 'original_text', 'corrected_text', 'errors', and 'statistics'.
        """
        words = text.split()
        results = []
        error_count = 0
        corrected_words = []

        for word in words:
            # Strip punctuation for checking but preserve it
            stripped = word.strip(".,!?;:\"'()-[]{}").lower()
            if not stripped or not stripped.isalpha():
                corrected_words.append(word)
                continue

            result = self.check_word(stripped)
            if not result["correct"]:
                error_count += 1
                results.append(result)
                # Replace the misspelled word while preserving punctuation
                prefix = ""
                suffix = ""
                for ch in word:
                    if ch.isalpha():
                        break
                    prefix += ch
                for ch in reversed(word):
                    if ch.isalpha():
                        break
                    suffix = ch + suffix
                corrected = prefix + result["correction"] + suffix
                corrected_words.append(corrected)
            else:
                corrected_words.append(word)

        corrected_text = " ".join(corrected_words)

        return {
            "original_text": text,
            "corrected_text": corrected_text,
            "errors": results,
            "statistics": {
                "total_words": len(words),
                "misspelled_words": error_count,
                "accuracy": round(
                    (1 - error_count / max(len(words), 1)) * 100, 2
                ),
            },
        }

    def get_levenshtein_distance(self, word1, word2):
        """
        Calculate the Levenshtein distance between two words.
        Uses dynamic programming approach.
        """
        m, n = len(word1), len(word2)
        dp = [[0] * (n + 1) for _ in range(m + 1)]

        for i in range(m + 1):
            dp[i][0] = i
        for j in range(n + 1):
            dp[0][j] = j

        for i in range(1, m + 1):
            for j in range(1, n + 1):
                cost = 0 if word1[i - 1] == word2[j - 1] else 1
                dp[i][j] = min(
                    dp[i - 1][j] + 1,       # deletion
                    dp[i][j - 1] + 1,       # insertion
                    dp[i - 1][j - 1] + cost  # substitution
                )

        return dp[m][n]

    def suggest_similar(self, word, max_distance=2):
        """
        Find dictionary words within a given Levenshtein distance.

        Args:
            word: The word to find suggestions for.
            max_distance: Maximum Levenshtein distance to consider.

        Returns:
            List of (suggestion, distance) tuples sorted by distance.
        """
        candidates = self.spell.candidates(word.lower())
        if not candidates:
            return []

        scored = []
        for candidate in candidates:
            dist = self.get_levenshtein_distance(word.lower(), candidate)
            if dist <= max_distance:
                scored.append((candidate, dist))

        scored.sort(key=lambda x: x[1])
        return scored


def interactive_mode(engine):
    """Run the spell checker in interactive mode."""
    print("=" * 60)
    print("  Spell Checker - Interactive Mode")
    print("=" * 60)
    print("\nCommands:")
    print("  check <text>       - Check spelling of text")
    print("  suggest <word>     - Get suggestions for a word")
    print("  add <word>         - Add word to custom dictionary")
    print("  remove <word>      - Remove word from custom dictionary")
    print("  list               - List custom dictionary words")
    print("  distance <w1> <w2> - Calculate Levenshtein distance")
    print("  quit               - Exit the spell checker")
    print("-" * 60)

    while True:
        try:
            user_input = input("\n> ").strip()
        except (EOFError, KeyboardInterrupt):
            print("\nGoodbye!")
            break

        if not user_input:
            continue

        parts = user_input.split(maxsplit=1)
        command = parts[0].lower()

        if command == "quit" or command == "exit":
            print("Goodbye!")
            break

        elif command == "check":
            if len(parts) < 2:
                print("Usage: check <text>")
                continue
            result = engine.check_text(parts[1])
            print(f"\nOriginal:  {result['original_text']}")
            print(f"Corrected: {result['corrected_text']}")
            stats = result["statistics"]
            print(
                f"Statistics: {stats['total_words']} words, "
                f"{stats['misspelled_words']} errors, "
                f"{stats['accuracy']}% accuracy"
            )
            if result["errors"]:
                print("\nMisspelled words:")
                for err in result["errors"]:
                    suggestions = ", ".join(err["suggestions"][:5])
                    print(f"  '{err['word']}' -> suggestions: [{suggestions}]")

        elif command == "suggest":
            if len(parts) < 2:
                print("Usage: suggest <word>")
                continue
            word = parts[1].strip()
            suggestions = engine.suggest_similar(word)
            if suggestions:
                print(f"Suggestions for '{word}':")
                for s, d in suggestions:
                    print(f"  {s} (distance: {d})")
            else:
                print(f"No suggestions found for '{word}'.")

        elif command == "add":
            if len(parts) < 2:
                print("Usage: add <word>")
                continue
            word = parts[1].strip()
            engine.add_to_dictionary(word)
            print(f"Added '{word}' to custom dictionary.")

        elif command == "remove":
            if len(parts) < 2:
                print("Usage: remove <word>")
                continue
            word = parts[1].strip()
            engine.remove_from_dictionary(word)
            print(f"Removed '{word}' from custom dictionary.")

        elif command == "list":
            words = engine.custom_dict.list_words()
            if words:
                print(f"Custom dictionary ({len(words)} words):")
                for w in words:
                    print(f"  {w}")
            else:
                print("Custom dictionary is empty.")

        elif command == "distance":
            if len(parts) < 2:
                print("Usage: distance <word1> <word2>")
                continue
            dist_parts = parts[1].split()
            if len(dist_parts) < 2:
                print("Usage: distance <word1> <word2>")
                continue
            d = engine.get_levenshtein_distance(dist_parts[0], dist_parts[1])
            print(
                f"Levenshtein distance between "
                f"'{dist_parts[0]}' and '{dist_parts[1]}': {d}"
            )

        else:
            print(f"Unknown command: '{command}'. Type a command or 'quit'.")


def main():
    """Main entry point."""
    custom_path = None
    if len(sys.argv) > 1:
        custom_path = sys.argv[1]

    engine = SpellCheckEngine(custom_dict_path=custom_path)

    if sys.stdin.isatty():
        interactive_mode(engine)
    else:
        # Process stdin in batch mode
        text = sys.stdin.read().strip()
        if text:
            result = engine.check_text(text)
            print(json.dumps(result, indent=2))


if __name__ == "__main__":
    main()