---
language:
  - zh
  - en
tags:
  - translation
  - ocr
  - screen-capture
  - chinese
  - easyocr
  - marianmt
license: mit
library_name: transformers
pipeline_tag: translation
model-index:
  - name: ChineseScreenTranslator
    results:
      - task:
          type: translation
          name: Chinese to English Screen OCR Translation
        metrics:
          - type: bleu
            value: N/A
            name: BLEU
---

# ChineseScreenTranslator

[![Python](https://img.shields.io/badge/Python-3.9%2B-3776AB?style=flat-square&logo=python&logoColor=white)](https://python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square)](LICENSE)
[![EasyOCR](https://img.shields.io/badge/OCR-EasyOCR-orange?style=flat-square)](https://github.com/JaidedAI/EasyOCR)
[![HuggingFace](https://img.shields.io/badge/HuggingFace-Helsinki--NLP-FFD21E?style=flat-square&logo=huggingface&logoColor=black)](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en)
[![Google Translate](https://img.shields.io/badge/Backend-Google%20Translate-4285F4?style=flat-square&logo=googletranslate&logoColor=white)](https://translate.google.com)
[![Platform](https://img.shields.io/badge/Platform-Windows%20%7C%20Linux%20%7C%20macOS-lightgrey?style=flat-square)](https://github.com)
[![Author](https://img.shields.io/badge/Author-algorembrant-e94560?style=flat-square)](https://github.com/algorembrant)

A screen region selection tool that automatically detects and translates Chinese text (Simplified, Traditional, Cantonese, Classical) to English in real time, powered by EasyOCR and neural machine translation.

---

## Features

| Feature | Details |
|---|---|
| Screen Selection | Click-and-drag rubber-band region selector |
| OCR Engine | EasyOCR - Simplified + Traditional Chinese |
| Translation | Google Translate, Microsoft, or offline Helsinki-NLP |
| Chinese Variants | Simplified, Traditional, Cantonese, Classical, Mixed |
| Hotkey Trigger | Global configurable hotkey (default: Ctrl+Shift+S) |
| Image Preprocessing | Upscale, contrast enhance, sharpen for accuracy |
| History | Persistent JSON history with search and export |
| Clipboard | Auto-copy or one-click copy |
| Offline Mode | Helsinki-NLP MarianMT (opus-mt-zh-en) |
| GPU Support | CUDA acceleration for OCR and offline translation |
| Settings UI | Built-in settings window |

---

## Screenshots

```
+--------------------------------------+
|  ChineseScreenTranslator  v1.0.0     |
|  Detected Chinese Text               |
|  +----------------------------------+|
|  | 你好，欢迎使用本翻译工具。         ||
|  +----------------------------------+|
|  English Translation                 |
|  +----------------------------------+|
|  | Hello, welcome to this           ||
|  | translation tool.                ||
|  +----------------------------------+|
|  [Copy Translation] [Copy Source]    |
+--------------------------------------+
```

---

## Requirements

### System Requirements

| Component | Minimum | Recommended |
|---|---|---|
| Python | 3.9 | 3.11+ |
| RAM | 4 GB | 8 GB+ |
| Disk Space | 1 GB | 2 GB (models + deps) |
| GPU (optional) | CUDA 11.8 | CUDA 12.1 |
| OS | Windows 10 / Ubuntu 20.04 / macOS 12 | Windows 11 / Ubuntu 22.04 |

### First Run Downloads

| Component | Size | Purpose |
|---|---|---|
| EasyOCR detection model | ~50 MB | Text region detection |
| EasyOCR ch_sim model | ~90 MB | Simplified Chinese recognition |
| EasyOCR ch_tra model | ~90 MB | Traditional Chinese recognition |
| Helsinki-NLP opus-mt-zh-en | ~300 MB | Offline translation (optional) |

---

## Installation

### Step 1 - Clone the repository

```bash
git clone https://github.com/algorembrant/ChineseScreenTranslator.git
cd ChineseScreenTranslator
```

### Step 2 - Create a virtual environment (recommended)

```bash
python -m venv venv

# Windows
venv\Scripts\activate

# Linux / macOS
source venv/bin/activate
```

### Step 3 - Install PyTorch

Choose the command that matches your hardware:

| Hardware | Command |
|---|---|
| CPU only (any OS) | `pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu` |
| NVIDIA GPU CUDA 11.8 | `pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118` |
| NVIDIA GPU CUDA 12.1 | `pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121` |
| macOS (Apple Silicon) | `pip install torch torchvision` |

### Step 4 - Install remaining dependencies

```bash
pip install -r requirements.txt
```

### Step 5 - Run the application

```bash
python translator.py
```

The first launch downloads OCR models (~230 MB total). This only happens once.

---

## Usage

### Basic Usage

| Action | How |
|---|---|
| Start application | `python translator.py` |
| Trigger screen capture | Press `Ctrl+Shift+S` |
| Select region | Click and drag on screen |
| View translation | Result window appears automatically |
| Copy translation | Click "Copy Translation" button |
| View history | Press `Ctrl+Shift+H` |
| Quit | Press `Ctrl+Shift+Q` or close window |

### Keyboard Shortcuts

| Shortcut | Action |
|---|---|
| `Ctrl+Shift+S` | Open screen region selector |
| `Ctrl+Shift+C` | Copy last translation to clipboard |
| `Ctrl+Shift+H` | Show/hide translation history |
| `Ctrl+Shift+Q` | Quit application |
| `Escape` | Cancel selection overlay |

### Command Line Options

| Option | Example | Description |
|---|---|---|
| `--hotkey` | `--hotkey "ctrl+alt+x"` | Set custom global hotkey |
| `--lang` | `--lang traditional` | Force OCR language mode |
| `--backend` | `--backend offline` | Set translation backend |
| `--offline` | `--offline` | Use offline MarianMT model |
| `--gpu` | `--gpu` | Enable CUDA GPU acceleration |
| `--upscale` | `--upscale 3` | Upscale factor for small text |
| `--confidence` | `--confidence 0.5` | OCR confidence threshold |
| `--no-preprocess` | `--no-preprocess` | Disable image preprocessing |
| `--export-history` | `--export-history out.json` | Export history on exit |
| `--verbose` | `--verbose` | Enable DEBUG logging |
| `--version` | `--version` | Show version and exit |
| `--help` | `--help` | Show full help |

### Translation Backends

| Backend | Speed | Requires | Quality |
|---|---|---|---|
| `google` (default) | Fast | Internet | Excellent |
| `microsoft` | Fast | Internet + API key | Excellent |
| `offline` | Moderate | ~300 MB model | Good |

For Microsoft Translator, add your Azure Cognitive Services key to `~/.chinese_screen_translator/config.json`:

```json
{
  "microsoft_api_key": "YOUR_KEY_HERE",
  "microsoft_region": "eastus"
}
```

### Supported Chinese Variants

| Variant | Mode | Notes |
|---|---|---|
| Simplified Chinese (普通话 / 简体) | `--lang simplified` | Mainland China standard |
| Traditional Chinese (繁體中文) | `--lang traditional` | Taiwan, Hong Kong |
| Cantonese / Yue (廣東話) | `--lang traditional` | Use traditional mode |
| Classical Chinese (文言文) | `--lang auto` | Auto-detected |
| Mixed Chinese-English | `--lang auto` | Handled by both models |
| Vertical Chinese text | `--lang auto` | EasyOCR handles automatically |

---

## Configuration

Config file location: `~/.chinese_screen_translator/config.json`

```json
{
  "hotkey": "ctrl+shift+s",
  "lang": "auto",
  "backend": "google",
  "use_gpu": false,
  "upscale_factor": 2,
  "confidence_threshold": 0.30,
  "preprocess": true,
  "auto_copy": false,
  "show_confidence": true,
  "max_history": 500,
  "microsoft_api_key": "",
  "microsoft_region": "eastus",
  "offline_model_dir": "~/.chinese_screen_translator/models"
}
```

---

## Troubleshooting

| Problem | Solution |
|---|---|
| `easyocr` not found | `pip install easyocr` then install PyTorch |
| Models not downloading | Check internet connection; proxy may block HuggingFace CDN |
| Hotkeys not working on Linux | Run as root: `sudo python translator.py` |
| Low OCR accuracy | Increase `--upscale 3`, zoom in on text, improve lighting |
| `keyboard` error on macOS | Grant Accessibility permissions in System Preferences |
| Out of memory | Use CPU mode (no `--gpu`), reduce `--upscale` |
| No text detected | Lower `--confidence 0.2`, ensure text is clear and large enough |
| Google Translate blocked | Use `--backend offline` or set up Microsoft API key |

---

## Project Structure

```
ChineseScreenTranslator/
  translator.py          Main application (single file)
  requirements.txt       Python dependencies
  README.md              This file
  .gitignore             Git ignore rules
  .gitattributes         Git / HuggingFace LFS config
  LICENSE                MIT License
```

---

## Architecture

```
Hotkey / Button
     |
     v
ScreenOverlay (tkinter fullscreen transparent)
     |  click + drag
     v
_capture_screenshot (mss / PIL)
     |
     v
ImagePreprocessor
  - Upscale (Lanczos)
  - Contrast enhance
  - Sharpen
     |
     v
OCREngine (EasyOCR)
  - ch_sim reader
  - ch_tra reader
  - Auto-detect or forced mode
     |
     v
TranslationEngine
  - Google Translate (deep-translator)
  - Microsoft Translator
  - Helsinki-NLP MarianMT (offline)
     |
     v
ResultWindow (tkinter)
HistoryManager (JSON persistence)
```

---

## License

MIT License. See `LICENSE` for details.

---

## Author

**algorembrant**

---

## Acknowledgments

- [EasyOCR](https://github.com/JaidedAI/EasyOCR) - OCR engine by JaidedAI
- [Helsinki-NLP](https://huggingface.co/Helsinki-NLP) - MarianMT translation models
- [deep-translator](https://github.com/nidhaloff/deep-translator) - Translation backends
- [mss](https://github.com/BoboTiG/python-mss) - Fast screen capture