README.md

---
license: apache-2.0
tags:
  - pytorch
  - keypoint-detection
  - human-pose-estimation
  - heatmap-regression
  - computer-vision
  - detr
  - coco
model-index:
  - name: detr-pose-coco50
    results:
      - task:
          type: pose-estimation
          name: Human Pose Estimation
        dataset:
          type: COCO
          name: COCO 2017 (50-person subset)
        metrics:
          - type: MSELoss
            value: ~0.02
            name: Heatmap MSE
---

# 📌 **DETR + Keypoint Estimation (COCO Subset)**
Author: [@Koushik](https://huggingface.co/Koushim)

---

### 🧠 Model Overview

This project combines:

* 🤖 [facebook/detr-resnet-50](https://huggingface.co/facebook/detr-resnet-50) (object detector)
* 🧱 Custom PyTorch keypoint head
* 📊 Trained on 500-person subset of [COCO 2017 Keypoints](https://cocodataset.org/#keypoints-2020)

The system detects people using DETR, then predicts 17 COCO-style keypoints (top-down) using heatmap regression.

---

### 📂 Files Included

| File                            | Description                                |
| ------------------------------- | ------------------------------------------ |
| `pytorch_model.bin`             | Trained PyTorch model weights              |
| `05_detr_pose_coco_colab.ipynb` | Full Colab notebook (training + inference) |
| `config.json`                   | Basic model metadata                       |
| `README.md`                     | Project description                        |

---

### 📚 Dataset

* **Subset**: 500 images from COCO val2017 with visible persons
* **Annotations**: 17 keypoints per person
* **Source**: [COCO Keypoints](https://cocodataset.org/#keypoints-2020)

---

### 🏗️ Architecture

```text
[ Input Image ]
      │
      ▼
[ DETR (Person BBox) ]
      │
      ▼
[ Crop + Resize (256×256) ]
      │
      ▼
[ CNN Keypoint Head ]
      │
      ▼
[ 17 Heatmaps (Keypoints) ]
```

---

### 🚀 Quick Start

```python
import torch
from model import KeypointHead

model = KeypointHead()
model.load_state_dict(torch.load('pytorch_model.bin'))
model.eval()
```

---

### 🧪 Inference Demo

```python
from PIL import Image
import cv2, numpy as np
from transformers import DetrImageProcessor, DetrForObjectDetection

img = Image.open('sample.jpg')
processor = DetrImageProcessor.from_pretrained("facebook/detr-resnet-50")
detector = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50")

inputs = processor(images=img, return_tensors="pt")
outputs = detector(**inputs)
results = processor.post_process_object_detection(outputs, target_sizes=[img.size[::-1]], threshold=0.8)[0]

# Use results['boxes'][0] to crop person
# Feed crop into model(img) to get 17 heatmaps
```

---

### 🧠 Training (optional)

To fine-tune on your own dataset:

* Convert your data to COCO format
* Use the notebook provided (`05_detr_pose_coco_colab.ipynb`)
* Change paths and re-train

---

### ✨ Credit

* [Hugging Face Transformers](https://github.com/huggingface/transformers)
* [COCO Dataset](https://cocodataset.org/)
* [facebook/detr](https://huggingface.co/facebook/detr-resnet-50)