Hugging Face's logo

qualcomm
/
Segment-Anything-Model-2

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community

Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.45, ONNX Runtime 1.25.0 Download
ONNX w8a8 Universal QAIRT 2.45, ONNX Runtime 1.25.0 Download
QNN_DLC float Universal QAIRT 2.45 Download
QNN_DLC w8a8 Universal QAIRT 2.45 Download
TFLITE w8a8 Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Model checkpoint: sam2.1_hiera_t
  • Input resolution: 720p (720x1280)
  • Number of parameters (encoder): 33.5M
  • Model size (encoder) (float): 128 MB
  • Number of parameters (decoder): 6.22M
  • Model size (decoder) (float): 23.7 MB

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 3.301 ms 16 - 16 MB NPU
decoder ONNX float Snapdragon® 8 Gen 3 Mobile 4.756 ms 2 - 243 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 6.758 ms 16 - 20 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.182 ms 1 - 221 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 3.748 ms 1 - 223 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ Q-8750 3.748 ms 1 - 223 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ IQ-9075 8.242 ms 16 - 61 MB NPU
decoder ONNX w8a8 Snapdragon® X2 Elite 1.331 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® X Elite 2.737 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 1.777 ms 4 - 163 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 3.507 ms 4 - 165 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 12.713 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 2.614 ms 2 - 8 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8450 3.507 ms 4 - 165 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Mobile 1.428 ms 0 - 130 MB NPU
decoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 3.372 ms 4 - 301 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 1.248 ms 1 - 157 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 3.186 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 15.968 ms 2 - 254 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 3.372 ms 4 - 301 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 1.428 ms 0 - 130 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.737 ms 4 - 4 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 3.092 ms 16 - 16 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 3.714 ms 4 - 216 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 10.004 ms 4 - 217 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 5.378 ms 16 - 20 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 10.004 ms 4 - 217 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.594 ms 4 - 192 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 2.94 ms 1 - 189 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 8.214 ms 0 - 195 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 2.94 ms 1 - 189 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 6.682 ms 16 - 34 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.315 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X Elite 2.235 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.292 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 2.595 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 9.851 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 1.954 ms 4 - 20 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8775P 2.639 ms 4 - 92 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8650P 2.639 ms 4 - 92 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8255P 2.639 ms 4 - 92 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8450 2.595 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 1.018 ms 1 - 89 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.68 ms 4 - 212 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.901 ms 1 - 109 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 2.472 ms 1 - 6 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 9.938 ms 4 - 275 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8295P 2.958 ms 4 - 88 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 2.68 ms 4 - 212 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 1.018 ms 1 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.235 ms 4 - 4 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.72 ms 0 - 213 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 1 Mobile 9.989 ms 0 - 219 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 5.328 ms 0 - 8 MB NPU
decoder TFLITE float Qualcomm® SA8775P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® SA8650P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® SA8255P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® QCS8450 9.989 ms 0 - 219 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.637 ms 0 - 184 MB NPU
decoder TFLITE float Qualcomm® SA8295P 8.184 ms 0 - 196 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 6.904 ms 0 - 30 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 5.786 ms 0 - 265 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 10.177 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 39.507 ms 12 - 58 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8275 67.255 ms 17 - 52 MB GPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 7.73 ms 0 - 28 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8775P 8.555 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8650P 8.555 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8255P 8.555 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8450 10.177 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 3.514 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 14.337 ms 12 - 201 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.121 ms 0 - 255 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 8.262 ms 0 - 14 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 32.784 ms 13 - 210 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8295P 9.455 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Qualcomm® SA7255P 67.255 ms 17 - 52 MB GPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 14.337 ms 12 - 201 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 3.514 ms 0 - 240 MB NPU
encoder ONNX float Snapdragon® X2 Elite 86.851 ms 70 - 70 MB NPU
encoder ONNX float Snapdragon® 8 Gen 3 Mobile 126.384 ms 58 - 2157 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 168.886 ms 8 - 95 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 78.771 ms 45 - 1714 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 97.703 ms 44 - 1798 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ Q-8750 97.703 ms 44 - 1798 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ IQ-9075 226.812 ms 12 - 69 MB NPU
encoder ONNX w8a8 Snapdragon® X2 Elite 35.618 ms 18 - 18 MB NPU
encoder ONNX w8a8 Snapdragon® X Elite 91.477 ms 52 - 52 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 63.775 ms 16 - 1945 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 124.433 ms 0 - 2391 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 676.253 ms 6 - 12 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 87.71 ms 0 - 57 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8450 124.433 ms 0 - 2391 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Mobile 39.644 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 88.7 ms 14 - 2358 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.125 ms 12 - 1680 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 74.824 ms 11 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 1257.361 ms 3 - 1934 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 88.7 ms 14 - 2358 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 39.644 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 91.477 ms 52 - 52 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 85.915 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 128.059 ms 12 - 2146 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 341.96 ms 13 - 1861 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 179.716 ms 9 - 16 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 341.96 ms 13 - 1861 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 78.373 ms 0 - 1706 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 100.862 ms 11 - 2522 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 282.738 ms 1 - 1582 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 100.862 ms 11 - 2522 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 218.985 ms 12 - 46 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X2 Elite 36.559 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X Elite 83.578 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 54.844 ms 3 - 1883 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 136.835 ms 3 - 1897 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 678.591 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 80.452 ms 3 - 6 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8775P 81.912 ms 3 - 1580 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8650P 81.912 ms 3 - 1580 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8255P 81.912 ms 3 - 1580 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8450 136.835 ms 3 - 1897 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 40.06 ms 3 - 1633 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 95.72 ms 3 - 2356 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 35.363 ms 3 - 1663 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 106.438 ms 1 - 10 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 1253.567 ms 3 - 1919 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8295P 95.717 ms 0 - 1591 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 95.72 ms 3 - 2356 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 40.06 ms 3 - 1633 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 83.578 ms 3 - 3 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 129.64 ms 20 - 2227 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 347.74 ms 21 - 1955 MB NPU
encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 180.195 ms 20 - 23 MB NPU
encoder TFLITE float Qualcomm® SA8775P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® SA8650P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® SA8255P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® QCS8450 347.74 ms 21 - 1955 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 76.287 ms 20 - 1777 MB NPU
encoder TFLITE float Qualcomm® SA8295P 282.438 ms 20 - 1665 MB NPU
encoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 218.941 ms 0 - 114 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 70.211 ms 5 - 2224 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 183.213 ms 5 - 2292 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 1720.565 ms 76 - 339 MB CPU
encoder TFLITE w8a8 Qualcomm® QCS8275 162.746 ms 6 - 1878 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 101.03 ms 5 - 10 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8775P 97.994 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8650P 97.994 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8255P 97.994 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS8450 183.213 ms 5 - 2292 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 56.455 ms 4 - 1791 MB NPU
encoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 1123.658 ms 28 - 256 MB CPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.4 ms 5 - 1795 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 96.155 ms 3 - 69 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 2367.706 ms 78 - 312 MB CPU
encoder TFLITE w8a8 Qualcomm® SA8295P 118.048 ms 5 - 1882 MB NPU
encoder TFLITE w8a8 Qualcomm® SA7255P 162.746 ms 6 - 1878 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 1123.658 ms 28 - 256 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 56.455 ms 4 - 1791 MB NPU

License

  • The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image Segmentation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/Segment-Anything-Model-2