Instructions to use microsoft/OmniParser with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/OmniParser with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="microsoft/OmniParser")# Load model directly from transformers import AutoProcessor, AutoModelForVisualQuestionAnswering processor = AutoProcessor.from_pretrained("microsoft/OmniParser") model = AutoModelForVisualQuestionAnswering.from_pretrained("microsoft/OmniParser") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use microsoft/OmniParser with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/OmniParser" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/microsoft/OmniParser
- SGLang
How to use microsoft/OmniParser with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/OmniParser" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/OmniParser" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/OmniParser", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use microsoft/OmniParser with Docker Model Runner:
docker model run hf.co/microsoft/OmniParser
add AIBOM
#50 opened 11 months ago
by
sabato-nocera
Dataset
➕ 3
1
#48 opened about 1 year ago
by
mrfakename
Not able to run the application
1
#47 opened about 1 year ago
by
Karthik-Pamidimarri
Create weights
#46 opened about 1 year ago
by
Scott240866
Identifying icon issues
#45 opened over 1 year ago
by
LiMengY
Quantize for speedup?
1
#44 opened over 1 year ago
by
scottsus
Inputs format on AWS Sagemaker
➕ 1
1
#43 opened over 1 year ago
by
yuleisheng
model usage
#42 opened over 1 year ago
by
wilsonny
Is there a way to fine tune it
➕ 16
#39 opened over 1 year ago
by
tiwariParth
Its not working?
➕ 1
1
#37 opened over 1 year ago
by
usMushi
Coordinates
😔 1
1
#36 opened over 1 year ago
by
krn11
action matching
#35 opened over 1 year ago
by
Reverendo
Request: DOI
#34 opened over 1 year ago
by
Bhaskarsai4
Rename README.md to Kimberly Jocson
6
#33 opened over 1 year ago
by
Chunl3ii
Finetuning detection model
2
#31 opened over 1 year ago
by
Vishwanathan00786
Missing configuration files for transformer support
👍 1
2
#30 opened over 1 year ago
by
berkantay
Upload 14 files
#29 opened over 1 year ago
by
amberfelts890
Update README.md
#27 opened over 1 year ago
by
Remy001
Upload Read Me.pdf
#26 opened over 1 year ago
by
RishuRs
Rename config.json to preprocessor_config.json
➕ 7
8
#25 opened over 1 year ago
by
neerajjulka
PowerApps Implementation Services
#21 opened over 1 year ago
by
johnmax2592
🚩 Report
3
#20 opened over 1 year ago
by
LeonBdd
Delete icon_caption_blip2
1
#18 opened over 1 year ago
by
Rajju123
Update README.md
2
#17 opened over 1 year ago
by
Khalilhau
Update README.md
#16 opened over 1 year ago
by
FinedYT
Will OmniParser be available in Azure ML ?
👀 1
2
#15 opened over 1 year ago
by
ducbernhardttien
Issues with running omniparser via hugging face.
3
#14 opened over 1 year ago
by
VishwaSriram
How can we tell Object is Icon? TextBox, etc?
1
#12 opened over 1 year ago
by
Verfinux
Is this a FAKE MODEL !
1
#11 opened over 1 year ago
by
LeroyDyer
laptop question
1
#10 opened over 1 year ago
by
ryytro
Detecting Handwritten text
1
#6 opened over 1 year ago
by
Utkarsh736
language
3
#5 opened over 1 year ago
by
fxxkTencent
icon_detect model removed
2
#4 opened over 1 year ago
by
korbih
No examples in readme
👀🔥 12
8
#3 opened over 1 year ago
by
jadbox
Create LICENSE
2
#1 opened over 1 year ago
by
jacklangerman