Additional Jetson Orin Nano Super benchmarks will be added continuously.
Jonna Matthiesen
JonnaMat
AI & ML interests
None yet
Recent Activity
replied to their post about 7 hours ago
Qwen3.5 on-device benchmarks on the Nvidia Jetson lineup are now live 🚀
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case. posted an
update
about 7 hours ago
Qwen3.5 on-device benchmarks on the Nvidia Jetson lineup are now live 🚀
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case. updated
a Space about 12 hours ago
embedl/Edge-Inference-Benchmarks Organizations
replied to their post about 7 hours ago
posted an
update about 7 hours ago
Post
52
Qwen3.5 on-device benchmarks on the Nvidia Jetson lineup are now live 🚀
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case.
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case.
-
- 1 reply
posted an
update 7 days ago
Post
1590
🤯 Edge-Grade Vision Reasoning. Now Practically Lossless. 🤯
Introducing
👉 embedl/Cosmos-Reason2-2B-W4A16-Edge2
Optimized for Jetson Orin Nano Super and AGX Orin
nvidia .
🚄 Try it out on Jetson (image+video+text):
🤓 What is Edge2? Most weights → INT4 | Activations → FP16 | Select sensitive layers → kept in FP16.
Edge2 preserves precision where it matters most; while keeping the model small and fast enough for edge GPUs. 😎
Introducing
👉 embedl/Cosmos-Reason2-2B-W4A16-Edge2
Optimized for Jetson Orin Nano Super and AGX Orin
🚄 Try it out on Jetson (image+video+text):
docker run --rm -it \
--network host \
--shm-size=8g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
--runtime=nvidia \
--name=vllm-serve \
-e HF_TOKEN=hf_*** \
-e HF_HOME=/root/.cache/huggingface \
ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
vllm serve "embedl/Cosmos-Reason2-2B-W4A16-Edge2" \
--max-model-len 8192 \
--gpu-memory-utilization 0.75 \
--max-num-seqs 2🤓 What is Edge2? Most weights → INT4 | Activations → FP16 | Select sensitive layers → kept in FP16.
Edge2 preserves precision where it matters most; while keeping the model small and fast enough for edge GPUs. 😎
reacted to HProg's post with 🚀 7 days ago
Post
1316
Cosmos-Reason2-2B on Nano Super
Hi! Today, me and my team is releasing a version of Cosmos-Reason2-2B that is quantized so that it fits on the NVIDIA Jetson Orin Nano Super.
We managed to find a mixed precision configuration such that it maintains virtually the same accuracy as the unquantized model while being able to run really efficiently on the Nano Super and other edge devices :)
embedl/Cosmos-Reason2-2B-W4A16-Edge2
Hi! Today, me and my team is releasing a version of Cosmos-Reason2-2B that is quantized so that it fits on the NVIDIA Jetson Orin Nano Super.
We managed to find a mixed precision configuration such that it maintains virtually the same accuracy as the unquantized model while being able to run really efficiently on the Nano Super and other edge devices :)
embedl/Cosmos-Reason2-2B-W4A16-Edge2
posted an
update 8 days ago
Post
2496
⚡ Blackwell-native Vision Reasoning at the edge ⚡
Released a NVFP4A16-variant of nvidia/Cosmos-Reason2-2B:
embedl/Cosmos-Reason2-2B-NVFP4A16
💖 Optimized for Blackwell with minimal accuracy drop compared to its FP16 counterpart.
Thorough on-device benchmarks on AGX Thor in the modelcard. 🤓 📊
Try it out:
Released a NVFP4A16-variant of nvidia/Cosmos-Reason2-2B:
embedl/Cosmos-Reason2-2B-NVFP4A16
💖 Optimized for Blackwell with minimal accuracy drop compared to its FP16 counterpart.
Thorough on-device benchmarks on AGX Thor in the modelcard. 🤓 📊
Try it out:
docker run --rm -it \
--network host \
--shm-size=8g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
--runtime=nvidia \
--name=vllm-serve \
-e HF_TOKEN=hf_*** \
-e HF_HOME=/root/.cache/huggingface \
nvcr.io/nvidia/vllm:26.01-py3 \
vllm serve "embedl/Cosmos-Reason2-2B-NVFP4A16" \
--host 0.0.0.0 \
--port 8000 \
--tensor-parallel-size 1 \
--max-model-len 16384 \
--gpu-memory-utilization 0.9 posted an
update 12 days ago
Post
279
👀 Multimodal VLM running on 8GB RAM
Dropped edge-optimized Cosmos-Reason2 running on Jetson Orin. 🚀
Check it out: embedl/Cosmos-Reason2-2B-W4A16
Dropped edge-optimized Cosmos-Reason2 running on Jetson Orin. 🚀
Check it out: embedl/Cosmos-Reason2-2B-W4A16