Explore real-time Computer Vision with Model Context Protocol (MCP)
OppaAI
OppaAI
AI & ML interests
Local AI implementation, Agentic AI workflows, AI Autonomous Robot
Recent Activity
repliedto their post about 20 hours ago
Another small 4B model comes out yesterday.
NeoHorse 1 4B
https://huggingface.co/TokenRhythm/NeoHorse-1-4B
There are quite a few good smaller parameter models that are capable for Agentic tasks:
The ones from the chart, I have tried a few already in my Jetson Orin Nano,
❌Gemma4 E2B IT - cannot fit my RAM usage if use with TTS and embedder
❓Qwen3.5 4B - just barely fit my RAM usage, need to add think/no_think
❌Spark X2.5 4B - need to build the forked llama.cpp; no vision
➡️Nanbeige 4.2 3B - need to build the forked llama.cpp; slower than Ministral3-3B by 25%; no vision but good for coding; maybe run this is separate server for doing coding tasks
➡️Agents A1 4B - This one is quite interesting. Another Qwen3.5 4B base. I just learnt this right now. This model may surpassed the Ministral3-3B that I'm currently running.
➡️NeoHorse 1 4B - wait for GGUF version comes out; Qwen 3.5 4B base with vision striped
➡️Needle2 45M - need to use separately from llama.cpp server; currently testing to see if it can be used as spawning sub-agents to do parallel tasks repliedto their post about 2 months ago
Benchmark test: Jev vs. Laya-ONNX (multilingual) vs. Harrier OSS 270M embedder 🔬
My AI wAIfu (Jetson Orin Nano 8GB) uses Harrier OSS 270M for semantic routing in 2 places. It reads vectors of router prompts (English only) and calculates cosine similarity:
- Quaternary routing: greeting, local chat (no websearch), web chat (needs websearch), or agentic chat
- Agentic routing: which tools in my AI's capability list to use
Benchmarked the 2 most hyped decision models — Jev and Laya (ONNX, multilingual) — against Harrier OSS 270M.
Setup: 221 quaternary + 58 capability-trigger examples, leave-one-out eval, argmax, no thresholds.
Results:
→ Harrier-270M (local, cosine): 94.6% / 93.1% accuracy, 17ms P50 ⚡
→ Jev API (hosted): 82.4% / 94.8% accuracy, ~195ms P50
→ Laya-ONNX multilingual (fp16, local): 48.0% / 20.7% accuracy, 25-40ms P50
Conclusion:
🚫 Laya is out of the question. 4 of 7 capability categories at 0.0% accuracy while reporting 80-90% confidence means it needs real training before it's practical.
☁️ Jev is a cloud API, not sure it can be trained further. Accuracy is high but not improvable on my end. Latency is ~10x my local embedder (network latency). Input token cost, though small, is still more than $0. Not fully sure about privacy implications either.
✅ Embedding is only semantic cosine similarity, not real reasoning. But it's already doing double duty for memory extraction and RAG — no extra RAM or token cost. Latency is 17ms, accuracy in the 90s%. Even tried Japanese/Chinese prompts, still got high accuracy with only English exemplars.
Bigger advantage: I just add exemplars to boost accuracy. When I add/modify/remove tools — often — no retraining needed, just update exemplars, vectors recompute once.
Turns out my self-invented routing method, built ~6 months ago, already solved what these now hyped up models — beating Jev and Laya on latency and convenience, matching/beating on accuracy. 🎯 repliedto their post about 2 months ago
Benchmark test: Jev vs. Laya-ONNX (multilingual) vs. Harrier OSS 270M embedder 🔬
My AI wAIfu (Jetson Orin Nano 8GB) uses Harrier OSS 270M for semantic routing in 2 places. It reads vectors of router prompts (English only) and calculates cosine similarity:
- Quaternary routing: greeting, local chat (no websearch), web chat (needs websearch), or agentic chat
- Agentic routing: which tools in my AI's capability list to use
Benchmarked the 2 most hyped decision models — Jev and Laya (ONNX, multilingual) — against Harrier OSS 270M.
Setup: 221 quaternary + 58 capability-trigger examples, leave-one-out eval, argmax, no thresholds.
Results:
→ Harrier-270M (local, cosine): 94.6% / 93.1% accuracy, 17ms P50 ⚡
→ Jev API (hosted): 82.4% / 94.8% accuracy, ~195ms P50
→ Laya-ONNX multilingual (fp16, local): 48.0% / 20.7% accuracy, 25-40ms P50
Conclusion:
🚫 Laya is out of the question. 4 of 7 capability categories at 0.0% accuracy while reporting 80-90% confidence means it needs real training before it's practical.
☁️ Jev is a cloud API, not sure it can be trained further. Accuracy is high but not improvable on my end. Latency is ~10x my local embedder (network latency). Input token cost, though small, is still more than $0. Not fully sure about privacy implications either.
✅ Embedding is only semantic cosine similarity, not real reasoning. But it's already doing double duty for memory extraction and RAG — no extra RAM or token cost. Latency is 17ms, accuracy in the 90s%. Even tried Japanese/Chinese prompts, still got high accuracy with only English exemplars.
Bigger advantage: I just add exemplars to boost accuracy. When I add/modify/remove tools — often — no retraining needed, just update exemplars, vectors recompute once.
Turns out my self-invented routing method, built ~6 months ago, already solved what these now hyped up models — beating Jev and Laya on latency and convenience, matching/beating on accuracy. 🎯