Stuck getting AI Hat2+ working

Support Raspberry Pi
1

Ok folks, long story. Sorry.

Just bought a Pi 5 8G, PSU, case AT-Hat2+, heat sink.

In an attempt to get an AI LLM working.

I’ve tried 3 sites now on how to get it working, but they all fail at the same point.

sudo dpkg -i hailo_gen_ai_model_zoo_*.deb
(Reading database ... 95624 files and directories currently installed.)
Preparing to unpack hailo_gen_ai_model_zoo_5.1.1_arm64.deb ...
Unpacking hailo-gen-ai-model-zoo (5.1.1) over (5.1.1) ...
Setting up hailo-gen-ai-model-zoo (5.1.1) ...
chmod: cannot access '/usr/share/hailo-ollama/models': No such file or directory
dpkg: error processing package hailo-gen-ai-model-zoo (--install):
 installed hailo-gen-ai-model-zoo package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 hailo-gen-ai-model-zoo

(Cringe, on my screen that looks horrible.)

The line of concern:
chmod: cannot access '/usr/share/hailo-ollama/models': No such file or directory

And - of course - it is true.
That directory doesn’t exist.
There is a hailo-models.
But no hailo-ollama/......

So I am really stuck with what to do.
THREE sites all get to that command and it fails.

People have used it, so it can’t be wrong. But it seems to hate me.

There are only about 4 commands before this, and all worked.

I’ve installed the cut down version of the latest O/S (64 bit)
It is headless BTW.

Anyway. I’m stuck.
Asked in a few places. But if I stop asking: Sod’s law will happen and I won’t get an answer.
But if I keep asking, the last one will get me the answer.

Thanks in advance.

2

Hey @Andrew41918,

First of all, it has been a good while since I last touched any AI HAT2 commands, so I’m pretty rusty, but I think I was getting a similar issue, and I had a squizz and found a GitHub issue for the thing you are getting stuck on.

I ended up getting frustrated and did it all in a more manual route. I can’t remember the exact instructions but just to point you in the right direction, I created a Hailo Developer Zone account, then I went to the page for the Hailo 10H (the AI HAT 2 chip), and downloaded the PCIe driver, Runtime Library, and Python Wheel for it. I have the commands copy pasted here but they are going to be out of date, head to there and look for the newest versions. But it should look something like:

cd ~/Downloads
# PCIe Driver
wget https://hailo-model-zoo.s3.us-west-2.amazonaws.com/HailoRT/v5.1.0/hailort-pcie-driver_4.19.0_all.deb
# Runtime Library
wget https://hailo-model-zoo.s3.us-west-2.amazonaws.com/HailoRT/v5.1.0/hailort_5.1.0_arm64.deb
# Python Wheel (cp312)
wget https://hailo-model-zoo.s3.us-west-2.amazonaws.com/HailoRT/v5.1.0/hailort-5.1.0-cp312-cp312-linux_aarch64.whl

But update the versions with the newest versions you find in Dev Zone. Then you will need to install the PCIe driver, and Runtime library, then reboot.

Create your Venv, then install the wheel. Then I installed Hailo Apps, and also installed the compiled model from Hailo Dev Zone as well:

wget https://hailo-model-zoo.s3.us-west-2.amazonaws.com/ModelZoo/Compiled/v2.13.0/hailo10h/qwen2_vl_instruct_2b_hailo10h.hef -O ~/hailo_apps_work/qwen2_vl_instruct_2b_hailo10h.hef

Sorry I can’t go into exact steps for all of this, but this is hopefully enough to show you another possible route. There were a lot of little weird quirks we had to do as we installed this pre-release and compatibility was weird at the time.

If you punch this into an LLM, they might be able to help get you going as well!

Hope this helps!

1 Like
3

Well,
for reasons unknown: I have got past that point now.
I am needing to choose which LLM to install.

I play with JS a lot.
Rather complicated code.

Suggestions?

I uninstalled the entire ollama package and did it again.

4

Great news!

It is very close to working now.

But I’m stuck on this part.

The pages loaded, I created an account - bogus email as it is all local - and it gets to this point.

Again: to the best of my knowledge, I’ve done every step to this point.
So I don’t get why the server connection error is happening.

Screenshot from 2026-07-04 06-32-16
Screenshot from 2026-07-04 06-32-161917×935 42.5 KB

5

Latest:

This error doesn’t look good:

curl http://localhost:8000/api/generate -d '{
  "model": "qwen2.5-coder:1.5b",
  "prompt": "hello",
  "stream": false
}'
server=oatpp/1.4.0
code=500
description=Internal Server Error
stacktrace:
  - No suitable mapper found to deserialize the request body.
  - [ApiController]: Error processing request
  - Error processing request

The AI model is to be accessed on/from OTHER machines on this network.
NOT locally.
It is headless.