I have a Framework Desktop that I want to put AI models on. I keep hearing that local models are getting more and more impressive. I would love to one day get off of my Claude dependence for the sake of privacy and running things in house.
I am currently trying out using pi agent with GLM-4.7-Flash Q8_0.
I don't have much of a reference to say how it compares to other local models, but it's definitely not something I could switch over to as a main driver instead of Claude.
What are you guys thoughts?
This is also the first time I've tried Pi. I've also been recommended hermes, which I know very little about.
▗▄▄▄ ▗▄▄▄▄ ▄▄▄▖ root@nixos
▜███▙ ▜███▙ ▟███▛ ------------
▜███▙ ▜███▙▟███▛ OS: NixOS 26.05 (Yarara) x86_64
▜███▙ ▜██████▛ Host: Desktop (AMD Ryzen AI Max)
▟█████████████████▙ ▜████▛ ▟▙ Kernel: Linux 6.18.44
▟███████████████████▙ ▜███▙ ▟██▙ Uptime: 11 days, 21 hours, 26 ms
▄▄▄▄▖ ▜███▙ ▟███▛ Packages: 500 (nix-system)
▟███▛ ▜██▛ ▟███▛ Shell: bash 5.3.9
▟███▛ ▜▛ ▟███▛ Terminal: /dev/pts/7
▟███████████▛ ▟██████████▙ CPU: AMD RYZEN AI MAX+ 395 (32)z
▜██████████▛ ▟███████████▛ GPU: AMD Radeon 8060S Graphics ]
▟███▛ ▟▙ ▟███▛ Memory: 39.81 GiB / 125.09 GiB )
▟███▛ ▟██▙ ▟███▛ Swap: 6.73 MiB / 7.45 GiB (0%)
▟███▛ ▜███▙ ▝▀▀▀▀ Disk (/): 119.94 GiB / 3.57 TiB4
▜██▛ ▜███▙ ▜██████████████████▛ Local IP (enp191s0): 192.168.0.4
▜▛ ▟████▙ ▜████████████████▛ Locale: en_US.UTF-8
▟██████▙ ▜███▙
▟███▛▜███▙ ▜███▙
▟███▛ ▜███▙ ▜███▙
▝▀▀▀ ▀▀▀▀▘ ▀▀▀▘
I've got the same goal as you. My findings follow, I hope they're helpful:
Frontier is good for planning which takes relatively few tokens compared to writing the code. You can run a frontier model like Claude and tell it to use your local AI model as a coding resource - hand it tasks and check the output. Qwen-3.8:27B is my local. It's good, and fast.
If you want to break your dependency on American frontier models specifically, you can use GLM-5.3 through openrouter for planning. You might find yourself using it for everything, because it's pretty cheap.
I'm doing the first one, myself. Often I'll use Hermes running GLM-5.3-flash (on ollama cloud) directly instead of Claude Code for smaller tasks.
Great, thanks for the info! How do you like hermes, have you tried out pi?
Hermes is alright! It doesn't ship with a way for the classifier to stop the agent sending you a message that doesn't check out (e.g. "all tests green" when there's failures, or "job done" when it created just a stub). Claude Code does, so I built that hook for my fork of Hermes.
I haven't seen pi yet, I'll check it out! What do you like about it? Have you tried any others/got preferences yet?
I am brand new to Pi as of this week, so far my evolution in the past years went from... Visual Studio (oh wow it'd been so long i've forgotten the name 🤣).... then Cursor... then Claude Code. I tried a few neovim integrated AI workflows over a year ago, but they were all too buggy.
I caught wind of Pi I think from the youtuber omerxx and decided to give it a spin cause of its claims to be highly customizable. I haven't had the same time I did in the past to really tinker with my setups, but I figure i'll do it in baby steps over time.