I have a Framework Desktop that I want to put AI models on. I keep hearing that local models are getting more and more impressive. I would love to one day get off of my Claude dependence for the sake of privacy and running things in house.
I am currently trying out using pi agent with GLM-4.7-Flash Q8_0.
I don't have much of a reference to say how it compares to other local models, but it's definitely not something I could switch over to as a main driver instead of Claude.
What are you guys thoughts?
This is also the first time I've tried Pi. I've also been recommended hermes, which I know very little about.
▗▄▄▄ ▗▄▄▄▄ ▄▄▄▖ root@nixos
▜███▙ ▜███▙ ▟███▛ ------------
▜███▙ ▜███▙▟███▛ OS: NixOS 26.05 (Yarara) x86_64
▜███▙ ▜██████▛ Host: Desktop (AMD Ryzen AI Max)
▟█████████████████▙ ▜████▛ ▟▙ Kernel: Linux 6.18.44
▟███████████████████▙ ▜███▙ ▟██▙ Uptime: 11 days, 21 hours, 26 ms
▄▄▄▄▖ ▜███▙ ▟███▛ Packages: 500 (nix-system)
▟███▛ ▜██▛ ▟███▛ Shell: bash 5.3.9
▟███▛ ▜▛ ▟███▛ Terminal: /dev/pts/7
▟███████████▛ ▟██████████▙ CPU: AMD RYZEN AI MAX+ 395 (32)z
▜██████████▛ ▟███████████▛ GPU: AMD Radeon 8060S Graphics ]
▟███▛ ▟▙ ▟███▛ Memory: 39.81 GiB / 125.09 GiB )
▟███▛ ▟██▙ ▟███▛ Swap: 6.73 MiB / 7.45 GiB (0%)
▟███▛ ▜███▙ ▝▀▀▀▀ Disk (/): 119.94 GiB / 3.57 TiB4
▜██▛ ▜███▙ ▜██████████████████▛ Local IP (enp191s0): 192.168.0.4
▜▛ ▟████▙ ▜████████████████▛ Locale: en_US.UTF-8
▟██████▙ ▜███▙
▟███▛▜███▙ ▜███▙
▟███▛ ▜███▙ ▜███▙
▝▀▀▀ ▀▀▀▀▘ ▀▀▀▘
I know it's not exactly what you're asking for, but I've been using qwen3.8 flash-next, glm 5.3 flash, deepseek v4 flash, v4 pro and gemma-4 26b-a4b through Cortecs (openrouter alternative) and they've surprised me by how good they are for most tasks. I still have access to big models like Sonnet and so on but I rarely reach out for them.
If you're not completely ready to abandon the big models, perhaps try something like Cortecs or Openrouter as a middle ground? You can still self-host the smaller ones and reach out and use something like Sonnet through them when needed.
Some providers have strict data retention policies, are powered by green energy and so on... I use open-webui front-end to interact with Cortecs.
GLM 5.3 flash if i remember correctly had even some benchmarks that even beat Opus. I know there might be some benchmaxxing going on but still. And it's really cheap.
Thanks for the info, it's totally welcomed - I am trying my best to learn by sponge mode 🧽 😄
Yeah, I had not considered using one of those middle grounds and will check them out 🙏