I've been working on that exact problem for the last couple weeks. My solution for now is rt patch and a dedicated cpu core for rt tasks. This already works pretty reliable, but I notice small delays from time to time. I gather from the article that my problem might be page swapping. I don't know how to improve that, yet.
Also for anybody working on rt problems: I highly recommend the stress-ng
tool for stress testing and finding bottlenecks of your system.
Big oof for the Haas boys