This desktop app for hosting and running LLMs locally is rough in a few spots, but still useful right out of the box.
CIDE contains extensive support for C/C++ thanks to using libclang. CUDA is supported to varying degrees depending on how well the libclang and CUDA versions on the system work together. There is also ...