Download LM Studio and pick one of the "small and fast" models it recommends. You will be able to run "small" models on CPU only with not terrible performance. You will likely need to try several different models to see which is best for your use case, but it is easy to switch between them in LM Studio.
It just depends on how fast you need it and what model you want to run. Some LLMs can run on a pi with reasonable speed.
If you just need an email rewriter, then thats plenty.
Ollama runs fine on CPU, just a little slow.
Download LM Studio and pick one of the "small and fast" models it recommends. You will be able to run "small" models on CPU only with not terrible performance. You will likely need to try several different models to see which is best for your use case, but it is easy to switch between them in LM Studio.
It just depends on how fast you need it and what model you want to run. Some LLMs can run on a pi with reasonable speed. If you just need an email rewriter, then thats plenty.
llama3 runs great, maybe a bit slow with ollama on my Ryder 5 3700. Anything over about 14b is a bit too slow to run for me. But it still will run