4 points | by tarruda 2 hours ago ago
1 comments
The official Q4_K_S gguf is quite good and has very good 35 tps generation on a M1 mac studio. Should be much faster on recent Macs, especially M5.
The official Q4_K_S gguf is quite good and has very good 35 tps generation on a M1 mac studio. Should be much faster on recent Macs, especially M5.