Add llama.cpp/README.md

2025-03-14 04:29:14 +00:00
parent 25310c4ffc
commit 9d7db43261
1 changed files with 4 additions and 0 deletions
--- a/llama.cpp/README.md
+++ b/llama.cpp/README.md
@@ -0,0 +1,4 @@
+# Gemma3:12b:Q4 on CUDA
+```sh
+docker run -p 1032:8080 -v $HOME/.cache/llama.cpp:/root/.cache/llama.cpp --gpus all ghcr.io/ggml-org/llama.cpp:server-cuda -t 2 --prio 3 -ngl 99 -fa --temp 0 -hf ggml-org/gemma-3-12b-it-GGUF:Q4_K_M
+```