diff --git a/llama.cpp/README.md b/llama.cpp/README.md new file mode 100644 index 0000000..bf54beb --- /dev/null +++ b/llama.cpp/README.md @@ -0,0 +1,4 @@ +# Gemma3:12b:Q4 on CUDA +```sh +docker run -p 1032:8080 -v $HOME/.cache/llama.cpp:/root/.cache/llama.cpp --gpus all ghcr.io/ggml-org/llama.cpp:server-cuda -t 2 --prio 3 -ngl 99 -fa --temp 0 -hf ggml-org/gemma-3-12b-it-GGUF:Q4_K_M +``` \ No newline at end of file