diff --git a/.gitignore b/.gitignore index 36ed7f7fd..6c4402430 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,4 @@ -.vscode/ +# .vscode/ _skbuild/ diff --git a/.vscode/launch.json b/.vscode/launch.json new file mode 100644 index 000000000..2b2502c69 --- /dev/null +++ b/.vscode/launch.json @@ -0,0 +1,16 @@ +{ + // Use IntelliSense to learn about possible attributes. + // Hover to view descriptions of existing attributes. + // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 + "version": "0.2.0", + "configurations": [ + { + "name": "Python: Current File", + "type": "python", + "request": "launch", + "program": "${file}", + "console": "integratedTerminal", + "justMyCode": false + } + ] +} \ No newline at end of file diff --git a/tests/test_llama.py b/tests/test_llama.py index 941287de6..2bdcfec18 100644 --- a/tests/test_llama.py +++ b/tests/test_llama.py @@ -169,3 +169,16 @@ def test_llama_server(): } ], } + + +def test_model(): + # ./main -m ./models/ggml-model-q4_0.bin -p "Q: 广东2018的文科分数线是多少。这个问题的年份和省份分别是多少,用json的方式回答的结果是 A: " -n 12 + # import os + # os.environ["N_THREAD"] = "16" + model_path = "/home/faith/llama-cpp-python/vendor/llama.cpp/models/ggml-model-q4_0.bin" + llm = llama_cpp.Llama(model_path=model_path, n_threads=16) + output = llm("Q: 广东2018的文科分数线是多少。这个问题的年份和省份分别是多少,用json的方式回答的结果是 A: ", max_tokens=12, stop=["Q:", "\n"], echo=True) + print(output) + + +test_model() \ No newline at end of file