分享据说媲美chatGPT的llama2 本地部署和使用

不知道有人感兴趣不？超级简单，用来处理简单的问题应该很方便
我是用的linux，跑的7b模型（需要16g显存）

curl https://ollama.ai/install.sh | sh
ollama run llama2

有api接口可以调用，所以可以也开启远程：

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo vi /etc/systemd/system/ollama.service.d/environment.conf

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

systemctl daemon-reload
systemctl restart ollama

不带上下文的远程简单对话py

import requests
import json


def generate(prompt):
    try:
        url = "http://********:11434/api/generate"
        payload = {
            "model": "llama2",
            "prompt": prompt,
        }
        with requests.post(url, json=payload, stream=True) as response:
            response.raise_for_status()
            final_context = None
            full_response = ""
            for line in response.iter_lines():
                if line:
                    chunk = json.loads(line)
                    if not chunk.get("done"):
                        response_piece = chunk.get("response", "")
                        full_response += response_piece
                        print(response_piece, end="", flush=True)
                    else:
                        final_context = chunk.get("context")
            return full_response, final_context
    except requests.exceptions.RequestException as e:
        print(f"An error occurred: {e}")
        return None, None


while 1:
    generate(input("\n:  "))

生成的数据还是仅供参考