用Docker部署大语言模型Llama 3

更新于 2025-11-27 分类于 ollama 阅读次数： 7 阅读时长 ≈ 19 分钟

AI, 大语言模型

有什么用

体验一下AI目前在CPU上运行的应用; 用来学习AI大语言模型

本文被停滞了将近1年半时间; 受制于硬件和时间关系; 即便最近跑通了也更尴尬. 只要一请求打一点的运算, CPU马上100%占用. 下一步: 安排英伟达的GPU登场

怎么用

*拉取Ollama镜像命令

1	sudo docker pull ollama/ollama:latest

然后在bash命令界面,执行ollama run llama2命令，接着等待下载即可,最后出现success,表示下载运行Llama 2模型成功

docker exec -it ollama-ollama1 bash    
root@ollama-ollama1:/# ollama run llama2                                                                  
pulling manifest                                                                                          
pulling 8934d96d3f08...  89% ▕█████████████████████████████████████     ▏ 3.4 GB/3.8 GB  4.8 MB/s   1m26s

教程：https://github.com/ollama/ollama

Remove a model
ollama rm llama2

卡死；删除整个container，空间没有得到释放，郁闷。。。

20240423放弃，因为对CPU的占用太大。

*拉取Chatbot-Ollama（这是UI）镜像命令

1	sudo docker pull ghcr.io/ivanfioravanti/chatbot-ollama:main

然后设置这个UI的一个变量：该变量就是连接我们上面运行Ollama框架服务的地址,我们设置本地地址:

http://群晖局域网IP:11434

实现方法

version: "3"
services:
    localai:
        tty: true
        stdin_open: true
        container_name: local-ai
        ports:
            - 8105:8080
        image: localai/localai:latest-aio-cpu
        volumes:
            - ~/models:/models:cached

运行之前:

运行之后:

可见CPU直接飙满了~. 还好当没有请求的时候cpu不是一直这样保持爆满的状态

curl http://192.168.6.117:8105/tts -H “Content-Type: application/json” -d ‘{
“input”: “Hello world”,
“model”: “tts”
}’

1	Load model from /models/tts failed:Load model /models/tts failed. File doesn't exist"

CPU only

version: "3"
services:
    ollama:
        volumes:
            - ./ollama:/root/.ollama
        ports:
            - 8106:11434
        container_name: ollama
        image: ollama/ollama

Run a model

Now you can run a model like Llama 2 inside the container.

1	docker exec -it ollama ollama run llama2-chinese

用上一次的webui 关联起来就是

给ollama弄一个前端UI

version: "3"
services:
    open-webui:
        ports:
            - 8107:8080
            - 11434:11434
        extra_hosts:
            - host.docker.internal:host-gateway
        volumes:
            - ./open-webui-data:/app/backend/data
            - /root/ollama/ollama:/root/.ollama
        container_name: open-webui
        restart: always
        image: ghcr.io/open-webui/open-webui:main

        - /root/ollama/ollama:/root/.ollama 
        - 尝试关联本机中的ollama(独立运行的)
        -

https://www.runoob.com/ollama/ollama-open-webui.html

TTS

Coqui TTS Setup via Docker: Voice AI on Linux Mint

docker run -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu

python3 TTS/server/server.py --model_name tts_models/en/vctk/vits

version: "3"
services:
    ollama:
        volumes:
            - ./ollama:/root/.ollama
        ports:
            - 8106:11434
        container_name: ollama
        image: ollama/ollama

http://192.168.6.117:8106/

1	Ollama is running

Run a model

Now you can run a model like Llama 2 inside the container.

1 2	docker exec -it ollama ollama run ChatTTS-Ollama

StyleTTS - https://github.com/yl4579/StyleTTS2?t... Eleven’s Style TTS - https://github.com/IIEleven11/StyleTT...

Coqui TTS - https://github.com/coqui-ai/TTS

Daswers XTTS GUI - https://github.com/daswer123/xtts-fin...

Suno Bark - https://github.com/suno-ai/bark
VallE-X - https://github.com/Plachtaa/VALL-E-X

~~Tortoise TTS Installation - • Local AI Voice Cloning with Tortoise TTS -…~~ NVIDIA GPUs

最终的成功案例:

https://github.com/jianchang512/ChatTTS-ui/tree/main

安装

拉取项目仓库

在任意路径下克隆项目，例如：

1	git clone https://github.com/jianchang512/ChatTTS-ui.git chat-tts-ui

启动 Runner

进入到项目目录：

1	cd chat-tts-ui

启动容器并查看初始化日志：

gpu版本
docker compose -f docker-compose.gpu.yaml up -d 

cpu版本    
docker compose -f docker-compose.cpu.yaml up -d

docker compose logs -f --no-log-prefix

访问 ChatTTS WebUI

启动:['0.0.0.0', '9966']，也即，访问部署设备的 IP:9966 即可，例如：
- 本机：http://127.0.0.1:9966
- 服务器: http://192.168.1.100:9966

Attaching to chat-tts-ui                                                                                        
chat-tts-ui  | Starting...                                                                                      
chat-tts-ui  | Traceback (most recent call last):                                                               
chat-tts-ui  |   File "/app/app.py", line 24, in                                                        
chat-tts-ui  |     import ChatTTS                                                                               
chat-tts-ui  |   File "/app/ChatTTS/__init__.py", line 1, in                                            
chat-tts-ui  |     from .core import Chat                                                                       
chat-tts-ui  |   File "/app/ChatTTS/core.py", line 18, in                                               
chat-tts-ui  |     from .model import DVAE, GPT, gen_logits, Tokenizer                                          
chat-tts-ui  |   File "/app/ChatTTS/model/__init__.py", line 4, in                                      
chat-tts-ui  |     from .tokenizer import Tokenizer                                                             
chat-tts-ui  |   File "/app/ChatTTS/model/tokenizer.py", line 20, in                                    
chat-tts-ui  |     class Tokenizer:                                                                             
chat-tts-ui  |   File "/app/ChatTTS/model/tokenizer.py", line 22, in Tokenizer                                  
chat-tts-ui  |     self, tokenizer_path: torch.serialization.FILE_LIKE, device: torch.device                    
chat-tts-ui  | AttributeError: module 'torch.serialization' has no attribute 'FILE_LIKE'                        
chat-tts-ui exited with code 1                                                                                  
chat-tts-ui  | Starting...                                                                                      
  Gracefully stopping... (press Ctrl+C again to force)                                                          
[+] Stopping 1/1                                                                                                
 ✔ Container chat-tts-ui  Stopped

解决办法: https://github.com/2noise/ChatTTS/issues/933

修改requirements.txt 中的 torch==2.1.0

用法1: https://tts1.carlzeng.com:3/

用法2:

# API调用代码

import requests

res = requests.post('https://tts1.carlzeng.com:3/tts', data={
  "text": "若不懂无需填写",
  "prompt": "",
  "voice": "3333",
  "temperature": 0.3,
  "top_p": 0.7,
  "top_k": 20,
  "skip_refine": 0,
  "custom_voice": 0
})
print(res.json())

收藏图片/logo在线编辑工具

把SVG压缩以后,背景就消失了
https://www.iloveimg.com/zh-cn/download/hvprdj3nv68q7vbb5j2b304x3kfkj4g0c5by0207q39vxwg0pjjdp188dy2vn2rlj0986Ab0wkfcb7rty217p5hk9nmAqtvhtpn2m43ln4btdhqpw6vhrfskftn04g4zzkfv6wg3w5n13hx6qwdyg6pvylnkqpgtgcAs2z467t0tjwtb0g2q/12

结果: 
<svg xmlns="http://www.w3.org/2000/svg" width="180" height="180"><path fill="#94334B" d="M111 72c-.01.816-.02 1.631-.032 2.472q-.051 4.629-.078 9.259c-.015 1.969-.04 3.937-.066 5.906-.081 16.044 2.86 30.22 9.969 44.613C122 137 122 137 122 140c-11.961.095-23.923.182-35.875-.375l-2.372-.094c-3.718-.2-5.588-.421-8.753-2.531l-3.437-.687c-15.817-3.868-27.508-17.323-35.849-30.555C30.363 96.917 29.838 89.128 30 79l3.875.813C41.878 81.42 49.945 82.683 58 84v2c4.479.117 8.957.188 13.438.25l3.818.102C82.813 86.43 88.432 85.832 95 82l2.75-1.437c2.421-1.577 2.421-1.577 4.75-4.126C104.908 74.09 107.445 72 111 72"/><path fill="#95354D" d="M115 69c3.666-.147 7.332-.234 11-.312l3.121-.127c8.516-.136 12.948 1.984 19.234 7.51 9.312 10.442 12.457 21.242 11.824 34.969-.7 7.651-5.041 14.439-10.179 19.96-6.524 5.165-11.967 6.099-20 6-15.184-26.364-15.184-26.364-15.098-39.484l.005-2.472c.005-2.598.018-5.196.03-7.794q.009-2.646.014-5.293.017-6.478.049-12.957m14 10c-.417 2.5-.417 2.5 0 5 2.77 2.315 2.77 2.315 5.625 1.688 2.31-.462 2.31-.462 3.375-1.688.25-2.458.25-2.458 0-5-1.827-2.25-1.827-2.25-4.5-2.25s-2.673 0-4.5 2.25"/><path fill="#AF9E5A" d="M113 72h1l.044 2.488q.094 4.65.218 9.298c.05 1.98.084 3.959.119 5.939.318 11.425 2.523 20.805 7.557 31.025l1.013 2.129c2.162 4.49 4.39 8.903 7.049 13.121 2.621 1.045 2.621 1.045 5 1a149 149 0 0 1-5.383 3.445c-4.628 2.835-8.78 5.578-12.617 9.43-6.724 6.272-15.674 9.486-24 13.125l-2.038.927C88.115 165.204 86.16 166 83 166l-.563-2.447a1226 1226 0 0 0-2.624-10.928l-.885-3.852-.908-3.671-.81-3.396c-.975-2.927-.975-2.927-3.836-4.087L71 137c1.813-.625 1.813-.625 4-1l3 2c2.076.363 2.076.363 4.38.41l2.668.122 2.87.1 2.947.127c3.107.13 6.215.248 9.323.366q3.159.13 6.318.262 7.746.32 15.494.613l-1.251-3.105-1.64-4.083-.822-2.04c-1.954-4.869-3.748-9.755-5.287-14.772l-1.125-3.562c-1.713-6.73-2.07-13.403-2.062-20.313v-2.377c.144-14.705.144-14.705 3.187-17.748"/><path fill="#B2A35E" d="M19 50c11.074 0 11.074 0 15 2 5.35 7.817 9 20.6 9 30q-2.407-.466-4.812-.937l-2.708-.528C33 80 33 80 30 79l2 18-25-1C1 83.25 1 83.25 1 81q4.185-1.505 8.375-3l2.406-.867 2.305-.82 2.126-.762C18 75 18 75 20 75z"/><path fill="#B0A25C" d="M60 62c4.594.599 8.45 1.834 12.719 3.617l1.881.781q1.959.816 3.913 1.64c1.979.834 3.961 1.658 5.944 2.482C90.059 72.857 95.592 75.234 101 78c-3.559 4.333-7.957 5.926-13 8l-2 1c-2.844.1-5.657.141-8.5.125l-2.457-.006C69.305 87.078 63.696 86.735 58 86l3-1-.496-1.918c-.679-4.15-.61-8.198-.566-12.394l.013-2.534q.02-3.077.049-6.154"/><path fill="#B48179" d="m123.938 67.688 2.65.068c2.138.058 4.275.148 6.412.244v1h-18v32h-1l-1-28-1 6h-1l-1-6h-3c5.724-4.932 9.497-5.68 16.938-5.312"/><path fill="#AA9450" d="M133.5 76.75C136 77 136 77 138 79c.313 2.5.313 2.5 0 5-3 2-3 2-5.625 1.688C130 85 130 85 129 84c-.187-2.437-.187-2.437 0-5 2-2 2-2 4.5-2.25"/></svg>


BgRemover - 在线图片去底工具 - 将纯色背景的图片转换为背景透明的图片
https://www.aigei.com/bgremover/

转化为SVG
https://www.freeconvert.com/

灵感来源

群晖NAS使用Docker部署大语言模型Llama 2结合内网穿透实现公网访问本地GPT聊天服务

docker 部署 Liama

https://github.com/LlamaFamily/Llama-Chinese

个性化需求沟通扫客服加V加群：

有什么用

怎么用

*拉取Ollama镜像命令

*拉取Chatbot-Ollama（这是UI）镜像命令

相关内容

实现方法

CPU only

Run a model

TTS

Run a model

安装

收藏图片/logo在线编辑工具

灵感来源