Gpustack安装

https://www.cnblogs.com/sealio/p/18541188

https://zhuanlan.zhihu.com/p/6364612902

环境

• py 3.12.5

• pipx

  https://pipx.pypa.io/stable/installation/

  https://scoop.sh/

  # Scope环境安装
  # PowerShell非管理员 运行
  Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
  Invoke-RestMethod -Uri https://get.scoop.sh | Invoke-Expression
  
  # pipx安装
  scoop install pipx
  pipx ensurepath
  

  或者使用pip安装

  py -m pip install --user pipx
  py -m pipx ensurepath
  

  验证

  pipx --version
  

• cuda ≥ 12.4 & cudnn 9

  cuda教程

Gpustack服务端

服务端不能和worker安装在同一设备

Linux

Centos7

依赖

• py 3.12.5

  # pyenv
  curl https://pyenv.run | bash
  vim ~/.bashrc
  
  export PYENV_ROOT="$HOME/.pyenv"
  [[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
  eval "$(pyenv init - bash)"
  
  # 安装 Py
  su - 
  pyenv install 3.12.5
  pyenv global 3.12.5
  
  # 安装失败提示ssl,需要提前编译 openssl 1.1.1k
  CPPFLAGS="$(pkg-config --cflags openssl11)" \
  LDFLAGS="$(pkg-config --libs openssl11)" \
  pyenv install -v 3.12.5
  
  
  python3 --version
  pip3 --version
  

• openssl 1.1.1k

  wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz -O /opt/download/
  tar zxvf openssl-1.1.1g.tar.gz -C /opt/env/
  cd /opt/env/openssl-1.1.1g
  ./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl no-ssl2
  make -j8 && make install
  

• lsof

  sudo yum update -y
  sudo yum groupinstall "Development Tools" -y 
  sudo yum install build-essential libopenfst-dev openfst-tools openssl-devel libffi-devel bzip2-devel lsof wget -y
  # gcc --version
  

安装

# 修改pipx识别的默认Py版本,仅本终端有效
export PIPX_DEFAULT_PYTHON=$(pyenv which python)

curl -sfL https://get.gpustack.ai | sh -s - --tools-download-base-url 'https://pypi.tuna.tsinghua.edu.cn/simple' --port 36666 --data-dir '/opt/env/gpustack-data'

Docker

The installation of GPUStack worker on a Linux system requires that the GLIBC version be 2.29 or higher. If your system uses a lower GLIBC version, consider using theDocker Installationmethod as an alternative.

需要保证GLIBC 版本,请百度

ldd --version

https://docs.gpustack.ai/0.6/installation/cpu/online-installation/#docker-installation

https://docs.gpustack.ai/0.6/installation/nvidia-cuda/online-installation/#docker-installation

# docker run -d --gpus all -p 36666:80 --ipc=host -v gpustack-data:/var/lib/gpustack gpustack/gpustack
docker run -d --name gpustack --restart=unless-stopped -p 36666:80 -p 10150:10150 -v gpustack-data:/var/lib/gpustack gpustack/gpustack:latest-cpu
# 密码
docker exec -it gpustack cat /var/lib/gpustack/initial_admin_password
# token
docker exec -it gpustack cat /var/lib/gpustack/token

Windows

# PowerShell管理员 运行
Invoke-Expression "& { $((Invoke-WebRequest -Uri 'https://get.gpustack.ai' -UseBasicParsing).Content) } --tools-download-base-url 'https://pypi.tuna.tsinghua.edu.cn/simple' --port 36666 --data-dir 'H:\gpustack-data'"

# 上述安装命令结束会提示
#GPUStack UI is available at http://localhost:36666.
#Default username is 'admin'.
#To get the default password, run 'Get-Content -Path "H:\gpustack-data\initial_admin_password" -Raw'.

# 获取初始密码
Get-Content -Path "H:\gpustack-data\initial_admin_password" -Raw
# 获取token
Get-Content -Path "H:\gpustack-data\token" -Raw

Gpustack客户端(worker)

worker => 客户端,集群设备

命令和服务端没什么差别,就是多了--token和--worker-ip

手动安装

依赖

• py 3.12.5

• cuda ≥ 12.4 & cudnn 9

  http://www.package-online.cn/cuda/

• gpustack

• gpustack 0.6

# 检查环境
python3 --version
pip3 --version
nvcc --version
Get-ChildItem -Path C:\ -Recurse -Filter "cudnn*.dll" -ErrorAction SilentlyContinue
gpustack version

# 安装环境
pyenv install 3.12.5
pyenv global 3.12.5
pip install "gpustack"
pip install gpustack

启动bat

:: --worker-ip 自己的ip  指定当前设备为worker   --server-url 连接的服务端
@echo off && chcp 65001 && cls
gpustack start --server-url http://172.26.176.189:36666 --token xxxx --worker-ip 172.26.176.5 --data-dir H:\gpustack-data
pause

官方自动安装版

# 官方一键命令(耗时较长,请等待)(建议使用上面的手动安装)
# Invoke-Expression "& { $((Invoke-WebRequest -Uri "https://get.gpustack.ai" -UseBasicParsing).Content) } --tools-download-base-url 'https://pypi.tuna.tsinghua.edu.cn/simple' --server-url 'http://server_ip:port' --token 'server_token'"

Invoke-Expression "& { $((Invoke-WebRequest -Uri "https://get.gpustack.ai" -UseBasicParsing).Content) } --tools-download-base-url 'https://pypi.tuna.tsinghua.edu.cn/simple' --server-url 'http://172.26.176.189:36666' --token 'xxxx'"

# --server-url  服务端  --token 服务端秘钥  --worker-ip  本机ip
gpustack start --server-url 'http://172.26.176.189:36666' --token 'xxxx' --worker-ip '172.26.176.5'