免费背景图片素材网站,北京企业建站程序,网站建设功能选择表,皖icp网站建设一、LMDeply方式部署
使用 LMDeploy 以本地对话方式部署 InternLM-Chat-7B 模型#xff0c;生成 300 字的小故事 2.api 方式部署
运行 结果#xff1a; 显存占用#xff1a; 二、报错与解决方案
在使用命令#xff0c;对lmdeploy 进行源码安装是时#xff0c;报错
1.源…一、LMDeply方式部署
使用 LMDeploy 以本地对话方式部署 InternLM-Chat-7B 模型生成 300 字的小故事 2.api 方式部署
运行 结果 显存占用 二、报错与解决方案
在使用命令对lmdeploy 进行源码安装是时报错
1.源码安装语句
pip install lmdeploy[all]v0.1.0
2.报错语句
Building wheels for collected packages: flash-attnBuilding wheel for flash-attn (setup.py) ... errorerror: subprocess-exited-with-error× python setup.py bdist_wheel did not run successfully.│ exit code: 1╰─ [9 lines of output]fatal: not a git repository (or any of the parent directories): .gittorch.__version__ 2.0.1running bdist_wheelGuessing wheel URL: https://github.com/Dao-AILab/flash-attention/releases/download/v2.4.2/flash_attn-2.4.2cu118torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whlerror: urlopen error Tunnel connection failed: 503 Service Unavailable[end of output]note: This error originates from a subprocess, and is likely not a problem with pip.ERROR: Failed building wheel for flash-attnRunning setup.py clean for flash-attn
Failed to build flash-attn
ERROR: Could not build wheels for flash-attn, which is required to install pyproject.toml-based projects
3.解决方法
1在https://github.com/Dao-AILab/flash-attention/releases/ 下载对应版本的安装包
2通过pip 进行安装
pip install flash_attn-2.3.5cu117torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl 4.参考链接
https://github.com/Dao-AILab/flash-attention/issues/224
