HuggingFace模型下载
使用HuggingFace官方库下载模型
安装官方库
github地址:https://github.com/huggingface/huggingface_hub
pip install huggingface_hub
下载:
指定文件
from huggingface_hub import hf_hub_download
# 也可以不指定filename
hf_hub_download(repo_id="TheBloke/Llama-2-7B-Chat-GGML", filename="llama-2-7b-chat.ggmlv3.q2_K.bin")
并行下载
from huggingface_hub import hf_hub_download
import joblib
REPO_ID = "YOUR_REPO_ID"
FILENAME = "sklearn_model.joblib"
model = joblib.load(
hf_hub_download(repo_id=REPO_ID, filename=FILENAME)
)
下载整个库
from huggingface_hub import snapshot_download
snapshot_download(repo_id="lysandre/arxiv-nlp")
代理规则:
# HuggingFace模型
DOMAIN-SUFFIX,cdn-lfs.huggingface.co