Petals以BitTorrent方式在家中运行超过100B的语言模型。
本笔记本介绍如何使用Langchain和Petals (opens in a new tab)。
安装Petals#
要使用Petals API，需要安装petals包。使用pip3 install petals进行安装。
!pip3 install petals
导入#
import os
from langchain.llms import Petals
from langchain import PromptTemplate, LLMChain
设置环境API密钥#
请确保从Huggingface获取API密钥 (opens in a new tab)。
from getpass import getpass
HUGGINGFACE_API_KEY = getpass()
os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY
Create the Petals instance#
You can specify different parameters such as the model name, max new tokens, temperature, etc.
# this can take several minutes to download big files!
llm = Petals(model_name="bigscience/bloom-petals")
Downloading: 1%|▏ | 40.8M/7.19G [00:24<15:44, 7.57MB/s]
Create a Prompt Template#
We will create a prompt template for Question and Answer.
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
Initiate the LLMChain#
llm_chain = LLMChain(prompt=prompt, llm=llm)
Run the LLMChain#
Provide a question and run the LLMChain.
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
llm_chain.run(question)