一、安装依赖
pip install transformers
pip install torch
pip install SentencePiece
二、下载模型
https://huggingface.co/openai-community/gpt2/tree/main
拷贝到目录:
运行代码:
from transformers import GPT2Tokenizer, GPT2LMHeadModel, pipeline# 加载预训练的模型和分词器
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')# 使用pipeline简化流程
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)# 文本生成
prompt = "the world is big? "
generated_text = generator(prompt, max_length=50, num_beams=5)
print(generated_text[0]['generated_text'])
得到结果:
the world is big? I don't think so. In fact, I think we're going to see a lot more of that in the next couple of years. I don't think we're going to see a lot