Concise LLAMA Experience Guide

This is a very concise method for using Llama.

Llama is a language model provided by Meta, designed for experimentation, innovation, and expanding ideas. It can be used for natural language processing tasks, such as generating text and answering questions.

Meta provides two versions of the model, namely 7B and 7B-chat, which have slightly different use cases. The 7B version is more general-purpose, while the chat version focuses on conversation.

Using the Llama model requires compliance with Meta's commercial license agreement, acceptable use policy, and privacy policy. To use the Llama model, you need to download the model weights and set the correct paths. When using the Llama model, it is important to be aware of potential risks and take appropriate measures to mitigate them.

Step 1 Preparation

You need to prepare a computer with GPU computing power. Both Windows and Linux are supported. How to get a GPU farm?

Install Nvidia graphics drivers, bash, Python 3.10, Pip, Git, md5sum, and wget on it.

How do I know if I have successfully installed Nvidia drivers, CUDA, and cuDNN?

Step 2 Fill in the Form

Fill out the form here: https://ai.meta.com/llama/

Submit the form after filling it out.

You will receive an email shortly.

Step 3 Clone

Clone the repository using the command.

git clone https://github.com/facebookresearch/llama

Step 4 Download

Run download.sh in the cloned repository:

cd llama
chmod +x ./download.sh
./download.sh

It will ask you for the URL in the email. Find it in the email and provide it to it.

For example:

https://download.llamameta.net/*?Policy=eyJTdGFAAAAAAAA

Step 5 Installation

After downloading the model, run pip install -e . to set up the environment.

pip install -e .

Step 6 Run

Run it!

torchrun --nproc_per_node 1 example_text_completion.py \
    --ckpt_dir llama-2-7b/ \
    --tokenizer_path tokenizer.model \
    --max_seq_len 128 --max_batch_size 4

Step 7 Custom Modifications

It's time to use your own question!

Edit the file using any editor of your choice: complete.python

import fire

from llama import Llama

def main(
    ckpt_dir: str,
    tokenizer_path: str,
    temperature: float = 0.6,
    top_p: float = 0.9,
    max_seq_len: int = 128,
    max_gen_len: int = 64,
    max_batch_size: int = 4,
):
    generator = Llama.build(
        ckpt_dir=ckpt_dir,
        tokenizer_path=tokenizer_path,
        max_seq_len=max_seq_len,
        max_batch_size=max_batch_size,
    )

    prompts = [
        "The first person to reach the South Pole was the Norwegian explorer Roald Amderson and his team. However, the second person",
    ]
    results = generator.text_completion(
        prompts,
        max_gen_len=max_gen_len,
        temperature=temperature,
        top_p=top_p,
    )
    for prompt, result in zip(prompts, results):
        print(prompt)
        print(f"> {result['generation']}")
        

if __name__ == "__main__":
    fire.Fire(main)

Give it a try after running it!

torchrun --nproc_per_node 1 ./complete.python --ckpt_dir llama-2-7b/ --tokenizer_path tokenizer.model --max_seq_len 128 --max_batch_size 4

You have now successfully set up a large language model running on your computer.