Title: Hands-On Large Language Models
Tags: Chapter-1 Chapter-2

Q: What is the difference between traditional machine learning training and LLM training?
A: Traditional machine learning generally involves training a model for a specific task, like classification. As shown in Figure below, we consider this to be a one-step process:
Creating LLMs, in contrast, typically consists of at least 2 steps:
1. Language modeling: not yet directed toward specific tasks or applications beyond predicting the next word
2. Fine-tuning

Q: What are important factors in purchasing a laptop for AI-development?
A: - A powerful GPU (graphics card) will make both training and using LLMs much more efficient and faster
- The amount of VRAM (video random-access memory), the amount of memory available on the GPU. In practice, the more VRAM the better. The reason for this is that some models simply cannot be used at all if we do not have sufficient VRAM.

Q: What is the pros and cons of proprietary and open model?
A: - Proprietary
  - pros
    - User does not need to have a strong GPU to use the LLM
    - no expertise necessary concerning hosting and using the model
  - cons
    - costly paid
    - cannot fine-tune
    - data is shared with the provider, which is not desirable in many common use cases, such as sharing patient data
- Open
  - pros
    - We can download these models and use them on your device as long as you have a powerful GPU
    - complete control over the model
  - cons
    - need powerful hardware
    - requires technical knowledge to set up and use these models

Q: When you use an LLM, what two models are loaded?
A: 1. The generative model itself
2. Its underlying tokenizer
We can use transformers to load both the tokenizer and model transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    device_map="cuda",
    torch_dtype="auto",
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
transformers.pipeline encapsulates transformers the model, tokenizer, and text generation process into a single function:
import torch
from transformers import pipeline
# Using Meta's Llama-3.2-3B-Instruct (Non-Chinese, US-based Meta)
model_id = "meta-llama/Llama-3.2-3B-Instruct"
# Initialize the pipeline for text generation
# The pipeline handles tokenization and model loading internally
pipe = pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": "auto"},
    device_map="cuda",  # Moves logic to GPU automatically
)
# Example usage with a chat-style input structure
messages = [
    {
        "role": "user",
        "content": "Explain Kant's distinction between analytic and synthetic judgments."
    },
]
# Run generation
outputs = pipe(
    messages,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.7,
)
# The pipeline automatically applies the chat template for Llama 3
print(outputs[0]["generated_text"][-1])

Q: What Is Language AI?
A: Language AI refers to a subfield of AI that focuses on developing technologies capable of understanding, processing, and generating human language. The term Language AI can often be used interchangeably with natural language processing (NLP)

Q: Why is language a tricky concept for computers?
A: Text is unstructured in nature and loses its meaning when represented by zeros and ones. There has been a large focus on representing language in a structured manner so that it can more easily be used by computers

Q: Describe bag-of-word, its limitations, and its successor for addressing such limitation
A: Bag-of-words is a method for representing unstructured text and works as follows
The first step of the bag-of-words model is tokenization whose most common method is splitting by whitespace:
Then we combine all unique words from each sentence to create a vocabulary that we can use to represent the sentences:
Next count how often a word in each sentence appears, creating a bag of words as vector representations:
Bag-of-words ignores the semantic nature, or meaning, of text. word2vec attempts to captures the meaning of text in embeddings

Q: What are the shortages of word2vec? What addresses this issue?
A: word2vec creates static representations of words; embeddings, however, should change depending on the context addressed by RNN

Q: What are the shortages of RNN? How is this issue addressed?
A: This context embedding of RNN makes it difficult to deal with longer sentences. Attention allows a model to focus on parts of the input sequence that are relevant to one another (“attend” to each other) and amplify their signal

Q: What distinguishes transformer from attention mechanism?
A: Transformer removed RNN while original attantion mechanism still uses it

Q: How do Encoder-Decoder (transformer) & Encode-Only & Decoder Only differ from each other?
A: - The original Transformer model is an encoder-decoder architecture that serves translation tasks well but cannot easily be used for other tasks, like text classification.
- Encoder-Only (BERT) deals with such issue with its capability of transfer-learning, which involves first pretraining it for language modeling and then fine-tuning it for a specific task. Encoder-only models mainly focus on representing language, for instance, by creating embeddings, and typically do not generate text.
- Decoder-Only (Generative Pre-trained Transformer (GPT)) focus primarily on generating text and typically are not trained to generate embeddings. Generative LLMs take in some text and attempt to autocomplete it. By fine-tuning these models, we can create instruct or chat models (take in a user query (prompt) and output a response that would most likely follow that prompt)

Q: What are the 3 major factors that dictate how a tokenizer breaks down an input prompt?
A: 1. Choosing a tokenization method
2. tokenizer parameter tunning
3. Dataset used to train the tokenizer

Q: What are 4 tokenization schemes?
A: 1. Word tokens
  - common with earlier methods like word2vec but is being used less and less in NLP
  - often used, however, outside of NLP such as recommendation system
  - vocabulary cannot deal with new words
  - vocabulary with tiny-difference tokens (apology, apologize, apologetic, apologist)
2. Subword tokens
  - Deals with tiny-difference tokens (apolog, -y, -ize, -etic, -ist)
  - can deal with new words by decomposing new tokens into parts already presented in the vocabulary
3. Character token
4. Byte tokens (tokenization-free encoding)
  - deal with unicode characters
  - multilingual potential

Q: What is the relationship between Tokenizer and Token Embeddings?
A: A Language Model Holds Embeddings for the Vocabulary of Its Tokenizer.
A language model holds an embedding vector associated with each token in its tokenizer
Language, to machine, is essentially a sequence of tokens. If we train a good-enough model on a large-enough set of tokens, it
starts to capture the complex patterns that appear in its training dataset:
- If the training data contains a lot of English text, that pattern reveals itself as a model capable of representing and generating the English language.
- If the training data contains factual information (Wikipedia, for example), the model would have the ability to generate some factual information (see the following note).
Once the pattern is captured, machine needs to somehow encode or "memorize" that pattern so that machine can use the pattern effectively. Hence what comes next after tokenization is to find the best numerical representation for these tokens that the model can use to calculate and properly model the patterns in the text. This is when embeddings come into play - it encodes such pattern.
Before the beginning of the training process, embedding vectors are randomly initialized, but the training process creates better (contextualized) embedding by assigning them the values that enable the useful behavior they’re trained to perform. These trained embedding vectors are essentially the pattern.
Finally, 3 facts are inferred:
1. Tokenizer and embeddings are separate concepts; both are separately trained
2. Tokenizer is trained first, after which it is used in the training process of its associated language model - this is why a pretrained language model is linked with its tokenizer and can’t use a different tokenizer without training
3. When we download a pretrained language model, a portion of the model is this embeddings matrix holding all of these vectors.