Download to read offline
![THE RISE OF
THE
TRANSFORMER
S
• There were two major "winters" approximately 1974–1980 and
1987–2000,[3]
and several smaller episodes, including the
following:1966, 1969, 1971-75, 1973, 1987, 1988, 1990s,
• Beginning about 2012, interest in AI (and especially the sub-
field of machine learning) from the research and corporate
communities led to a dramatic increase in funding and
investment, leading to the current (as of 2025) AI boom.
• Transformers (introduced in 2017, “Attention is All You Need”)
changed everything.(RNN – LTSM – Encoder-Decoder – Enc-
Dec with attention)
They enabled scaling to billions of parameters, making LLMs
like GPT, BERT, and LLaMA possible.
Before
• CNN
• RNN
2017
• Transfo
rmers
2018
• BERT
2018
• GPT
2022
• Chat
GPT](https://image.slidesharecdn.com/doyoucontroltheai-260207152808-5fa92fd6/85/Do-You-Control-the-AI-or-Does-the-AI-Control-You-10-320.jpg)
![HOW A PROMPT IS PROCESSED IN AN LLM
•You type the instruction + paragraph.
•Example: “Summarize the following paragraph in 1 sentence.”
1. Prompt Input
•Text is split into small units (tokens).
•E.g. Summarize [Sum, marize], paragraph [para, graph].
→ →
2. Tokenization
•Each token is converted into a numeric vector.
•Position is added so the model knows word order.
3. Embedding + Position Info
•The model looks at all tokens and decides which words matter most.
•Example: “Summarize…1 sentence” gets high attention guides the
→
output.
4. Self-Attention (Transformer
magic ✨)
•The model predicts the most likely next word, step by step.
•Example: it might predict: The, then text, then explains….
5. Prediction (Next Token)
•Depending on settings (greedy, top-k, temperature), it picks words
deterministically or creatively.
6. Sampling / Decoding
• Tokens are converted back into text.
•Final result: a concise, single-sentence summary.
7. Output](https://image.slidesharecdn.com/doyoucontroltheai-260207152808-5fa92fd6/85/Do-You-Control-the-AI-or-Does-the-AI-Control-You-22-320.jpg)
Do You Control the AI, or Does the AI Control You?
- 1. DO YOU CONTROLTHE AI, OR DOES THE AI CONTROL YOU? A Practical Journey into Prompt Engineering, Alignment, and the Hidden Language of Large Models
- 2. 1. INTRODUCTIO N For decades, humanscontrolled machines with rigid commands: push buttons, type code, configure systems. But with the arrival of Large Language Models (LLMs), a new phenomenon emerged: we can now talk to machines in natural language. This sounds simple — just ask a question and get an answer. But here’s the twist: your wording, tone, and structure (your “prompt”) shape the AI’s behavior as much as code shapes a program. So, are you controlling the AI, or is the AI controlling how you ask? That’s the central puzzle of prompt engineering — and the journey we’ll explore in this webinar.
- 3. HIERARCHY VIEW OFAI ML DL NLP → → → AI - Artificial Intelligence ML - Machine Learning DL - Deep Learning Transformers LLM •Artificial Intelligence (AI) •The broad field of making machines “intelligent.” •Includes symbolic reasoning, robotics, planning, NLP, ML, etc. •Machine Learning (ML) (branch of AI) •Focused on making machines learn from data rather than explicit rules. •Contains many approaches: decision trees, SVMs, Bayesian methods, etc. •Deep Learning (DL) (sub-branch of ML) •Uses artificial neural networks with many layers. •Dominates speech recognition, computer vision, and modern NLP. •Natural Language Processing (NLP) •⚠️Important: NLP is not a sub-branch of ML/DL — it’s an application domain of AI. •NLP existed before ML (e.g., rule-based translation, symbolic parsing). •Modern NLP mostly relies on Deep Learning Transformers LLMs. → →
- 4.
- 5. 2. FOUNDATIONS OFMACHINE LEARNING • To understand prompt engineering, we must first understand where AI models come from. • Supervised Learning: model learns from labeled data (e.g., spam vs. not spam). • Unsupervised Learning: model finds patterns without labels (e.g., clustering documents). • Reinforcement Learning (RL): model learns by trial and error with rewards (e.g., teaching a robot to walk). • Deep Learning evolved when neural networks became large and powerful, handling images, speech, and text.
- 6.
- 7.
- 8.
- 9. SAMPLE DEMO 4:REWARD FUNCTION & POLICY
- 10. THE RISE OF THE TRANSFORMER S •There were two major "winters" approximately 1974–1980 and 1987–2000,[3] and several smaller episodes, including the following:1966, 1969, 1971-75, 1973, 1987, 1988, 1990s, • Beginning about 2012, interest in AI (and especially the sub- field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment, leading to the current (as of 2025) AI boom. • Transformers (introduced in 2017, “Attention is All You Need”) changed everything.(RNN – LTSM – Encoder-Decoder – Enc- Dec with attention) They enabled scaling to billions of parameters, making LLMs like GPT, BERT, and LLaMA possible. Before • CNN • RNN 2017 • Transfo rmers 2018 • BERT 2018 • GPT 2022 • Chat GPT
- 11. “ATTENTION IS ALL YOUNEED” Authored by researchers at Google Brain & Google Research: 1. Ashish Vaswani (Indian): Google Brain → Essential AI 2. Noam Shazeer (American): Google Brain → Character.AI → Google Deepmind 3. Niki Parmar (American): Google Brain → Essential AI → Anthropic 4. Jakob Uszkoreit (American): Google Brain → Inceptive 5. Llion Jones (UK-w): Google Brain → Sakana AI 6. Aidan N. Gomez (British-Canadian): Google Brain → Cohere 7. Lukasz Kaiser (Polish-Mathematician): Google Brain → OpenAI 8. Illia Polosukhin (Ukrainian): Google Brain → NEAR Protocol This work proposed the now-famous self-attention mechanism, removing the need for recurrent or convolutional layers in sequence modeling. That single paper is what laid the foundation for BERT (2018), GPT (2018), and all modern LLMs.
- 12. THE RISE OF LARGE LANGUAGE MODELS (LLMS) •What is an LLM? A model trained on massive text corpora to predict the next word (token). • What is a corpus/ corpora? A collection of texts used to train models. • Examples : • C4 (Colossal Clean Crawled Corpus) used in T5. → • Wikipedia widely used in many NLP models. → • Common Crawl large internet scrape. → • The Pile curated 800GB dataset with → academic + web content. • Project Gutenberg (Shakespeare, classic literature). Transformers (introduced in 2017, “Attention is All You Need”) changed everything. They enabled scaling to billions of parameters, making LLMs like GPT, BERT, and LLaMA possible. Beginning about 2012, interest in AI (and especially the sub-field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment, leading to the current (as of 2025) AI boom.
- 13. AI/ML TASKS • ForExample: The SuperNaturalInstructions dataset contains over 1.6K tasks, 3M examples
- 14. TRAINING & ALIGNMENT MECHANISMS — THE EVOLUTION •Pretraining •Trainon massive corpora (Common Crawl, Wikipedia, books). •✅ Gives general language ability. •⚠️Issue: Too generic → outputs often irrelevant, verbose, or unhelpful. •Fine-tuning •Train on domain/task-specific data (e.g., medical QA, coding). •✅ Improves accuracy for specific tasks. •⚠️Issue: Still rigid; doesn’t adapt well to natural instructions. •Instruction Tuning •Train on human-written instruction–response pairs. •✅ Models now follow natural language instructions. •⚠️Issue: Quality depends on dataset; may misinterpret ambiguous prompts. •RLHF (Reinforcement Learning with Human Feedback) •Humans rank model outputs → reward model → align with preferences. •✅ Produces more helpful, polite, safe answers. •⚠️Issue: Expensive, limited annotator diversity → can introduce hidden biases. •DPO (Direct Preference Optimization) •Newer, simpler alternative to RLHF; optimizes preferences directly. •✅ Reduces complexity, improves stability. •⚠️Issue: Still evolving; not as widely deployed. •Constitutional AI (Anthropic) •Instead of humans labeling everything, use a set of written “principles” to guide training. •✅ More transparent, scalable alignment. •⚠️Issue: Depends on whose “constitution” is chosen → subjective.
- 15. TRAINING & ALIGNMENTMECHANISMS Pretrained 💡Train massive corpora(Wikipedia, books) ✅ Gives general language ability ⚠️Issue: Too generic → outputs often irrelevant, verbose, or unhelpful Fine-tuning 💡Train on domain/task-specific data (e.g., medical QA, coding) ✅ Improves accuracy for specific tasks. ⚠️Issue: Still rigid; doesn’t adapt well to natural instructions. Instruction Tuning 💡Train on human-written instruction–response pairs. ✅ Models now follow natural language instructions. ⚠️Issue: Quality depends on dataset; may misinterpret ambiguous prompts. Reinforcement Learning (RL) 💡Model learns via trial-and-error with a reward signal. ✅ Useful for sequential decision- making (games, robotics, dialog flow) ⚠️Issue: Hard to design good reward functions for open-ended tasks like conversation.
- 16. TRAINING & ALIGNMENTMECHANISMS — THE EVOLUTION Reinforcement Learning (RL) 💡Model learns via trial-and-error with a reward signal. ✅ Useful for sequential decision-making (games, robotics, dialog flow) ⚠️Issue: Hard to design good reward functions for open-ended tasks like conversation. RLHF -RL with Human Feedback 💡Replace hand-crafted reward with human feedback rankings ✅ More natural, human-preferred answers (helpful, polite, safe). ⚠️Issue: Expensive, time-consuming, biased toward annotator perspectives. DPO (Direct Preference Optimization) 💡Newer, simpler alternative to RLHF; optimizes preferences directly. ✅ Reduces complexity, improves stability. ⚠️Issue: Still evolving; not as widely deployed, potential for Biasing Constitutional AI (Anthropic) 💡Instead of humans labeling everything, use a set of written “principles” to guide training. ✅ More transparent, scalable alignment. ⚠️Issue: Depends on whose “constitution” is chosen → subjective and Biasing.
- 17. WHAT IS PROMPTENGINEERING?
- 18. NEXT-WORD PREDICTION EXAMPLE: 1 • Next-wordprediction is the core of how LLMs generate text: • Prompt A: • "The king lived in a large castle with high stone ____" • Prompt B: • "The king lived in a large palace with high stone ____" • 👉 In A, the model is highly likely to predict “walls”, “towers”, “gates”. 👉 In B, the model might lean toward “columns”, “arches”, “pillars”. Although castle and palace are synonyms in human language, they trigger different next-word distributions because of different contexts learned during training.
- 19. NEXT-WORD PREDICTION EXAMPLE: 2 • Next-wordprediction is the core of how LLMs generate text: • Prompt A: • "The scientist recorded the results in a notebook." • Prompt B: • "The scientist logged the results in a notebook." • 👉 In A, the model may predict follow-ups like “and analyzed them later”. 👉 In B, the word logged is more technical more likely to → yield continuations like “into the system” or “for the database”. Swapping recorded with logged shifts the domain context from casual note-taking to digital/technical workflows.
- 20. SO, WHY PROMPTENGINEERING? • Because LLMs are probabilistic the same model can give very different answers depending on how → you ask. • A well-crafted prompt acts like programming in natural language: • You set context (role, task). • You define constraints (length, format). • You steer style & tone. • Same as following Code standards: • Clean Code • DRY Code • Solid Principles
- 21. PROMPT PROCESSING EXAMPLE Prompt: “Summarizethe following paragraph in 1 sentence.” How the model handles it: 1. The model processes the prompt by breaking it into tokens → mapping them into embeddings → predicting the most likely next token step by step. 2. Uses self-attention to weigh important words
- 22. HOW A PROMPTIS PROCESSED IN AN LLM •You type the instruction + paragraph. •Example: “Summarize the following paragraph in 1 sentence.” 1. Prompt Input •Text is split into small units (tokens). •E.g. Summarize [Sum, marize], paragraph [para, graph]. → → 2. Tokenization •Each token is converted into a numeric vector. •Position is added so the model knows word order. 3. Embedding + Position Info •The model looks at all tokens and decides which words matter most. •Example: “Summarize…1 sentence” gets high attention guides the → output. 4. Self-Attention (Transformer magic ✨) •The model predicts the most likely next word, step by step. •Example: it might predict: The, then text, then explains…. 5. Prediction (Next Token) •Depending on settings (greedy, top-k, temperature), it picks words deterministically or creatively. 6. Sampling / Decoding • Tokens are converted back into text. •Final result: a concise, single-sentence summary. 7. Output
- 23. TYPES OF PROMPTS 1.Q&A Prompt Ask a direct question get an → answer. Prompt: 👉 “What is the capital of Japan?” Response: ➡️“Tokyo.” 2. Instruction-based Give a task get an action. → Prompt: 👉 “Summarize this paragraph in one sentence.” Response: ➡️“The text explains how renewable energy reduces environmental impact.” 3. Demonstration (Few- shot) Show examples model learns → the style and follows. Prompt: 👉 “Translate the following words into French: Dog Chien → Cat Chat → House ” → Response: ➡️“Maison” 4. Role based Assign the model a persona/role to shape tone & expertise. Prompt: 👉 “You are a financial advisor. Explain stock market basics to a beginner.” Response: ➡️“The stock market is a place where companies sell shares, and investors buy them…”
- 24.
- 25.
- 26. BIAS AND PERSPECTIVES Biascomes from: • Training corpora (dominant narratives). • Human feedback (RLHF reflect annotators’ perspectives). • Safety policies (over-cautious or selective filtering). No model is bias-free.
- 27. CASE STUDY B: GAZAWAR Neutral Question: • What are the main causes of the current war in Gaza? Multi-perspective Prompt: • Provide three perspectives: (1) Israeli government, (2) Palestinian voices (including Gaza), (3) Human rights organizations. Comparison Table Prompt: • Present these perspectives in a table with columns { Perspective, Key Points, Typical Sources }. ⚡ Reflection for audience: Even naming the conflict (“Gaza War,” “Israel–Palestine conflict,” “War on Gaza”) signals bias. Prompt engineering allows surfacing multiple perspectives side by side.
- 28. FUTURE DIRECTIONS IN PROMPT ENGINEERING LLMs mayevolve from “one-size-fits-all” to user-controlled alignment. • OpenAI Custom Instructions (2023): users set style and tone preferences. • Anthropic Constitutional AI (2022): guided by transparent rulesets. • Meta (2024): steerability research in LLaMA — tone control “dials.” • Stanford HAI (2023): multi-alignment research → choose profiles (“scholarly neutral,” “regional framing”).
- 29. THEN HOW CAN? ANDHOW FAR? THE CONVERSATIONS WITH HUMANS CONTRIBUTE TO THE TRAINING PROCESS OF CHATGPT?
- 30. FOUNDATION AND TRAININGSTAGES 🔹 1. Pretraining (the foundation stage) Here, models are trained on static datasets (web, books, Wikipedia, code, etc.). Human conversations are not directly included, unless they are publicly available (e.g., Reddit discussions, forums, OpenSubtitles). At this stage, your private ChatGPT conversations are not used. 🔹 2. Supervised Fine-Tuning Human trainers write demonstrations of good answers to prompts. Example: A trainer is given a question → they write the “ideal” response the → model learns from these. This creates the first alignment layer (teaching the model how we want it to behave). 🔹 3. RLHF (Reinforcement Learning from Human Feedback) Humans rank multiple model outputs for the same prompt. These rankings train a reward model, which guides reinforcement learning (e.g., PPO optimization). This step makes the model more helpful, harmless, and honest. Conversations with humans here are structured training data, not random chat logs.
- 31. CONTINUOUS IMPROVEMENT FROM USER INTERACTIONS •OpenAI, Anthropic, Google, etc. may use real user conversations to improve their models. • But: • They require explicit user opt-in (e.g., in ChatGPT, you can choose to let your data be used for training). • Conversations are typically sampled, anonymized, filtered, and used only to fine-tune or evaluate models. • They are not dumped raw into pretraining — instead, they help build new instruction-following datasets. Example flow: 1. You ask ChatGPT a question. 2. Human reviewers (contractors, alignment teams) may later see some anonymized samples. 3. They label “good/bad” responses. 4. These go into the reward model or future fine-tuning sets. • Facebook Bot as an example.
- 32. HOW FAR DOESTHIS HELP? •Strengths added by human conversations: •Models learn to follow natural instructions better. •Tone, politeness, and empathy improve. •Safety filters get stronger (detecting harmful requests). •Domain-specific improvements (e.g., if users often ask medical- style questions, fine-tuning can adapt). •Limitations: •Scale is small compared to pretraining (billions of tokens vs. trillions). •Human feedback is expensive (labelers cost time & money). •It doesn’t give new knowledge, only alignment (e.g., it won’t teach new facts about 2025 unless connected to updated corpora).
- 33. THEN ANSWER IS: Humanconversations don’t build the base knowledge of ChatGPT (that comes from huge corpora). They do shape behavior — making the model more aligned, polite, safe, and useful through SFT, RLHF, and opt-in user feedback. The influence is qualitative, not quantitative: a relatively small number of curated human interactions can strongly change how the model responds.
- 34. CONCLUSION & KEY TAKEAWAYS Prompts aremore than questions → they are instructions that shape AI behavior. LLMs reflect human data & biases → always ask critically. Prompt engineering is experimental try Q&A, instructions, → demonstrations, roles. Future AI will be user-steerable → alignment as a setting, not a fixed rule. 👉 Final thought: Maybe prompts are not just questions… maybe they are the new programming language of intelligence.
Editor's Notes
- #9 Start from minute 15 to 17 Then 26:30 to 28:30
- #10 There were two major "winters" approximately 1974–1980 and 1987–2000,[3] and several smaller episodes, including the following: 1966: failure of machine translation 1969: criticism of perceptrons (early, single-layer artificial neural networks) 1971–75: DARPA's frustration with the Speech Understanding Research program at Carnegie Mellon University 1973: large decrease in AI research in the United Kingdom in response to the Lighthill report 1973–74: DARPA's cutbacks to academic AI research in general 1987: collapse of the LISP machine market 1988: cancellation of new spending on AI by the Strategic Computing Initiative 1990s: many expert systems were abandoned 1990s: end of the Fifth Generation computer project's original goals Source: https://en.wikipedia.org/wiki/AI_winter
- #11 Neural Networks – History: The 1940's to the 1970's . In 1943, neurophysiologist Warren McCulloch and mathematician Walter Pitts wrote a paper on how neurons might work. In order to describe how neurons in the brain might work, they modeled a simple neural network using electrical circuits. The first multilayered network was developed in 1975, an unsupervised network. Source: https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/History/history1.html#:~:text=Neural%20Networks%20%2D%20History&text=In%201943%2C%20neurophysiologist%20Warren%20McCulloch,neural%20network%20using%20electrical%20circuits.
- #12 There were two major "winters" approximately 1974–1980 and 1987–2000,[3] and several smaller episodes, including the following: 1966: failure of machine translation 1969: criticism of perceptrons (early, single-layer artificial neural networks) 1971–75: DARPA's frustration with the Speech Understanding Research program at Carnegie Mellon University 1973: large decrease in AI research in the United Kingdom in response to the Lighthill report 1973–74: DARPA's cutbacks to academic AI research in general 1987: collapse of the LISP machine market 1988: cancellation of new spending on AI by the Strategic Computing Initiative 1990s: many expert systems were abandoned 1990s: end of the Fifth Generation computer project's original goals Source: https://en.wikipedia.org/wiki/AI_winter
- #15 GPT stands for Generative Pre-Trained Transformers
- #17 Sample of next word prediction: next-word prediction is the core of how LLMs generate text
- #22 GPT Stands for Generative Pre-Trained Transformers