Building A GPT-Style LLM Classifier From Scratch
This article shows you how to transform pretrained large language models (LLMs) into strong text classifiers. But why focus on classification? First…
ai, deep-learning, llm, Machine Learning
deep-learning
ai, deep-learning, llm, Machine Learning
Building LLMs from the Ground Up: A 3-hour Coding Workshop
This tutorial is aimed at coders interested in understanding the building blocks of large language models (LLMs), how LLMs work, and how to code them from…
ai, deep-learning, llm, Machine Learning
New LLM Pre-training and Post-training Paradigms
There are hundreds of LLM papers each month proposing new techniques and approaches. However, one of the best ways to see what actually works well in…
ai, deep-learning, llm, Machine Learning
Instruction Pretraining LLMs
This article covers a new, cost-effective method for generating data for instruction finetuning LLMs; instruction finetuning from scratch; pretraining LLMs…
ai, deep-learning, llm, Machine Learning
LLM Research Insights: Instruction Masking and New LoRA Finetuning Experiments?
This article covers three new papers related to instruction finetuning and parameter-efficient finetuning with LoRA in large language models (LLMs). I work…
ai, deep-learning, llm, Machine Learning
Developing an LLM: Building, Training, Finetuning
This is an overview of the LLM development process. This one-hour talk focuses on the essential three stages of developing an LLM: coding the architecture…
ai, deep-learning, llm, Machine Learning
How Good Are the Latest Open LLMs? And Is DPO Better Than PPO?
What a month! We had four major open LLM releases: Mixtral, Meta AI’s Llama 3, Microsoft’s Phi-3, and Apple’s OpenELM. In my new article, I review and…
ai, deep-learning, llm, Machine Learning
Using and Finetuning Pretrained Transformers
What are the different ways to use and finetune pretrained large language models (LLMs)? The three most common ways to use and finetune pretrained LLMs…
data-science, deep-learning, llm, Machine Learning
Tips for LLM Pretraining and Evaluating Reward Models
It’s another month in AI research, and it’s hard to pick favorites. This month, I am going over a paper that discusses strategies for the continued…
deep-learning, Overviews, reinforcement-learning
Mamba Explained
Is Attention all you need? Mamba, a novel AI model based on State Space Models (SSMs), emerges as a formidable alternative to the widely used Transformer models, addressing their inefficiency in processing long sequences.