Do text embeddings perfectly encode text?
‘Vec2text’ can serve as a solution for accurately reverting embeddings back into text, thus highlighting the urgent need for revisiting security protocols around embedded data.
Interpretability, llm, nlp
llm
data-science, deep-learning, llm, Machine Learning
Research Papers in February 2024
Once again, this has been an exciting month in AI research. This month, I’m covering two new openly available LLMs, insights into small finetuned LLMs, and…
data-science, deep-learning, llm, Machine Learning
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Low-rank adaptation (LoRA) is a machine learning technique that modifies a pretrained model (for example, an LLM or vision transformer) to better suit a…
data-science, deep-learning, llm, Machine Learning
Optimizing LLMs From a Dataset Perspective
This article focuses on improving the modeling performance of LLMs by finetuning them using carefully curated datasets. Specifically, this article…
data-science, deep-learning, llm, Machine Learning
The NeurIPS 2023 LLM Efficiency Challenge Starter Guide
Large language models (LLMs) offer one of the most interesting opportunities for developing more efficient training methods. A few weeks ago, the NeurIPS…
data-science, deep-learning, llm, Machine Learning
Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch
Peak memory consumption is a common bottleneck when training deep learning models such as vision transformers and LLMs. This article provides a series of…
data-science, deep-learning, llm, Machine Learning
Finetuning Falcon LLMs More Efficiently With LoRA and Adapters
Finetuning allows us to adapt pretrained LLMs in a cost-efficient manner. But which method should we use? This article compares different…
large-language-models, llm, PaLM API, recommendation systems, TensorFlow Recommenders
Augmenting recommendation systems with LLMs
Posted by Wei Wei, Developer Advocate
Large language models (LLMs) are taking the world by storm, thanks to their powerful ability to generate text, translate languages, and answer questions in a coherent and informative way. At Google I/O 202…
data-science, deep-learning, llm, Machine Learning
Accelerating Large Language Models with Mixed-Precision Techniques
Training and using large language models (LLMs) is expensive due to their large compute requirements and memory footprints. This article will explore how…
data-science, deep-learning, llm, Machine Learning
Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA)
Pretrained large language models are often referred to as foundation models for a good reason: they perform well on various tasks, and we can use them as a…