Lil'Log
- Harness Engineering for Self-ImprovementLil'Log ·
The concept of recursive self-improvement (RSI) dates back to I. J.
- Scaling Laws, CarefullyLil'Log ·
Scaling laws are one of the most critical empirical findings in deep learning.
- Why We ThinkLil'Log ·
Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post. Test time compute ( Graves et al. 2016 , Ling, et al. 2017 , Cobbe et al.
- Reward Hacking in Reinforcement LearningLil'Log ·
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task.
- Extrinsic Hallucinations in LLMsLil'Log ·
Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content.
- Diffusion Models for Video GenerationLil'Log ·
Diffusion models have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation.
- Thinking about High-Quality Human DataLil'Log ·
[Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “Vox populi”) and nice feedback.
- Adversarial Attacks on LLMsLil'Log ·
The use of large language models in the real world has strongly accelerated by the launch of ChatGPT.
- LLM Powered Autonomous AgentsLil'Log ·
Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT , GPT-Engineer and BabyAGI , serve as inspiring examples.
- Prompt EngineeringLil'Log ·
also known as In-Context Prompting , refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights.
- The Transformer Family Version 2.0Lil'Log ·
Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago.
- Large Transformer Model Inference OptimizationLil'Log ·
[Updated on 2023-01-24: add a small section on Distillation .] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks.
- Some Math behind Neural Tangent KernelLil'Log ·
Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset.
- Generalized Visual Language ModelsLil'Log ·
Processing images to generate text, such as image captioning and visual question-answering, has been studied for years.
- Learning with not Enough Data Part 3: Data GenerationLil'Log ·
Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2 ). Let’s consider two approaches for generating synthetic data for training. Augmented data .
- Learning with not Enough Data Part 2: Active LearningLil'Log ·
This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks.
- Learning with not Enough Data Part 1: Semi-Supervised LearningLil'Log ·
When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.
- How to Train Really Large Models on Many GPUs?Lil'Log ·
[Updated on 2022-03-13: add expert choice routing .]
- What are Diffusion Models?Lil'Log ·
[Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)].
- Contrastive Representation LearningLil'Log ·
The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart.
- Reducing Toxicity in Language ModelsLil'Log ·
Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet.
- Controllable Neural Text GenerationLil'Log ·
[Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.] [Updated on 2021-05-26: Add P-tuning and Prompt Tuning in the “prompt design” section.]
- How to Build an Open-Domain Question Answering System?Lil'Log ·
[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta).
- Neural Architecture SearchLil'Log ·
Although most popular and successful model architectures are designed by human experts, it doesn’t mean we have explored the entire network architecture space and settled down with the best option.
- Exploration Strategies in Deep Reinforcement LearningLil'Log ·
[Updated on 2020-06-17: Add “exploration via disagreement” in the “Forward Dynamics” section . Exploitation versus exploration is a critical topic in Reinforcement Learning.
- The Transformer FamilyLil'Log ·
[Updated on 2023-01-27 : After almost three years, I did a big refactoring update of this post to incorporate a bunch of new Transformer models since 2020.
- Curriculum for Reinforcement LearningLil'Log ·
[Updated on 2020-02-03: mentioning PCG in the “Task-Specific Curriculum” section. [Updated on 2020-02-04: Add a new “curriculum through distillation” section.
- Self-Supervised Representation LearningLil'Log ·
[Updated on 2020-01-09: add a new section on Contrastive Predictive Coding ]. [Updated on 2020-04-13: add a “Momentum Contrast” section on MoCo, SimCLR and CURL.]
- Evolution StrategiesLil'Log ·
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option.
- Meta Reinforcement LearningLil'Log ·
In my earlier post on meta-learning , the problem is mainly defined in the context of few-shot classification.
- Domain Randomization for Sim2Real TransferLil'Log ·
In Robotics, one of the hardest problems is how to make your model transfer to the real world.
- Are Deep Neural Networks Dramatically Overfitted?Lil'Log ·
[Updated on 2019-05-27: add the section on Lottery Ticket Hypothesis.]
- Generalized Language ModelsLil'Log ·
[Updated on 2019-02-14: add ULMFiT and GPT-2 .] [Updated on 2020-02-29: add ALBERT .] [Updated on 2020-10-25: add RoBERTa .] [Updated on 2020-12-13: add T5 .] [Updated on 2020-12-30: add GPT-3 .]
- Object Detection Part 4: Fast Detection ModelsLil'Log ·
In Part 3 , we have reviewed models in the R-CNN family. All of them are region-based object detection algorithms.
- Meta-Learning: Learning to Learn FastLil'Log ·
[Updated on 2019-10-01: thanks to Tianhao, we have this post translated in Chinese !]
- Flow-based Deep Generative ModelsLil'Log ·
So far, I’ve written about two types of generative models, GAN and VAE .
- From Autoencoder to Beta-VAELil'Log ·
[Updated on 2019-07-18: add a section on VQ-VAE & VQ-VAE-2 .] [Updated on 2019-07-26: add a section on TD-VAE .]
- Attention? Attention!Lil'Log ·
[Updated on 2018-10-28: Add Pointer Network and the link to my implementation of Transformer.] [Updated on 2018-11-06: Add a link to the implementation of Transformer model.]
- Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI GymLil'Log ·
The full implementation is available in lilianweng/deep-reinforcement-learning-gym In the previous two posts, I have introduced the algorithms of many deep reinforcement learning models.
- Policy Gradient AlgorithmsLil'Log ·
[Updated on 2018-06-30: add two new policy gradient methods, SAC and D4PG .] [Updated on 2018-09-30: add a new policy gradient method, TD3 .]
- A (Long) Peek into Reinforcement LearningLil'Log ·
[Updated on 2020-09-03: Updated the algorithm of SARSA and Q-learning so that the difference is more pronounced. [Updated on 2021-09-19: Thanks to 爱吃猫的鱼, we have this post in Chinese ].
- The Multi-Armed Bandit Problem and Its SolutionsLil'Log ·
The algorithms are implemented for Bernoulli bandit in lilianweng/multi-armed-bandit . Exploitation vs Exploration The exploration vs exploitation dilemma exists in many aspects of our life.
- Object Detection for Dummies Part 3: R-CNN FamilyLil'Log ·
[Updated on 2018-12-20: Remove YOLO here. Part 4 will cover multiple fast object detection algorithms, including YOLO.] [Updated on 2018-12-27: Add bbox regression and tricks sections for R-CNN.]
- Object Detection for Dummies Part 2: CNN, DPM and OverfeatLil'Log ·
Part 1 of the “Object Detection for Dummies” series introduced: (1) the concept of image gradient vector and how HOG algorithm summarizes the information across all the gradient vectors in one image; (2) how the image segmentation algorithm works to detect regions that potentially contain objects; (3) how the Selective Search algorithm refines the outcomes of image segmentation for better region proposal.
- Object Detection for Dummies Part 1: Gradient Vector, HOG, and SSLil'Log ·
I’ve never worked in the field of computer vision and has no idea how the magic could work when an autonomous car is configured to tell apart a stop sign from a pedestrian in a red hat.
- Learning Word EmbeddingLil'Log ·
Human vocabulary comes in free text. In order to make a machine learning model understand and process the natural language, we need to transform the free-text words into numeric values.
- Anatomize Deep Learning with Information TheoryLil'Log ·
Professor Naftali Tishby passed away in 2021. Hope the post can introduce his cool idea of information bottleneck to more people.
- From GAN to WGANLil'Log ·
[Updated on 2018-09-30: thanks to Yoonju, we have this post translated in Korean !] [Updated on 2019-04-18: this post is also available on arXiv .]
- How to Explain the Prediction of a Machine Learning Model?Lil'Log ·
The machine learning models have started penetrating into critical areas like health care, justice systems, and financial industry.
- Predict Stock Prices Using RNN: Part 2Lil'Log ·
In the Part 2 tutorial, I would like to continue the topic on stock price prediction and to endow the recurrent neural network that I have built in Part 1 with the capability of responding to multiple stocks.