791: Reinforcement Learning from Human Feedback (RLHF) — with Dr. Nathan Lambert
#ReinforcementLearning #RLHF #GenerativeAI
Reinforcement learning through human feedback (RLHF) has come a long way. In this episode, research scientist Nathan Lambert talks to @JonKrohnLearns about the technique’s origins of the technique. He also walks through other ways to fine-tune LLMs, and how he believes generative AI might democratize education.
This episode is brought to you by AWS Inferentia (https://go.aws/3zWS0au) and AWS Trainium (https://go.aws/3ycV6K0), and Crawlbase (https://crawlbase.com), the ultimate data crawling platform. Interested in sponsoring a SuperDataScience Podcast episode? Visit https://passionfroot.me/superdatascience for sponsorship information.
In this episode you will learn:
• [00:00:00] Introduction
• [00:01:52] Why it is important that AI is open
• [00:06:11] The efficacy and scalability of direct preference optimization
• [00:13:12] Robotics and LLMs
• [00:21:36] The challenges to aligning reward models with human preferences
• [00:27:35] How to make sure AI’s decision making on preferences reflect desirable behavior
• [00:36:27] Why Nathan believes AI is closer to alchemy than science
Additional materials: https://www.superdatascience.com/791
#ReinforcementLearning #RLHF #GenerativeAI
Reinforcement learning through human feedback (RLHF) has come a long way. In this episode, research scientist Nathan Lambert talks to @JonKrohnLearns about the technique’s origins of the technique. He also walks through other ways to fine-tune LLMs, and how he believes generative AI might democratize education.
This episode is brought to you by AWS Inferentia (https://go.aws/3zWS0au) and AWS Trainium (https://go.aws/3ycV6K0), and Crawlbase (https://crawlbase.com), the ultimate data crawling platform. Interested in sponsoring a SuperDataScience Podcast episode? Visit https://passionfroot.me/superdatascience for sponsorship information.
In this episode you will learn:
• [00:00:00] Introduction
• [00:01:52] Why it is important that AI is open
• [00:06:11] The efficacy and scalability of direct preference optimization
• [00:13:12] Robotics and LLMs
• [00:21:36] The challenges to aligning reward models with human preferences
• [00:27:35] How to make sure AI’s decision making on preferences reflect desirable behavior
• [00:36:27] Why Nathan believes AI is closer to alchemy than science
Additional materials: https://www.superdatascience.com/791
