👨💻 to get started with AI engineering, check out this Scrimba course: https://scrimba.com/the-ai-engineer-path-c02v?via=yacineMahdid
How does In-context Learning work in LLMs (big and small)?
# Table of Content
– Introduction: 0:00
– Definition of In-Context Learning: 0:59
– Why is In-Context Learning Surprising?: 2:10
– Bayesian Framework for In-Context Learning: 3:13
– LLMs are Implementing Learning Algorithms for In-Context Learning: 7:59
– Conclusion: 10:55
Cool Resource:
https://thegradient.pub/in-context-learning-in-context/
http://ai.stanford.edu/blog/understanding-incontext/
Interesting Papers:
– Language Models are Few-Shot Learners: https://arxiv.org/pdf/2005.14165
– An Explanation of In-context Learning as Implicit Bayesian Inference: https://arxiv.org/abs/2111.02080
– Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? https://arxiv.org/abs/2202.12837
– In-context Learning and Induction Heads: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
– Transformers learn in-context by gradient descent https://arxiv.org/abs/2212.07677
– Larger language models do in-context learning differently : https://research.google/blog/larger-language-models-do-in-context-learning-differently/
Really interesting topic! Hit me up in the comments if you want to discuss this in depth!
—-
Join the newsletter for weekly AI content: https://yacinemahdid.com
Join the Discord for general discussion: https://discord.gg/QpkxRbQBpf
—-
Follow Me Online Here:
Twitter: https://twitter.com/yacineaxya
GitHub: https://github.com/yacineMahdid
LinkedIn: https://www.linkedin.com/in/yacinemahdid/
___
Have a great week! 👋
source
Leave a Reply