Schedule

Note: topics and deadlines are subject to change.

Date		Topic	Assignments
01/17	W	Introduction to Robot Learning
01/19	F	Intro to Learning from Demonstration
[Lecture 1] Argall, Chernova, Veloso, & Browning. (2009). "A survey of robot learning from demonstration." Robotics and Autonomous Systems, 57(5): 469-483. [Lecture 2] Amershi, Cakmak, Knox, & Kulesza. (2014). "Power to the people: The role of humans in interactive machine learning." AI Magazine, 35(4): 105-120.
Module 1: Policy & Reward Learning
01/22	M	Learning from Rewards 1
[Lecture 1] Kober, Bagnell, & Peters. (2013). "Reinforcement learning in robotics: A survey." IJRR 32.11: 1238-1274. [Lecture 2] Thomaz & Breazeal. "Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance." AAAI 2006.
01/24	W	Learning from Rewards 2
[Lecture] Knox & Stone. "Interactively shaping agents via human reinforcement: The TAMER framework." International Conference on Knowledge Capture. 2009.
01/29	M	Apprenticeship Learning
[Lecture] Abbeel & Ng. "Apprenticeship learning via inverse reinforcement learning." ICML 2004. [Neil Sachdeva] Kolter, Abbeel, & Ng. "Hierarchical apprenticeship learning with application to quadruped locomotion." NeurIPS 2007.
01/31	W	Bayesian IRL
[Lecture] Ramachandran & Amir. "Bayesian Inverse Reinforcement Learning." IJCAI 2007. [Ulas] Palan et al. "Learning reward functions by integrating human demonstrations and preferences." RSS 2019.
02/05	M	Learning from Preferences
[James] Brown et al. "Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations." ICML 2019. [Liam] Lee, Smith, & Abbeel. "Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre-training." ICML 2021.
Module 2: Bottom-Up Learning
02/07	W	Learning Task Goals and Structure 1	HW 1 Released Project Idea Due
[Zubin] Bajcsy et al. "Learning robot objectives from physical human interaction." CoRL 2017. [Chuhan] Akgun & Thomaz. (2016). "Simultaneously learning actions and goals from demonstration." Autonomous Robots 40.2: 211-227.
02/12	M	Learning Task Goals and Structure 2
[Liam] Niekum et al. (2015). "Learning grounded finite-state representations from unstructured demonstrations." IJRR 34.2: 131-157. [Peter] Konidaris et al. (2012). "Robot learning from demonstration by constructing skill trees." IJRR 31.3: 360-375.
02/14	W	Probabilistic Models
[Fern] Chernova & Veloso. "Confidence-based policy learning from demonstration using gaussian mixture models." AAMAS 2007. [Fengyu] Maeda et al. "Active incremental learning of robot movement primitives." CoRL 2017.
02/19	M	Class Cancelled
02/21	W	Dynamic Movement Primitives	Project 1 Due
[Lecture] Schaal. (2006). "Dynamic movement primitives-a framework for motor control in humans and humanoid robotics." Adaptive Motion of Animals and Machines. 261-280. [Hrishi] Pastor et al. "Learning and generalization of motor skills by learning from demonstration." ICRA 2009.
02/26	M	Generative Adversarial Imitation Learning
[Neil Song] Li, Song, & Ermon. "InfoGAIL: Interpretable imitation learning from visual demonstrations." NeurIPS 2017. [Shuangge] Wang et al. "Co-GAIL: Learning diverse strategies for human-robot collaboration." CoRL 2022.
Module 3: Top-Down Learning
02/28	W	Grounding High-level Tasks 1
[Neil Sachdeva] Calinon, Guenter, & Billard. (2007). "On learning, representing, and generalizing a task in a humanoid robot." IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37.2: 286-298. [Fern] Bullard et al. "Grounding action parameters from demonstration." RO-MAN 2016.
03/04	M	Grounding High-level Tasks 2	Project 2 Draft Due
[Neil Sachdeva] Calinon, Guenter, & Billard. (2007). "On learning, representing, and generalizing a task in a humanoid robot." IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37.2: 286-298. [Jacky] Tenorth, Nyga, & Beetz. "Understanding and executing instructions for everyday manipulation tasks from the world wide web." ICRA 2010.
03/06	W	Grounding Language 1
[Ziyao] Brohan et al. "Do as I can, not as I say: Grounding language in robotic affordances." CoRL 2023. [Chuhan] Nasiriany, et al. "PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs." arXiv preprint arXiv:2402.07872 (2024).
–Spring Break–
03/25	M	Grounding Language 2	HW 1 Due
[Ethan] Kulick et al. "Active Learning for Teaching a Robot Grounded Relational Symbols." IJCAI 2013. [Ulas] Peng et al. "Preference-Conditioned Language-Guided Abstraction." HRI 2024.
03/27	W	Grounding Vision	Project 2 Due
[Neil Song] Shi et al. "RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools." CoRL 2023. [James] Zhang et al. "A Universal Semantic-Geometric Representation for Robotic Manipulation." CoRL 2023.
Module 4: Human Factors
04/01	M	Active Learning 1	HW 2 Released
[Zubin] Cui et al. "Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning." IJCAI 2021. [Jacky] Bıyık et al. "Asking easy questions: A user-friendly approach to active reward learning." CoRL 2020.
04/03	W	Active Learning 2
[Peter] Balsells et al. "Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback." CoRL 2023. [Anushka] Racca, Oulasvirta, & Kyrki. "Teacher-aware active robot learning." HRI 2019.
04/08	M	Human Factors 1
[Eason] Rigter, Lacerda, & Hawes. (2020). "A framework for learning from demonstration with minimal human effort." IEEE Robotics and Automation Letters 5.2: 2023-2030. [Hrishi] Ke et al. "Telemanipulation with chopsticks: Analyzing human factors in user demonstrations." IROS 2020.
04/10	W	Human Factors 2
[Ethan] Hedlund, Johnson, & Gombolay. "The Effects of a Robot's Performance on Human Teachers for Learning from Demonstration Tasks." HRI 2021. [Shuangge] Krening & Feigh. (2018). "Interaction algorithm effect on human experience with reinforcement learning." THRI 7.2: 1-22.
04/15	M	Implicit Signals 1	HW 2 Due HW 3 Released
[Anushka] Zhang et al. "Human gaze assisted artificial intelligence: A review." IJCAI 2020. [Fengyu] Aronson & Admoni. "Gaze Complements Control Input for Goal Prediction During Assisted Teleoperation." RSS 2022.
04/17	W	Implicit Signals 2
[Ziyao] Cui et al. "The empathic framework for task learning from implicit human feedback." CoRL 2020. [Eason] Cederborg et al. "Policy shaping with human teachers." IJCAI 2015.
Module 5: Project Presentations
04/22	M	Project Peer-Review 1	Project 3 Draft Due (before class)
04/24	W	Project Peer-Review 2	Project 3 Due
[Anushka] Zhang et al. "Human gaze assisted artificial intelligence: A review." IJCAI 2020.
–Reading Period–
05/01	W	No class	HW 3 Due
05/08	M	No class	Project 4 Due