question → how can RLHF deal with possibly conflicting feedback coming from multiple people? (4)

Moscow analysis of the project plan → what is really important; keep it small

start writing already

maybe it's possible to continue during summer if research is promising

helpful docs to do research: https://www.academic-skills.org/

How do we evaluate things? Be careful with 10 weeks

submit possible must have by Thursday night or Friday morning (informal feedback loop)

Plan:

• 2 weeks → literature review + start coding + write introduction and literature review
• 3 weeks → developing method
• 2 weeks → analysis
• 1 week → writing conclusion and discussion
• 1 week → paper

look into frameworks for RLHF → imitation library has an implementation (imitation.readthedocs.io/en/latest/algorithms/preference_comparisons.html) → approach we pick will affect how much knowledge/control we need to have

grids: https://mo-gymnasium.farama.org/environments/grid-world/

The supervisors will also for access to DAIC: https://doc.daic.tudelft.nl/ → be careful as it may have a steep learning curve

we can send the draft of the paper before week 7, but it needs to be specific

use IEEE citation style (they prefer numbers)

give short presentation of research plan $\backsim$ 4 minutes presentation → have some structure to present things

they will make research channel