https://openreview.net/forum?id=xPWZTvdobm

Quality Diversity:

• Search a space of policies/programs/agents for diverse behaviour.

• Diversity measured on basis of:

  • Behaviour: e.g. which limbs are used by the robot.
  • Quality: e.g. How far does the robot walk.

• See MAP-Elites:

  TL;DR: create a solution → categorise on the basis of behaviour:

  if behaviour seen before:

  archive

  else:

  check quality (score)

  if quality > neighbour (past agent with similar behaviour):

  replace neighbour with new agent

Self-Play:

PROS:

• curriculum generation

CONS:

• diversity problem

IDEA:

• Use self-play’s training prowess and add quality-diversity to correct its diversity problem.
• Use LLMs as search operators and as a source for human priors and knowledge

Evaluation:

• Homicidal Chauffeur → 2 body evader-pursuer game with finite discrete-time system
• LLM used: GPT-4o