Projects

VIPCGRL: Human-Aligned Procedural Level Generation

Motivation

Game level design is a creative process where human designers express intent through spatial structures, sketches, and natural language descriptions. Existing AI-driven procedural content generation (PCGRL) systems accept only scalar conditions, missing the rich communicative modalities humans naturally use. This project asks: can an RL agent be trained to generate levels that look and feel like they were made by a human designer, while accepting flexible multi-modal instructions?

Issues

  • Prior PCGRL methods (CPCGRL, IPCGRL) are limited to scalar or text-only inputs.
  • No existing system supports sketch-based level control — a common tool for game designers.
  • Generated levels lack human-likeness even when conditions are satisfied numerically.
  • Aligning an RL policy with human aesthetic preferences is an open problem.

Method

Two-phase training:

  1. Quadruple Contrastive Learning — A shared 64-dimensional embedding space is trained to align text descriptions, level images, user sketches, and human/AI style signals via a multi-positive InfoNCE objective.

  2. Human-Aligned DRL — A PPO policy conditioned on the shared embedding receives an auxiliary reward based on the change in cosine similarity to the same human reference before and after a level edit, encouraging the policy to produce levels closer to human style.

The sketch input pipeline uses edge detection + spline interpolation to convert levels to 224×224 grayscale sketches, enabling zero-shot cross-modal generalization at inference time.

My Contribution

  • Contributed to the main model design and architecture proposals.
  • Designed the loss function for multimodal representation learning.
  • Implemented the CLIP-based representation-learning model, dataset, sampler and training code.
  • Integrated the CLIP encoder into the reinforcement-learning training code and implemented precomputation of text embeddings for reuse during training.

Research Results

  • Quantitative evaluation: improved generated-level human-likeness metrics (TPKL-Div and ViT-Sim) compared with CPCGRL and IPCGRL.
  • User evaluation: in a study with 26 participants, generated levels received higher human-likeness ratings than CPCGRL and IPCGRL, and higher text-instruction alignment ratings than IPCGRL.
  • Cross-modal inputs: the reinforcement-learning policy trained with text instructions accepted level images and sketches without additional policy training. The shared encoder had been trained with level images and sketches during representation learning.
  • This research extends the M.S. thesis (HL-PCGRL, GIST 2025).

Publication Updates

VIPCGRL is published in IEEE Transactions on Games, Early Access (2026). Publication DOI · arXiv · Code.

The related MIPCGRL paper was accepted to IEEE CoG 2026 and selected as a Best Paper Award Nominee. It is listed as regular paper #217 and scheduled for a 20-minute presentation in Best of CoG on 2 September 2026. Conference · Schedule · arXiv.