Ray Luo

Ray Luo

Ph.D. Candidate

Mila, McGill University

Biography

Hi there! I’m Ziyan “Ray” Luo, a Ph.D. candidate at Mila, McGill. I’m so lucky to work with (and learn from) Dr. Xujie Si, Dr. Doina Precup, and my talented colleagues!

My research interest mainly lies at the intersection of Reinforcement Learning and Abstraction. I’m eager to find a balance point between formal methods (strict and delicate) and machine learning approaches (more “magical” but general).

For RL, I’m trying to understand the underlying principles of abstraction techniques (e.g., state, action, temporal abstraction; behavior metric learning), and how those abstractions benefit sample efficiency, generalization and planning. For formal methods, I’m interested in understanding abstractions in formal verification problems (e.g., solving Constrained Horn Clauses). It also potentially poses important challenges for RL in dealing with “logical-intensive” tasks and combinatorial spaces.

I was once a research intern at Microsoft Research Asia (System Research Group, Bing), and a research assistant at Tsinghua University.

Have fun with me - I’m a music enthusiast who loves to create various genres of “storytelling-styled” music, here is my music portfolio (you may also find them in popular rhythm games). Using a mixture of electronic genres and acoustic instruments, I create music that impresses people and conveys intended themes. I also enjoy ball sports like tennis, table tennis, badminton, and billiards! Besides, I love animals (especially but not limited to cats - Instagram)!

Interests

  • Reinforcement Learning
  • Abstraction
  • Formal Verification

Education

  • Ph.D. in Computer Science, 2021

    Mila, McGill University

Experience

 
 
 
 
 

Ph.D. Candidate

Mila, McGill University

Sep 2021 – Present Montreal, Quebec
  • It’s my honor to have the chance to work with Prof. Xujie Si in “REAP” group, and Prof. Doina Precup in Reasoning and Learning Lab!
  • TA for COMP 579 - Reinforcement Learning (Instructors: Doina Precup, Isabeau Prémont-Schwarz), McGill TA Award Winner
  • Reviewer - NeurIPS'24
  • Reviewer - TMLR
  • Reviewer and Volunteer - International Conference on Computer Aided Verification (CAV'24)
  • Reviewer - NeurIPS'23 MathAI workshop
 
 
 
 
 

Research Intern

Microsoft Research Asia

Nov 2019 – Mar 2020 SRG, MSRA, Beijing, China

Match Plan Generation in Web Search with Parameterized Action Reinforcement Learning

  • Adapting DRL algorithms for Inverted Index Match Plan Generation to replace current hand-crafted heuristics at the L1 stage of the query process in Bing
  • Collaborating with the team of Bing at Systems and Networking Research Group (SRG)
 
 
 
 
 

Research Assistant

Tsinghua University

Mar 2019 – Jan 2021 Beijing

Spatio-temporal Trajectory Prediction

  • Supervised by Jilong Wang (chair of APAN) and Congcong Miao
  • ACN: accepted by WSDM'20 (the top 7% oral paper)
  • RLMob: accepted by WSDM'22

Projects

Ether Forum System

Independently designed forum system with C++/QT, fulfilling all basic functions with elegant interface and user-friendly interaction

Mini Compiler from Pascal to C

Conducted grammar design, lexical analysis, syntactic analysis, semantic analysis, and the generation of target code using bottom-up techniques of S attribute