Ray Luo

Ray Luo

Ph.D. Candidate

Mila, McGill University


Hi there! I’m Ziyan “Ray” Luo, a Ph.D. candidate at Mila, McGill. I’m so lucky to work with (and learn from) Dr. Xujie Si, Dr. Doina Precup, and my talented colleagues!

My research interest mainly lies at the intersection of Reinforcement Learning and Abstraction. I’m eager to find a balance point between formal methods (strict and delicate) and machine learning approaches (more “magical” but general).

For RL, I’m trying to understand the underlying principles of abstraction techniques (e.g., state, action, temporal abstraction), and how those abstractions benefit learning and planning. For formal methods, I’m interested in understanding abstractions in formal verification problems (e.g., solving Constrained Horn Clauses). It also potentially poses important challenges for RL in dealing with “logical-intensive” tasks and combinatorial spaces.

I was once a research intern at Microsoft Research Asia (System Research Group, Bing), and a research assistant at Tsinghua University.

Have fun with me - I’m a music enthusiast who loves to create various genres of “storytelling-styled” music, here is my music portfolio (you may also find them in popular rhythm games). Using a mixture of electronic genres and acoustic instruments, I create music that impresses people and conveys intended themes. I also enjoy ball sports like tennis, table tennis, badminton, and billiards! Besides, I love animals (especially but not limited to cats - Instagram)!


  • Reinforcement Learning
  • Abstraction
  • Formal Verification


  • Ph.D. in Computer Science, 2021

    Mila, McGill University



Ph.D. Candidate

Mila, McGill University

Sep 2021 – Present Montreal, Quebec

Research Intern

Microsoft Research Asia

Nov 2019 – Mar 2020 SRG, MSRA, Beijing, China

Match Plan Generation in Web Search with Parameterized Action Reinforcement Learning

  • Adapting DRL algorithms for Inverted Index Match Plan Generation to replace current hand-crafted heuristics at the L1 stage of the query process in Bing
  • Collaborating with the team of Bing at Systems and Networking Research Group (SRG)

Research Assistant

Tsinghua University

Mar 2019 – Present Beijing

Spatio-temporal Trajectory Prediction

  • Supervised by Jilong Wang (chair of APAN) and Congcong Miao
  • ACN: accepted by WSDM'20 (the top 7% oral paper)
  • RLMob: accepted by WSDM'22


Ether Forum System

Independently designed forum system with C++/QT, fulfilling all basic functions with elegant interface and user-friendly interaction

Mini Compiler from Pascal to C

Conducted grammar design, lexical analysis, syntactic analysis, semantic analysis, and the generation of target code using bottom-up techniques of S attribute