I am a Research Scientist at Apple AI/ML working with Jeff Nichols. My research focuses on developing multi-modal LLMs and their UI and agentic applications. Previously I was a Research Scientist at Google Research working on generative models for creative and design applications. I am based in Hong Kong | Mountain View, CA.
I recently graduated with a Ph.D. in Computer Science from University of California, Berkeley advised by Prof. John Canny.
12/11/2023 I recently joined Apple AI/ML, working with Jeff Nichols starting Winter 2023.
07/25/2022 I graduated from UC Berkeley with a Ph.D. in EECS! I am excited to be joining Google Research working with Yang Li starting Fall 2022.
07/27/2022 Our Paper Enabling Multi-modal Search for Inspirational Design Stimuli using Deep Learning is published in the AI EDAM Journal.
04/30/2022 Our Workshop Paper Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models is published at the Computational UI Workshop at CHI 2022.
11/20/2019 I am excited to receive an Honourable Mention for the Adobe Research Fellowship 2020.
Sketch-based Creativity Support Tools using Deep Learning
Forrest Huang, Eldon Schoop, David Ha, Jeffrey Nichols, and John Canny
In Artificial Intelligence for Human Copmuter Interaction: A Modern Approach (Springer 2021) | Preprint | Book
An Early Rico Retrospective: Three Years of Uses for a Mobile App Dataset
Biplab Deka, Bardia Doosti, Forrest Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Ranjitha Kumar, Tao Dong, and Jeffrey Nichols
In Artificial Intelligence for Human Copmuter Interaction: A Modern Approach (Springer 2021) | Book
Automatic Macro Mining from Interaction Traces at Scale
Forrest Huang, Gang Li, Tao Li, and Yang Li
Proceedings of CHI '24 | Paper (Preprint)
PLay: Parametrically Conditioned Layout Generation using Latent Diffusion
Chin-Yi Cheng, Forrest Huang, Gang Li, and Yang Li
Proceedings of ICML '23 | Paper
Multi-modal Search for Inspirational Examples in Design
Elisa Kwon, Forrest Huang, and Kosa Goucher-Lambert
Proceedings of IDETC '21 | Paper
UMLAUT: Debugging Deep Learning Programs using Program Structure and Model Behavior
Eldon Schoop, Forrest Huang, and Björn Hartmann
Proceedings of CHI '21 | Paper
Scones: Towards Conversational Authoring of Sketches
Forrest Huang, Eldon Schoop, David Ha, and John F. Canny
Proceedings of IUI '20 (Long Paper) | Paper
Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications
Forrest Huang, and John F. Canny
Proceedings of UIST '19 | Paper
Swire: Sketch-based User Interface Retrieval
Forrest Huang, John F. Canny, and Jeffrey Nichols
Proceedings of CHI '19 | Paper
MakerLens: What Sign-In, Reservation and Training Data Can (and Cannot) Tell You About Your Makerspace
Eldon Schoop, Forrest Huang, Nathan Khuu, and Björn Hartmann
Proceedings of ISAM '18 | Paper
ZIPT: Zero-Integration Performance Testing of Mobile App Designs
Biplab Deka, Zifeng Huang, Chad Franzen, Jeffrey Nichols, Yang Li, and Ranjitha Kumar
Proceedings of UIST '17 | Paper
Rico: A Mobile App Dataset for Building Data-Driven Design Applications
Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar
Proceedings of UIST '17 | Paper
ERICA: Interaction Mining for Mobile Apps
Biplab Deka, Zifeng Huang, and Ranjitha Kumar
Proceedings of UIST '16 | Paper
Enabling Multi-modal Search for Inspirational Design Stimuli using Deep Learning
Elisa Kwon, Forrest Huang, and Kosa Goucher-Lambert
AI EDAM '22 | Paper
GPU-Accelerated t-Distributed Stochastic Neighbor Embedding
David M. Chan*, Roshan Rao*, Forrest Huang*, and John F. Canny
Journal of Parallel and Distributed Computing | Paper
Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models
Forrest Huang, Gang Li, Xin Zhou, John F. Canny, and Yang Li
CHI 2022 Computational UI Workshop | Paper
SCRAM: Simple Checks for Realtime Analysis of Model Training for Non-Expert ML Programmers
Eldon Schoop, Forrest Huang, and Björn Hartmann
Late-Breaking Works of CHI '20 and ICML 2020 Workshop on Human in the Loop Learning | LBW
t-SNE-CUDA: GPU-Accelerated t-SNE and its Applications to Modern Data
David M. Chan*, Roshan Rao*, Forrest Huang*, and John F. Canny
HPML Workshop '18 Outstanding Paper Award | Paper
Ranking Designs and Users in Online Social Networks
Biplab Deka, Haizi Yu, Devin Ho, Zifeng Huang, Jerry O. Talton, and Ranjitha Kumar
Extended Abstracts of CHI '15 | Poster | Extended Abstract