Biography

Dr. Wuyang Chen is an incoming tenure-track Assistant Professor in the School of Computing Science at Simon Fraser University (Fall 2024). Currently, Dr. Chen is a postdoc researcher in Statistics at the University of California, Berkeley, working with Professor Michael Mahoney. He obtained his Ph.D. from the ECE Department at UT Austin in 2023, under the supervision of Professor Atlas Wang. Dr. Chen's research focuses on the theoretical understanding of deep learning, with applications in foundation models, AutoML, computer vision, natural language processing, and addressing scientific problems. His work also encompasses domain adaptation/generalization and self-supervised learning. He published papers on CVPR, ECCV, ICLR, ICML, Neurips, etc. Dr. Chen's work on training-free neural architecture design was highlighted as the "Featured Advances in Artificial Intelligence" in the National Science Foundation (NSF) newsletter in 2022. Dr. Chen co-organized the 4th and 5th versions of UG2+ workshop and challenge in CVPR 2021 and 2022. He also holds a position on the board of the One World Seminar Series on the Mathematics of Machine Learning.


wuyang.chen AT berkeley.edu
wuyang AT sfu.ca

Research:
Theory-accelerated AI, with Implications for Scientific Problems


  • [Core Method] Theory-accelerated AI
    We are dedicated to examining key aspects of Deep Learning, including data, model architectures (trainability/expressivity/generalization), and training dynamics. This systematic analysis aims to expedite the AI development pipeline by optimizing data quality, model design/scaling, and training recipe in a zero-shot manner.
  • [Core Application] AI-for-Science
    Our methodologies, guided by theoretical principles, aim to expedite scientific discovery, with a specific focus on the human genome and scientific machine learning.
    In addition to the core aims mentioned above, we are also interested in a broad range of topics, including computer vision, natural language processing, model efficiency, automated machine learning (AutoML), self-supervised learning, and domain generalization.

Publication

Transferable and Principled Efficiency for Open-Vocabulary Segmentation
Jingxuan Xu, Wuyang Chen, Yao Zhao, Yunchao Wei
CVPR 2024
Paper Code
Principled Architecture-aware Scaling of Hyperparameters
Wuyang Chen, Junru Wu, Zhangyang Wang, Boris Hanin
ICLR 2024
Paper Code
Mixture-of-Experts Meets Instruction Tuning: A Winning Combination for Large Language Models
Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Y Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou
ICLR 2024
Paper
"No Free Lunch" in Neural Architectures? A Joint Analysis of Expressivity, Convergence, and Generalization
Wuyang Chen*, Wei Huang*, Zhangyang Wang
AutoML Conference 2023
Paper
Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices
Yimeng Zhang, Akshay Karkal Kamath, Qiucheng Wu, Zhiwen Fan, Wuyang Chen, Zhangyang Wang, Shiyu Chang, Sijia Liu, Cong Hao
Proceedings of the 28th Asia and South Pacific Design Automation Conference (ASP-DAC 2023)
Paper
Lifelong Language Pretraining with Distribution-Specialized Experts
Wuyang Chen, Yanqi Zhou, Nan Du, Yanping Huang, James Laudon, Zhifeng Chen, Claire Cui
ICML 2023
Paper
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis
Wuyang Chen*, Wei Huang*, Xinyu Gong, Boris Hanin, Zhangyang Wang
Neurips 2022
Paper Code
A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation
Wuyang Chen, Xianzhi Du, Fan Yang, Lucas Beyer, Xiaohua Zhai, Tsung-Yi Lin, Huizhong Chen, Jing Li, Xiaodan Song, Zhangyang Wang, Denny Zhou
ECCV 2022
Paper
Auto-Scaling Vision Transformers without Training
Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wang, Denny Zhou
ICLR 2022
Paper Code
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining
Miao Lu, Xiaolong Luo, Tianlong Chen, Wuyang Chen, Dong Liu, Zhangyang Wang
ICLR 2022
Paper Code
Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity
Xinyu Gong, Wuyang Chen, Tianlong Chen, Zhangyang Wang
WACV 2022
Paper Code
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Chaojian Li, Wuyang Chen, Yuchen Gu, Tianlong Chen, Yonggan Fu, Zhangyang Wang, Yingyan Lin
ACM Transactions on Design Automation of Embedded Systems (TODAES)
Paper
Learning to Optimize: A Primer and A Benchmark
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, Wotao Yin
JMLR
Paper Code
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
Wuyang Chen, Xinyu Gong, Zhangyang Wang
ICLR 2021
Paper Code Project Video NSF News UT ECE News
Contrastive Syn-to-Real Generalization
Wuyang Chen, Zhiding Yu, Shalini De Mello, Sifei Liu, Jose M. Alvarez, Zhangyang Wang, Anima Anandkumar
ICLR 2021
Paper Project Code
Automated Synthetic-to-Real Generalization
Wuyang Chen, Zhiding Yu, Zhangyang Wang, Animashree Anandkumar
ICML 2020
Paper Project Code
Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training
Wuyang Chen*, Xuxi Chen*, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, Zhangyang Wang
ICML 2020
Paper Code
Auto-GAN-Distiller: Searching to Compress Generative Adversarial Networks
Yonggan Fu, Wuyang Chen, Haotao Wang, Haoran Li, Yingyan Lin, Zhangyang Wang
ICML 2020
Paper Code
FasterSeg: Searching for Faster Real-time Semantic Segmentation
Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang
ICLR 2020
Paper Code
AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
Xinyu Gong, Wuyang Chen, Yifan Jiang, Ye Yuan, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang
Paper Code
Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images
Wuyang Chen*, Ziyu Jiang*, Zhangyang Wang, Kexin Cui, Xiaoning Qian
CVPR 2019 (oral)
Paper Code Video
Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification
Ye Yuan, Wuyang Chen, Tianlong Chen, Yang Yang, Zhou Ren, Zhangyang Wang, Gang Hua
WACV 2020
Paper Code
In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation
Ye Yuan, Wuyang Chen, Yang Yang, Zhangyang Wang
Paper Code
ABD-Net: Attentive but Diverse Person Re-Identification
Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, Zhangyang Wang
ICCV 2019
Paper Code

Talks


Professional Experience & Activity


Professional Services:

Awards and Honors:

  • Professional Development Award, University of Texas at Austin, 2022.
  • Travel Award, Mathematical/Statistical approaches in DAta Science (MSDAS) Workshop, UT Dallas, 2019.
  • Best Student Speaker and Outstanding Researcher, CSST program, UCLA, 2013.
  • UCLA CSST Scholarship, 2013
  • Gold Medal (Team Leader of 20 members) iGEM, MIT, 2012

2022 Wuyang Chen. Powered by Shuai Yang