Xiaohang Zhan (詹晓航)
ORCID iD iconorcid.org/0000-0003-2136-7592

Email: xiaohangzhan[AT]outlook[dot]com

About Me (GitHub) (Google scholar) (LinkedIn)

Xiaohang Zhan is currently a senior research scientist in AI Lab, Tencent. Before that, He received his B.Eng. degree from Tsinghua University in 2016, and Ph.D. degree from the Multimedia Laboratory (MMLab) at the Chinese University of Hong Kong in 2020 (1 year shortened), supervised by Chen Change Loy and Xiaoou Tang. Afterwards, he joined Huawei (2020) and then Tencent (2023). He is the reviewer of most top conferences and journals in AI&CV. He was awarded Top 100 Chinese New Stars in Artificial Intelligence by Baidu Scholar in 2021.
About his research preference: He is interested in curiosity-driven research on unsolved problems. He is keen on taking challenges of new problems as long as they are valuable in research or business.

Education

The Chinese University of Hong Kong (MMLab)

August 2017 - August 2020 (1 year shortened). Ph.D. in Information Engineering.
August 2016 - August 2017. Research assistant.
With “Top 100 Chinese Rising Stars in Artificial Intelligence”.
Tsinghua University

August 2012 - July 2016. Bachelor.
With "Outstanding Undergraduates Award" (top 1.8%), National Scholarship (top 1.5%).

Professional Experience

AI Lab, Tencent

Mar 2023 - Now.
Senior Research Scientist, Tech Leader.
Research in 2D/3D/4D contents generation, and technology transfer to products in game industry and social media.
2012 Labs, Huawei

Aug 2020 - Feb 2023.
Research Scientist.
Research in 3D avatars, including parametric modeling, bone rigging, motion synthesis.
SenseTime

Dec 2017 - April 2018, Mar 2016 - Oct 2016.
Research Intern.
Research on large-scale clustering and semi-supervised learning for face recognition.
Research on lane detection for autonomous driving.
Multimedia Laboratory, The Chinese University of Hong Kong

Oct 2016 - Aug 2017, July 2015 - Sept 2015.
Junior Research Assistant.
Research on self-supervised learning and optical flow estimation.

Productized Projects

Audio-Driven 3D Face Animation Solution @Tencent 2023-now

Achievements: real-time high-quality audio-driven 3D face animation algorithm based on MetaHuman rigs, surpassing NVIDIA Omniverse Audio2Face in lip-sync accuracy and naturalness by far in large-scale user study.
Productization: LLM-based conversational digital human, cutscenes animation production, short promotional videos production, AI NPCs in game projects. Related products: Honor of Kings (王者荣耀), Game for Peace (和平精英), etc.
Publicity: Game for Peace news
Data-Efficient Auto-Rigging Solution for Biped Characters @Huawei 2021-2022

Achievements: robust results for variance in shape, pose and mesh topology, industry-leading accuracy and efficiency.
Related products: HMS Core.
Publicity: Huawei Developer Conference 2022 (start from 1:01:00), Bilibili.
MMSelfSup Open-source Project @MMLab 2020-2021

Achievements: MMSelfSup is a widely-used open-source self-supervised representation learning toolbox based on PyTorch. Github stars: ~3.1k.
Project page: mmselfsup
Related products: OpenMMLab platform

Selective Publications

Programmable Motion Generation for Open-Set Motion Control Tasks

Hanchao Liu, Xiaohang Zhan, Shaoli Huang, Tai-Jiang Mu, Ying Shan
CVPR 2024 (Highlight, 11.9% of accepted papers)

[Paper] [Project Page] [Code] [Demo]
HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting

Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, Ziwei Liu
CVPR 2024 (Highlight, 11.9% of accepted papers)

[Paper] [Project Page] [Code] [Demo]
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo
TPAMI 2021 & ECCV 2020 (Oral)

[Paper] [Project Page] [Code]
Self-Supervised Scene De-occlusion

Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy
CVPR 2020 (Oral)

[Paper] [Project Page] [Code] [Demo]
Online Deep Clustering for Unsupervised Representation Learning

Xiaohang Zhan*, Jiahao Xie*, Ziwei Liu, Yew-Soon Ong, Chen Change Loy
CVPR 2020

[Paper] [Code]
Learning to Cluster Faces on an Affinity Graph

Lei Yang, Xiaohang Zhan, Dapeng Chen, Junjie Yan, Chen Change Loy, Dahua Lin
CVPR 2019 (Oral)

[PDF] [Paper] [Code]
Large-scale Long-Tailed Recognition in an Open World

Ziwei Liu*, Zhongqi Miao*, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu
CVPR 2019 (Oral)

[Paper] [Project Page] [Code]
Self-Supervised Learning via Conditional Motion Propagation

Xiaohang Zhan, Xingang Pan, Ziwei Liu, Dahua Lin, Chen Change Loy
CVPR 2019

[Paper] [Project Page] [Code] [Demo]
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition

Xiaohang Zhan, Ziwei Liu, Junjie Yan, Dahua Lin, Chen Change Loy
ECCV 2018

[Paper] [Project Page] [Code]

Other Publications

DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mapping

Zeyu Cai, Duotun Wang, Yixun Liang, Zhijing Shao, Ying-Cong Chen, Xiaohang Zhan, Zeyu Wang
Pacific Graphics 2024

[Paper]
HeadEvolver: Text to Head Avatars via Locally Learnable Mesh Deformation

Duotun Wang*, Hengyu Meng*, Zeyu Cai*, Zhijing Shao, Qianxi Liu, Lin Wang, Mingming Fan, Ying Shan, Xiaohang Zhan, Zeyu Wang
arXiv preprint

[Paper] [Project Page]
TapMo: Shape-aware Motion Generation of Skeleton-free Characters

Jiaxu Zhang, Shaoli Huang, Zhigang Tu, Xin Chen, Xiaohang Zhan, Gang Yu, Ying Shan
ICLR 2024

[Paper]
RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent Dataset

Zhongjin Luo*, Shengcai Cai*, Jinguo Dong, Ruibo Ming, Liangdong Qiu, Xiaohang Zhan, Xiaoguang Han
CVPR 2023

[Paper] [Project Page] [Code] [Dataset]
Masked Frequency Modeling for Self-Supervised Visual Pre-Training

Jiahao Xie, Wei Li, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
ICLR 2023

[Paper] [Project Page] [Code]
Delving into Inter-Image Invariance for Unsupervised Visual Representations

Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
IJCV 2022

[Paper] [Code]
Open Long-Tailed Recognition in a Dynamic World

Ziwei Liu*, Zhongqi Miao*, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu
TPAMI 2022

[Paper] [Project Page] [Code] [Demo]
Style‐ERD: Responsive and Coherent Online Motion Style Transfer

Tianxin Tao, Xiaohang Zhan, Zhongquan Chen, Michiel van de Panne
CVPR 2022

[Paper] [Project Page] [Code]
Beyond Single Instance Multi-view Unsupervised Representation Learning

Xiangxiang Chu, Xiaohang Zhan, Xiaolin Wei
BMVC 2022

[Paper]
Unsupervised Object-Level Representation Learning from Scene Images

Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
NeurIPS 2021

[Paper] [Project Page] [Code]
DetCo: Unsupervised Contrastive Learning for Object Detection

Enze Xie*, Jian Ding*, Wenhai Wang, Xiaohang Zhan, Hang Xu, Zhenguo Li, Ping Luo
ICCV 2021

[Paper] [Code]
Learning to Cluster Faces via Confidence and Connectivity Estimation

Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, Dahua Lin
CVPR 2020

[Paper] [Project Page] [Code]
Compound Domain Adaptation in an Open World

Ziwei Liu*, Zhongqi Miao*, Xingang Pan, Xiaohang Zhan, Stella X. Yu, Dahua Lin, Boqing Gong
CVPR 2020 (Oral)

[Paper] [Project Page] [Demo] [Code]
Collaborative Online Deep Clustering for Unsupervised Representation Learning

Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew Soon Ong, Chen Change Loy
The Champion of Facebook AI Self-Supervision Challenge (all tracks), in ICCV Extreme Vision Workshop, 2019

Switchable Whitening for Deep Representation Learning

Xingang Pan, Xiaohang Zhan, Jianping Shi, Xiaoou Tang, Ping Luo
ICCV 2019

[Paper] [Code]
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation

Xiaohang Zhan, Ziwei Liu, Ping Luo, Xiaoou Tang, Chen Change Loy
AAAI 2018 (Spotlight)

[Paper] [Project Page] [Code]

Other Projects

CULane Dataset

CULane is a large scale challenging dataset for academic research on traffic lane detection.
[Project Page]

Invited Talks

  • 2017 @ JD.com: Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
  • 2019 @ HKSTP: Self-Supervised Learning via Conditional Motion Propagation
  • 2020 @ SenseTime TITAN Open Classes [Video Record]: Self-Supervised Learning
  • 2020 @ Tsinghua: Exploiting Unlabeled Data in Computer Vision
  • 2020 @ Zhidongxi Open Classes [Video Record]: Understanding and Practice of OpenSelfSup: an Open-source Library for Self-Supervised Learning
  • 2021 @ TechBeat [Video Record]: Occlusion Problems in Natural Scenes
  • 2022 @ Huawei Developer Conference [Video Record]: 3D Creation Pipeline Upgraded, Anything Can Be Animated

    • Honors and Awards

  • Top 100 Chinese Rising Stars in Artificial Intelligence, AI华人新星百强 (100 out of 45205), 2021, Baidu Scholar.
  • Huawei Genius Youth Program (华为天才少年计划), 2020
  • Winner of Facebook AI Self-Supervision Challenge 2019
  • Hong Kong PhD Fellowship, 2017, the Research Grants Council (RGC) of Hong Kong
  • Outstanding Undergraduates Award of Tsinghua University, 2016 (63 winners out of about 3500 graduates)
  • Outstanding Undergraduates Award of Beijing, 2016
  • National Scholarship, 2014, Ministry of Education of China
  • Technological Innovation Scholarship, 2014, 2015, Tsinghua University
  • First class Academic Scholarship, 2013, 2014, Tsinghua University
  • FastGear Special class Integrated Excellence Scholarship, 2013, Tsinghua University
  • “Spark” Innovative Talent Cultivation Program for Students of Tsinghua University (50 out of about 3500)

    • Published with GitHub Pages