System Group, University of Wisconsin, Madison
Research Assistant, Sep 2019 – Sep 2020
Advisor: Shivaram Venkataraman
• Developed a flexible framework for processing video DNN tasks with ∼2500 lines of C++
• Used the framework to perform tasks in video DNN pipelines including object detection and object tracking
ByteDance AI Lab @Beijing, China
Research Intern, Sep 2020 – Feb 2021
Advisor: Bairen Yi, Yibo Zhu
• Designed several strategies for overlapping computation and communication across operators
• Developed high performance DNN operators for model parallelism using CUDA/CUTLASS/NCCL
• Saved up to 30% of the communication time for a single operator in the end-to-end benchmark
ByteDance AML (Applied Machine Learning) @Beijing, China/ Madison, WI, United States
Research & Engineering Intern, Feb 2021 – Nov 2021
Advisor: Yibo Zhu, Shivaram Venkataraman
• Research
– Explored ways to accelerate distributed recommendation system training
– Formulated the training process as a graph problem and used Metis library to find better parameter placement in asynchronous training
– Investigated data access improvements such as caching to accelerate synchronous training
• Engineering
– Incorporated new DNN operators into the recommendation system training framework
– Built tools for quickly checking correctness of gradients during the training process