Di Zhang

About Me

I am a Research Scientist at Meta. I obtained Ph.D. degree from UNC Charlotte, supervised by Prof. Dong Dai. Before that, I worked as a Machine Learning Engineer focusing on Natural Language Processing in iFLYTEK Research Lab. I obtained my bachelor’s degree from University of Science and Technology of China (USTC)

My research interests include Reinforcement Learning, Scheduling and Anomaly Detection. My current research focuses on applying deep learning models to anomaly detection and automated batch job schedulers.

News

• [Jan. 2024] I am joining Meta as a Research Scientist working on Instagram Feed Ranking.
• [Dec. 2023] Our paper about HPC and AI clusters is accepted by IPDPS’24. Congrats to Monish Soundar Raj. Monish is an Undergraduate Student and my mentee, and this is his first paper.
• [Sept. 2023] Our paper about HPC Job Scheduling is accepted by PMBS’23 at SC’23. Congrats to Elliot Kolker-Hicks. Elliot is an Undergraduate Student and my mentee, and this is their first paper.
• [May. 2023] I will start my summer internship as a return Research Scientist Intern at Meta.
• [May. 2023] Our poster about HPC Job Scheduling wins the Outstanding Poster Award at the IPDPS’23 Ph.D. forum.
• [May. 2023] I will present our work about Anomaly Detection at IPDPS’23.
• [Mar. 2023] Our poster proposal about HPC Job Scheduling has been accepted for presentation at the IPDPS’23 Ph.D. forum.
• [Jan. 2023] Our paper about Anomaly Detection is accepted by IPDPS’23.
• [Sept. 2022] Our paper about Anomaly Detection is accepted by SC’22 FTXS. Congrats to Chris Egersdoerfer. Chris is an Undergraduate Student and my mentee, and this is his first paper.
• [Jun. 2022] I will present our work at HPDC’22.
• [May. 2022] I will start my summer internship as a Research Scientist Intern at Meta.
• [May. 2022] I recived CCI Essam El-Kwae Student-Faculty Research Award together with my advisor Dr. Dong Dai.
• [Mar. 2022] Our paper about HPC Job Scheduling is accepted to HPDC’22.
• [Sept. 2021] Our paper about Parallel File Systems is accepted by ACM TOS’22.
• [Jun. 2021] Our paper about Anomaly Detection is accepted by HotStorage’21 Best Paper Nominee
• [Jun. 2020] Our paper about HPC Job Scheduling is accepted by SC’20.

Publications

• Cross-System Analysis of Job Characterization and Scheduling in Large-Scale Computing Clusters.
  Di Zhang, Monish Soundar Raj, Bing Xie, Sheng Di, Dong Dai
  Accepted to appear in the 38th IEEE International Parallel & Distributed Processing Symposium (IPDPS’24), 2024.
  PDF(TBA)

• A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs.
  Elliot Kolker-Hicks, Di Zhang, Dong Dai
  14th IEEE International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS’23), 2023.
  [PDF]

• Early Exploration of Using ChatGPT for Log-based Anomaly Detection on Parallel File Systems Logs.
  Chris Egersdoerfer, Di Zhang, Dong Dai
  In Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing (HPDC’23 Poster), 2023.
  [PDF]

• Optimizing Resource Management for Machine Learning Workloads in High-Performance Clusters
  Di Zhang, Dong Dai
  In Proceedings of the 37th IEEE International Parallel & Distributed Processing Symposium Workshop (IPDPS’23 Phd Forum), 2023.
  [PDF][Poster]

• Drill: Log-based Anomaly Detection for Large-scale Storage Systems Using Source Code Analysis.
  Di Zhang, Chris Egersdoerfer, Tabassum Mahmud, Mai Zheng, Dong Dai
  37th IEEE International Parallel & Distributed Processing Symposium (IPDPS’23), 2023
  [PDF][Code][Slides]

• ClusterLog: Clustering Logs for Effective Log-based Anomaly Detection
  Chris Egersdoerfer, Di Zhang, Dong Dai
  Workshop on Fault Tolerance for HPC at eXtreme Scale (with SC’22).
  [PDF]

• SchedInspector: A Batch Job Scheduling Inspector Using Reinforcement Learning
  Di Zhang, Dong Dai, Bing Xie
  31st International ACM Symposium on High-Performance Parallel and Distributed Computing. HPDC’22.
  [PDF] [Code] [Slides]

• A Study of Failure Recovery and Logging of High-Performance Parallel File Systems
  Runzhou Han, Om Rameshwar Gatla, Mai Zheng, Jinrui Cao, Di Zhang, Dong Dai, Yong Chen, Jonathan Cook
  ACM Transactions on Storage. TOS’22.
  [PDF]

• SentiLog: Anomaly Detecting on Parallel File Systems via Log-based Sentiment Analysis
  Di Zhang, Dong Dai, Runzhou Han, Mai Zheng
  13th ACM Workshop on Hot Topics in Storage and File Systems. HotStorage’21.
  [PDF] [Slides]

• RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning
  Di Zhang, Dong Dai, Youbiao He, Forrest Sheng Bao, Bing Xie
  International Conference for High Performance Computing, Networking, Storage and Analysis. SC’20.
  [PDF] [Code] [Slides]

Services

• Student volunteer in SC’21, ISSRE’22.
• External paper reviewer for ICPP’23, IPDPS’23, BIGDATA’22, IPDPS’22, ICPP’22, CCGrid’22, IPDPS’20, ICPP’20.
• Paper reviewer for TPDS’23.

Mentoring Activities

• Tony Li, Master Research Project. Now at Google as a software engineer.
• Chris Egersdoerfer, Bachelor Research Project. Now at UNCC as a master student.
• Elliot Kolker-Hicks, Bachelor Research Project.
• Alex Moore, Bachelor Research Project.
• Monish Soundar Raj, Bachelor Research Project.

Awards & Honors

• Outstanding Poster Award, IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2023
• Merit Scholarship $1500, UNC Charlotte, 2023
• Student Travel Grant $1000, ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2022
• CCI Essam El-Kwae Student-Faculty Research Award $500, UNC Charlotte, 2022