Networking is a well-known bottleneck for ML systems and the cluster demands efficient scheduling for huge traffic (up to 1GB per flow) generated by ML jobs.

Abstract—In recent years, many companies have developed various distributed computation frameworks for processing ma- chine learning (ML) jobs in clusters.

Nov 24, 2022 · As a practical approach to handling decentralized data, Federated Learning (FL) enables collaborative global machine learning model training ...

Cross-Layer Self-Similar Coflow Scheduling for Machine Learning Clusters ... ARS: Cross-layer adaptive request scheduling to mitigate TCP incast in data center ...

Jul 15, 2022 · This paper focuses on coflow scheduling with the goal of optimizing completion time, where we review the existing scheduling frameworks and ...

HQTimer: A Hybrid Q -Learning-Based Timeout Mechanism in Software-Defined Networks ... Cross-Layer Self-Similar Coflow Scheduling for Machine Learning Clusters ...

Cross-layer scheduling techniques. The idea of placing data and compute together has been explored in systems like. CAM [41] and Purlieus [43]. Unlike such ...

... Self-Learning Network. Speaker: Nicholas Zhang ... Scheduling. Kang Chen, Jianwei Liu, James Martin ... Cross-Layer Self-Similar Coflow Scheduling for Machine ...

Mar 20, 2024 · The state-of-the-art schedulers collect coflow information in the cloud to optimize coflow-level performance. However, most of the coflows, ...

Oct 11, 2020 · Our online algorithm consists of (i) an online scheduling framework that groups unprocessed ML training jobs into a batch iteratively, and (ii) ...