anthology-of-the-sota

SOTA-076: Place replicas to minimize cross-rack traffic

1 min read

Properties4
StatusRejected — a cluster-scheduling claim rather than a training one: it is the scheduler's decision, expressed as a placement-group request, not something in a training run. ADR-026 (superseding ADR-020) scopes this record by the kind of claim, and this is a different kind
Tagsdistributed-optimization
SourceLIT-051 — A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters
Introduced byLIT-051 — A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters

Graph View

Source

Jiang et al. (2020), LIT-051 — https://www.usenix.org/conference/osdi20/presentation/jiang.

A scheduling claim in a training registry

Placing the workers of a job so that its collectives stay inside a rack is sound and is what topology-aware schedulers do. It is also not a decision the person training the model usually makes: it belongs to the cluster scheduler, and on managed infrastructure it is expressed as a placement-group request rather than as anything in the training code.

That makes it a different kind of recommendation from the rest of this record, which is about what to do inside a training run. Worth keeping only if the anthology intends to hold cluster-scheduling practice, and worth saying so explicitly if it does — a scope question rather than a citation one, and the kind ADR-026 is about.

The citation, separately

LIT-051 does not argue for it. BytePS’s setting is a heterogeneous cluster whose spare CPU and network capacity it exploits; its contribution is the unified communication framework and the Summation Service split (SOTA-047). Placement is assumed as part of the environment, not recommended.

Backlinks