Chen et al. (2018) — ARXIV-1802.04799
Key takeaways
- The problem is portability of performance, not of code: frameworks lean on vendor-specific operator libraries tuned for a narrow range of server-class GPUs, so every new target — mobile, embedded, FPGA, ASIC — costs a manual re-optimization.
- TVM exposes both levels at once. Graph-level: high-level operator fusion, the transformation that turns a chain of small memory-bound ops into one kernel and stops the round-trip to memory between them. Operator-level: mapping to arbitrary hardware primitives and hiding memory latency.
- The low-level schedule space is too large to hand-tune per target, so it is searched, guided by a learning-based cost model — which is what makes the approach automated rather than a better library.
- Results competitive with hand-tuned vendor libraries across low-power CPU, mobile GPU and server GPU, plus an FPGA accelerator back-end. Open source and in production at several companies.
Standing in the anthology
The source of SOTA-081 (fuse small operations), SOTA-082 (optimize memory layout for the hardware), SOTA-083 (custom kernels for critical ops) and SOTA-084 (profile-guided optimization of hot paths). All four arrived under a fabricated citation and all four are TVM’s contributions.
Paired with LIT-066, which makes the same argument empirically for one workload: fusion and layout are where transformer training time actually goes.