CS 723 Reading List

  • Please submit your critiques before noon on Mondays to yunho.oh@epfl.ch
  • Week
    #1Sep. 17: Introduction
    PDF The Task of the Referee
    #2Sep. 24: Benchmarks and Analytics
    PDF MLPerf training benchmark [Google et al., MLSys 20]
    PDF MLPerf Inference Benchmark [Harvard et al., ISCA 20]
    #3Oct. 01: Systems & ML
    PDF A Systematic Methodology for Analysis of Deep Learning Hardware and Software Platforms [Harvard, MLSys 2020]
    PDF The Case for Learned Index Structures [MIT and Google, SIGMOD 18]
    #4Oct. 08: Federated Learning
    PDF Towards federated learning at scale: System design [Google, MLSys 19]
    PDF The Non-IID Data Quagmire of Decentralized Machine Learning [ETHZ and CMU, ICML 20]
    #5Oct. 15: Decentralized Learning
    PDF SwarmSGD: Scalable Decentralized SGD with Local Updates [IST Austria, Arxiv 2020]
    PDF Byzantine-Resilient Multi-Agent Optimization [MIT, IEEE 2020]
    #6Oct. 22: Deep Learning with Low-precision Computations
    PDF Training DNNs with Hybrid Block Floating Point [EPFL, NeurIPS 18]
    PDF Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks [Xilinx, MLSys 20]
    #7Oct. 29: Training with Low-precision Gradients
    PDF Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training [Tsinghua & Nvidia, ICLR 18]
    PDF PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning [EPFL, arXiv 20]
    #8Nov. 05: Distributed Imagenet & Transformers Training
    PDF Beyond Data and Model Parallelism for Deep Neural Networks [Stanford, MLSys 19]
    PDF Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism [Nvidia, arXiv 20]
    #9Nov. 12: Training with Model Parallelism
    PDF PipeDream: generalized pipeline parallelism for DNN training [Mircosoft et al., SOSP 19]
    PDF Decoupled Parallel Backpropagation with Convergence Guarantee [Pittsburgh, ICML 18]
    #10Nov. 19: Neural Architecture Search
    PDF ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware [MIT, ICLR 19]
    PDF MnasNet: Platform-Aware Neural Architecture Search for Mobile [Google, CVPR 19]
    #11Nov. 26: Domain Specific Languages for ML
    PDF Blink: Fast and Generic Collectives for Distributed ML [Microsoft et al., MLSys 20]
    PDF TVM: An Automated End-to-End Optimizing Compiler for Deep Learning [Washington, OSDI 18]
    #12Dec. 03: ML inference at scale
    PDF Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective [Facebook, HPCA 18]
    PDF The Architectural Implications of Facebook’s DNN-based Personalized Recommendation [Facebook, HPCA 20]
    #13Dec. 10: Hardware Accelerators for Deep Learning
    PDF Simba: Scaling Deep-Learning Inference with Multi-Chip-Module-Based Architecture [UCB et al., MICRO 19]
    PDF Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices [MIT, IEEE JETCAS 19]
    #14Dec. 17: Security
    PDF Game of Threads: Enabling Asynchronous Poisoning Attacks [UIUC, ASPLOS 20]
    PDF Stealing Machine Learning Models via Prediction APIs [EPFL, USENIX Security, 16]