This repository contains my notes and insights from studying for the AISYS prelim at berkeley.
The notes are organized according to the official AISYS prelim reading list.
| # | Paper Title | PDF Link | Notes | Category | Venue | Year |
|---|---|---|---|---|---|---|
| 1 | ImageNet Classification with Deep Convolutional Neural Networks | 📝 | Training Systems | NeurIPS | 2012 | |
| 2 | XGBoost: A Scalable Tree Boosting System | 📝 | ClassicML | KDD | 2016 | |
| 3 | Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization | 📝 | TensorOptimization | - | - | |
| 4 | Anatomy of High-Performance Matrix Multiplication | 📝 | Architecture | - | - | |
| 5 | Why Systolic Architectures | 📝 | Architecture | - | - | |
| 6 | PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs | 📝 | Graph Systems | OSDI | 2012 | |
| 7 | The Case for Learned Index Structures | 📝 | AI for Systems | SIGMOD | 2018 | |
| 8 | Using the BSP cost model to optimise parallel neural network training | 📝 | Pre-rec, Scaling Training | Future Generation Computer Systems | 1998 | |
| 9 | Machine Learning: The High Interest Credit Card of Technical Debt | 📝 | Industry Perspective | NeurIPS Workshop | 2014 | |
| 10 | Roofline: An Insightful Visual Performance Model for Multicore Architectures | 📝 | Architecture | CACM | 2009 | |
| 11 | AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving | 📝 | Inference Systems | OSDI | 2023 | |
| 12 | TensorFlow: A System for Large-Scale Machine Learning | 📝 | Training Systems | OSDI | 2016 | |
| 13 | GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers | 📝 | Quantization | - | 2022 | |
| 14 | Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism | 📝 | Scaling Training, Training Systems | - | 2018 | |
| 15 | Efficient Memory Management for Large Language Model Serving with PagedAttention | 📝 | Inference Systems | SOSP | 2023 |
Hardware architectures and overview: 📝
Parallelism: 📝
Transformer: 📝
Feel free to:
- Open issues for corrections or suggestions
- Open issues for open discussion on the paper
- Submit pull requests for improvements
- Use these notes for your own studies
- Share with fellow students
Note: This repository is a work in progress. Papers will be added and notes will be updated as I progress through the ready list.
Acknowledgement: Creating this repo is inspired by Shu's System study note: Berkeley OS Prelim Notes