Arithmetic Intensity Estimation of Large Language Models
This blog post discusses the arithmetic intensity of large language models and how it affects the performance of these models.
Deeplearning
A Brief Talk on Speculative Decoding
A brief talk on speculative decoding in large language models.
APC, SD, and SF
Explanation of Automatic Prefix Caching (APC), Speculative Decoding (SD), and Split Fuse (SF).
Create A LibTorch Project
How to create a LibTorch project.
Dive into Paged Attention
Dive into the paged attention mechanism of vLLM.
A Simple Pytorch Trainpipeline
How to build a simple Pytorch trainpipeline.