Arithmetic Intensity Estimation of Large Language Models
This blog post discusses the arithmetic intensity of large language models and how it affects the performance of these models.
Vllm
A Brief Talk on Speculative Decoding
A brief talk on speculative decoding in large language models.
Dive into Paged Attention
Dive into the paged attention mechanism of vLLM.