Arithmetic Intensity Estimation of Large Language Models
This blog post discusses the arithmetic intensity of large language models and how it affects the performance of these models.
Blogs
A Brief Talk on Speculative Decoding
A brief talk on speculative decoding in large language models.
APC, SD, and SF
Explanation of Automatic Prefix Caching (APC), Speculative Decoding (SD), and Split Fuse (SF).
WSL is All You Need
How do I work with WSL
Create A LibTorch Project
How to create a LibTorch project.
My vimrc
My configurations of vim.
VSCode: Debug C++
This post shows how to configure launch.json in VSCode for debugging C++.
VSCode: Debug Python
This post shows how to configure launch.json in VSCode for debugging Python.
Render Mathematics in Hugo
This post shows how to render mathematics in Hugo.
Dive into Paged Attention
Dive into the paged attention mechanism of vLLM.