Efficient AI 2

Efficient AI Navigator

모델 압축

Pruning · Quantization · Knowledge Distillation · Low-rank Decomposition

효율적 구조 설계

Efficient Attention · MoE · NAS · MobileNet

입력/토큰 축소

Token Pruning · Token Merging · Early Exit · Dynamic Inference

시스템/커널 최적화

FlashAttention · Kernel Fusion · Speculative Decoding · TensorRT · TVM

효율적 학습

LoRA · Mixed Precision · Gradient Checkpointing

Pruning Navigator

Granularity

Element pruning · N:M (2:4) · Head pruning · Width pruning · Block pruning

Criteria

Magnitude · Wanda · Hessian ( OBD · OBS · SparseGPT )

Schedule

One-shot · Iterative ( Lottery Ticket ) · Gradual · Pruning at Initialization ( SNIP · GraSP )

• Pruning 2 posts