torch-pipeline-parallelism by benchflow-ai

torch-pipeline-parallelism

benchflow-ai

9382423 months ago

Guidance for implementing PyTorch pipeline parallelism for distributed model training. This skill should be used when tasks involve implementing pipeline parallelism, distributed training with model partitioning across GPUs/ranks, AFAB (All-Forward-All-Backward) scheduling, or inter-rank tensor communication using torch.distributed.

>_
Quick Install
npxskills add benchflow-ai/skillsbench--skill torch-pipeline-parallelism
Instructions

No instructions available