Stas Bekman stas00

Created 21 commits in 4 repositories

bigscience-workshop/bigscience 9 commits

stas00/ipyexperiments 6 commits

huggingface/transformers 5 commits

bigscience-workshop/Megatron-DeepSpeed 1 commit

Created a pull request in huggingface/transformers that received 17 comments
Dec 3

[trainer] add tf32-mode control

This PR adds tr32-mode control support for HF Trainer for Ampere cards. RFC: #14450 pytorch had this mode on by default since pt-1.7, but are discu…

+90 −27 • 17 comments

Opened 3 other pull requests in 1 repository

huggingface/transformers 1 open 2 merged

[WIP] [t5] faster/leaner custom layer norm Dec 7

[deepspeed] fix --load_best_model_at_end Dec 6

[CI] move env print to util, add pt, nccl versions Dec 3

Reviewed 4 pull requests in 2 repositories

huggingface/transformers 3 pull requests

[trainer] add tf32-mode control Dec 3

[doc] bf16/tf32 guide Dec 1

Support for Training with BF16 Dec 1

pytorch/rfcs 1 pull request

RFC-0020/0021/0022 RFCs for Pipeline Parallelism Dec 7

Opened 2 other issues in 2 repositories

microsoft/DeepSpeed 1 open

[BUG] load_checkpoint fails after deepspeed engine started training Dec 6

huggingface/transformers 1 open

[Benchmark] tf32 vs fp16 vs bf16 vs fp32 Dec 3

Stas Bekman stas00

Achievements

Achievements

Organizations

Block or report stas00

Pinned Loading

2,040 contributions in the last year

Activity overview

Contribution activity

December 2021

Created a pull request in huggingface/transformers that received 17 comments

[trainer] add tf32-mode control

Created an issue in pytorch/pytorch that received 1 comment

[performance] a profiler util to show a rough break-down of the types of ops used by a model