Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: deepspeedai/DeepSpeedExamples
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: shenzheyu/DeepSpeedExamples
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 8 commits
  • 13 files changed
  • 9 contributors

Commits on Mar 5, 2025

  1. DeepSpeed-Domino (#929) 

    * add domino

* use transformer from deepspeed

* clean args

* mega opt

* add opt & timer

* add opt

* fix loss

* folder name

* Change arguent in pretrain script

* Add readme for domino

* Update readme for domino

* Fixing usage issues

* update dataset

* megatron dependencies

* path

* Update README.md

* remove imports

* update import

* Update README.md

* Minor example script changes

* train bash

* require

* Update README.md

---------

Co-authored-by: chengming-zhang <[email protected]>
Co-authored-by: Zheyu SHEN <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Olatunji Ruwase <[email protected]>
Co-authored-by: Logan Adams <[email protected]>
Signed-off-by: Zheyu SHEN <[email protected]>
    @zhangsmallshark @chengming-zhang
    @shenzheyu @tjruwase @loadams
    6 people committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    1a5292b View commit details
    Browse the repository at this point in the history

  2. Update DeepSpeed version requirement to >=0.16.0 in requirements.txt … 

    …for Domino

Signed-off-by: Zheyu SHEN <[email protected]>
    @shenzheyu
    shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    c6a4462 View commit details
    Browse the repository at this point in the history

  3. Example and benchmark of APIs to offload states (#942) 

    * add benchmarking for offloading states

* fix api names

Signed-off-by: Zheyu SHEN <[email protected]>
    @tohtana @shenzheyu
    tohtana authored and shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    710d83f View commit details
    Browse the repository at this point in the history

  4. remove-redundant-code (#947) 

    Signed-off-by: Zheyu SHEN <[email protected]>
    @simonJJJ @shenzheyu
    simonJJJ authored and shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    06f5df4 View commit details
    Browse the repository at this point in the history

  5. Add DPO support for DeepSpeed-Chat (#828) 

    * Add label_smoothing while calculating step2 DPO loss in DeepSpeed-Chat.

* Add training scripts for step2 DPO in DeepSpeed-Chat.

* Remove unused packages and format the code of step2 DPO in DeepSpeed-Chat.

* Update training scripts of step2 DPO in DeepSpeed-Chat.

* Follow upstream fixes.

* Update README.md for Step2 DPO finetuning.

* Add opt 350M training log demo for step 2 dpo finetuning in DeepSpeed-Chat.

* Address the formatting issue in step2 dpo finetuning in DeepSpeed-Chat.

---------

Co-authored-by: Logan Adams <[email protected]>
Co-authored-by: Olatunji Ruwase <[email protected]>
Signed-off-by: Zheyu SHEN <[email protected]>
    @stceum @loadams
    @tjruwase @shenzheyu
    3 people authored and shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    7d54633 View commit details
    Browse the repository at this point in the history

  6. Update references to torchvision (#949) 

    Signed-off-by: Zheyu SHEN <[email protected]>
    @loadams @shenzheyu
    loadams authored and shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    9dc9b0d View commit details
    Browse the repository at this point in the history

  7. Add llama3 8b 

    Signed-off-by: Zheyu SHEN <[email protected]>
    @shenzheyu
    shenzheyu committed Mar 5, 2025
    Configuration menu
    Copy the full SHA
    680108d View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' into master

    @shenzheyu
    shenzheyu authored Mar 5, 2025
    Configuration menu
    Copy the full SHA
    24c0969 View commit details
    Browse the repository at this point in the history
Loading