Kohaku-Blueleaf eb2f78b4e0

[Training Node] algo support, grad acc, optional grad ckpt (#9015) ...

* Add factorization utils for lokr

* Add lokr train impl

* Add loha train impl

* Add adapter map for algo selection

* Add optional grad ckpt and algo selection

* Update __init__.py

* correct key name for loha

* Use custom fwd/bwd func and better init for loha

* Support gradient accumulation

* Fix bugs of loha

* use more stable init

* Add OFT training

* linting

2025-07-23 20:57:27 -04:00

..

__init__.py

[Training Node] algo support, grad acc, optional grad ckpt (#9015)

2025-07-23 20:57:27 -04:00

base.py

[Training Node] algo support, grad acc, optional grad ckpt (#9015)

2025-07-23 20:57:27 -04:00

boft.py

Fix the bugs in OFT/BOFT moule (#7909)

2025-05-02 13:12:37 -04:00

glora.py

Unified Weight Adapter system for better maintainability and future feature of Lora system (#7540)

2025-04-21 20:15:32 -04:00

loha.py

[Training Node] algo support, grad acc, optional grad ckpt (#9015)

2025-07-23 20:57:27 -04:00

lokr.py

[Training Node] algo support, grad acc, optional grad ckpt (#9015)

2025-07-23 20:57:27 -04:00

lora.py

LoRA Trainer: LoRA training node in weight adapter scheme (#8446)

2025-06-13 19:25:59 -04:00

oft.py

[Training Node] algo support, grad acc, optional grad ckpt (#9015)

2025-07-23 20:57:27 -04:00