Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upAdd various pruner algorithm for different pruning schedule #2853
Comments
@tu1258 Thanks for reporting this issue. Do you have some specific requirement about adding which pruner to which scheduler? Some pruners such as Apoz need to collect activation before using, and TaylorFO needs to collect gradients, they may not fit every scheduler easily. We need to handle this case by case if you have specific requirement. |
@chicm-ms Thanks for the reply. Honestly, I don't plan to use specific pruner or pruning schedule, and that's why I report this issue. I want to try different setting and see which is better. I think the more choice the better. This issue is not only for me, I just feel that it would benefit a lot for many users. If anything, I think at least add fpgm which is the most popular? |
@tu1258 FPGM can be added since it does not need to collect activation/gradient. |
What would you like to be added: As title
Why is this needed: All pruning schedule except AGPPruner only support level, L1, L2. While there are FPGM, APoZ, MeanActivation and Taylor, it would be much better if we can choose any pruner with any pruning schedule.
Without this feature, how does current nni work:Still good.
Components that may involve changes: I am not so sure tho.
Brief description of your proposal if any: I look into source code of some pruning schedule and it's like
Just add more types into it I guess?
Maybe you guys have tried others but met some bug or the loss exploded or anything.