Originally created by @jaywonchung on GitHub (Dec 16, 2023).
First of all, thanks for the great resource.
I'm part of a group of people working on ML energy optimization at UMich (https://ml.energy). To list a couple works:
Training
Zeus paper (NSDI ’23): Understanding the trade-off between training time and energy consumption and proposing an online optimization algorithm.
Zeus open-source: A generic DNN energy measurement and optimization framework.
Perseus: Energy reduction without slowdown for large model training. arXiv, open-source
Inference
ML.ENERGY leaderboard: The first public leaderboard on the energy consumption of LLM text generation.
Chapter 17.3 Energy Consumption does a very nice survey on efforts for energy estimation, measurement, and reporting. I'm thinking of adding a paragraph on energy optimization. Please let me know what you think.
Originally created by @jaywonchung on GitHub (Dec 16, 2023).
First of all, thanks for the great resource.
I'm part of a group of people working on ML energy optimization at UMich (https://ml.energy). To list a couple works:
Training
- [Zeus paper (NSDI ’23)](https://www.usenix.org/conference/nsdi23/presentation/you): Understanding the trade-off between training time and energy consumption and proposing an online optimization algorithm.
- [Zeus open-source](https://ml.energy/zeus): A generic DNN energy measurement and optimization framework.
- Perseus: Energy reduction without slowdown for large model training. [arXiv](https://arxiv.org/abs/2312.06902), [open-source](https://ml.energy/zeus/perseus)
Inference
- [ML.ENERGY leaderboard](https://ml.energy/leaderboard): The first public leaderboard on the energy consumption of LLM text generation.
Chapter 17.3 Energy Consumption does a very nice survey on efforts for energy estimation, measurement, and reporting. I'm thinking of adding a paragraph on energy *optimization*. Please let me know what you think.
Or I think having a new subsection (17.XX) dedicated for energy optimization would make sense, since I believe that ultimately one of the important goals of MLSys is to optimize energy consumption, going beyond estimation and reporting.
@jaywonchung commented on GitHub (Dec 16, 2023):
Or I think having a new subsection (17.XX) dedicated for energy optimization would make sense, since I believe that ultimately one of the important goals of MLSys is to optimize energy consumption, going beyond estimation and reporting.
Hi @jaywonchung that's a great point, thanks for raising that. Please definitely go ahead and add a paragraph and in general it would be great to talk about the space broadly and of course we can include a reference to the point that Zeus is making.
Can you please draft what you have in mind and issue a PR? Would be happy to review it and make any edits and merge it in.
@profvjreddi commented on GitHub (Dec 16, 2023):
Hi @jaywonchung that's a great point, thanks for raising that. Please definitely go ahead and add a paragraph and in general it would be great to talk about the space broadly and of course we can include a reference to the point that Zeus is making.
Can you please draft what you have in mind and issue a PR? Would be happy to review it and make any edits and merge it in.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @jaywonchung on GitHub (Dec 16, 2023).
First of all, thanks for the great resource.
I'm part of a group of people working on ML energy optimization at UMich (https://ml.energy). To list a couple works:
Training
Inference
Chapter 17.3 Energy Consumption does a very nice survey on efforts for energy estimation, measurement, and reporting. I'm thinking of adding a paragraph on energy optimization. Please let me know what you think.
@jaywonchung commented on GitHub (Dec 16, 2023):
Or I think having a new subsection (17.XX) dedicated for energy optimization would make sense, since I believe that ultimately one of the important goals of MLSys is to optimize energy consumption, going beyond estimation and reporting.
@profvjreddi commented on GitHub (Dec 16, 2023):
Hi @jaywonchung that's a great point, thanks for raising that. Please definitely go ahead and add a paragraph and in general it would be great to talk about the space broadly and of course we can include a reference to the point that Zeus is making.
Can you please draft what you have in mind and issue a PR? Would be happy to review it and make any edits and merge it in.
@profvjreddi commented on GitHub (Dec 16, 2023):
If you are interested, then create a branch off of this PR.