mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-11-09 10:39:41 -06:00
Shawn Hymel's feedback #11
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/cs249r_book#11
Reference in New Issue
Block a user
Originally created by @profvjreddi on GitHub (Oct 14, 2023).
3.1.2. Add mention of Rina Dechter coining the term “deep learning” - https://web.archive.org/web/20160419054654/https://www.researchgate.net/publication/221605378_Learning_While_Searching_in_Constraint-Satisfaction-Problems
3.1.3 Add mention of LLMs due to their current hype
3.2 Add links out to places where readers can dive into the different architecture types if they want more details. Can be original papers, blog posts, YouTube explainers, etc. A great book for this kind of in-depth overview of ML is the “Hands-On Machine Learning” by Aurélien Géron.
3.2 Maybe link to a very simple Jupyter Notebook demonstrating a classification task (e.g. CNN trained on MNIST using TensorFlow or PyTorch)
4 Careful with the stylization of “TinyML,” as the TinyML Foundation can be protective of this term (we’ve gotten letters from them at Edge Impulse). On TinyML.org: “tinyML® and the tinyML logo are trademarks of tinyML Foundation.” My recommendation is to always space “Tiny ML” unless you’re talking about the TinyML Foundation.
4.3-4.5 The various descriptions of “Edge ML” and “Tiny ML” are a little confusing. The chart in 4.1 shows Edge AI as a continuum and Tiny ML as a part of Edge AI. Sections 4.3, 4.4, and 4.5 make it seem that Edge ML and Tiny ML are separate things. What’s the difference between Edge AI and Edge ML (if any)? Is Tiny ML a subset of Edge ML, or is it distinct and separate from Edge ML?
5.1 I recommend adding a section on “feature extraction” in the pipeline. There’s data cleaning and curation listed, but many edge ML applications (especially those that deal with time series data) require manual feature extraction, such as FFT, MFCC, etc.
5.3 Super useful section! We often debate about who exactly our audience is when discussing marketing, content, and tool development.
6.3 Definitely need a section on manual collection and what’s involved, eg writing or using software/firmware to log live data from sensors.
6.3 If you talk about the specifics of storing and compressing audio data, you’ll probably want to talk about other data types, like images. I might recommend simply providing an overview of what compression is and mention it can be useful for types like audio and images. From there, reference readers to external sources to dive deeper. Also, this might be a good place to talk about metadata, and how that can be useful for assisting with labeling or providing context for the data in question.
Note for above ^^^ while we’re used to seeing time-series data as the main focus area for Tiny ML, I’m starting to see a lot of interest in wanting to do power-efficient and low-latency object detection. With the new accelerators hitting the market, I expect full object detection solutions on MCU targets (and/or accelerators) to be widely available soon, which includes in the classrooms. I’m currently tinkering with the ESP32S3, and I know you picked up the Nicla Vision boards. Both are very promising for object detection.
6.5 need to cite source for the quote “Data preparation accounts for about 60-80% of the work of a data scientist.”
6.5 I feel that some of the data prep is glossed over and could use some deeper explanation. It took me a while to understand why normalization, categorical encoding, and dimensionality reduction are so important in ML. IIRC, Géron spends a chapter on each of those in his book. At the very least, send readers to an external site, book, or video to learn more. The best would be a hands-on example (eg notebook) that demonstrates why each of these are important.
Note for above ^^^ if you’re having students flesh out the book as part of their course work, this might be a great solo or group project in a project-based learning environment: have each student or team create a Jupyter Notebook demonstrating a concept, write a subsection on it for the book, do a PR to the repo with their edits and notebook, and give a presentation to the class on their topic/demo.
6.8 this section should probably mention the importance of ensuring that the data used to train the model should generally have the exact same format (eg width, height for images, number of MFCCs) as the data you plan to pass in from your sensor or feature extractor. For example, images collected from a phone look different than those collected from the Pi NoIR. A model trained on one will struggle on inference when dealing with data from the other. The exception to the resolution is when working with some architectures, eg MobileNet, that simply convolve or pool pixels such that the output is always the same dimension regardless of input (within reason).
6 I think more on biases (and the different types of bias) should be covered in this section. Bias is referenced a few times, but no strong definition is given. Again, a notebook showing bias or a few anecdotes about bias gone wrong would be very helpful.
7 I imagine this is going to be a continually evolving section as frameworks come and go
8.6 I’ve been diving into hyperparameter optimization recently, and I think discussing the various methods (eg grid search, random search, Bayesian optimization) would work well here.
8.8 Add part on how to identify over/underfitting.
9.2 Is BLERP still a useful acronym? Or has the academic community moved beyond that?
9.3 Probably worth mentioning some of the efficient YOLO variants here. Also, this might be a good place to discuss LSTM, as they seem to be very efficient with regards to time-series data. Linking to one (or more) of the overview papers that discusses model accuracy vs speed would be helpful.
9.5 there probably needs to be another section added for general-purpose AI acceleration (ie optimized hardware instructions for NN ops, matrix ops, etc), such as the Arm U55, U65?, and U85 architectures. I propose creating two sections: one that discusses fixed-function accelerators (ie those built for a single model architecture) and general-purpose accelerators (ie optimized for specific operations and can handle a wide variety of tensor/ML ops). Then, place the TPU under the “general-purpose accelerator” section.
9.6.2 I would specifically mention how a floating point unit (FPU) can make a huge difference in floating point computation speed and efficiency.
10.3 Mention the downsides of pruning. If I recall, pruning often requires ragged/non-uniform (can’t remember the exact name—I’m on a plane), which many tiny ML frameworks don’t support. Even if support is there, you often lose out on efficiency gains when using SIMD or other accelerated operations. It’s why Edge Impulse doesn’t support pruning (for now).
12.3 “Accuracy” is a whole can of worms that probably deserves its own separate section. There’s total accuracy and per-class accuracy for multi-class classification. There’s also true/false positives/negatives, precision, recall, and F1 scores to try to summarize the previous metrics. F1 scores can help identify naive classifiers, which basic “accuracy” won’t catch. ROC curves are also useful here. Then you should probably cover object detection metrics, such as mAP.
13.1 in the “Limitations” section of on-device learning, it might be useful to contrast on-device learning with remote data collection, server-side training, and model updates via OTA (which is probably covered in the embedded AIOps section).
14.4 I could see remote data collection being very useful here. For example, you want to create a baseline for a machine operation or background noise when you deploy to a new environment. That would require an automated ML pipe to collect data, train (or retrain) a model, and deploy the mode back to the device.
14 perhaps add a section on how IoT frameworks can assist with embedded AIOps? For example, standards like Matter and Zigbee can provide some security guarantees, new RTOSes (such as AWS-enables FreeRTOS and Zephyr) provide off-the-shelf connectivity functions, and fully managed IoT platforms, like Azure, Particle, and Golioth, offer remote data collection and OTA update capabilities.
16.8 A discussion of RAIL would probably be good here, as that seems to be a very popular framework.
17 I don’t think it’s a leap. Anything we see in big AI will likely make its way to embedded AI in 5-10 years. But, rather than just one chapter focused on generative AI, I might recommend making a more general “future of embedded ML” chapter that discusses current research/development in and potential edge/embedded use cases for things like generative AI, reinforcement learning, neuromorphic processing, and so on.