cs249r_book/training.qmd

# ML Training

<!--

## Model Selection and Development
- Overview of ML Models 
- Criteria for Model Selection 
- Model Development Considerations in Embedded Systems
- Scalability and Resource Optimization

## Hyperparameter Tuning
- Understanding Hyperparameters 
- Techniques for Hyperparameter Tuning 
- Tuning for Embedded Systems 
- Grid Search and Randomized Search Methods

## Limited training data - transfer learning
## Federated learning
## 

-->

## Introduction

- Importance of ML Training
- Overview of ML Training Process

## Data Preparation

- Data Collection 
- Data Cleaning
- Data Augmentation
- Feature Engineering
- Splitting the Data (Training, Validation, and Test Sets)

## Model Selection

- Overview of Model Types 
- Criteria for Model Selection
- Model Complexity and Capacity 

## Training Algorithms

- Gradient Descent
  - Batch Gradient Descent
  - Stochastic Gradient Descent
  -  Mini-Batch Gradient Descent
- Optimization Algorithms
  - Adam
  - RMSprop
- Momentum

## Loss Functions
- Mean Squared Error (MSE)
- Cross-Entropy Loss
- Huber Loss
- Custom Loss Functions

## Regularization Techniques
- L1 and L2 Regularization
- Dropout
- Batch Normalization
- Early Stopping

## Model Evaluation
- Evaluation Metrics 
  - Accuracy
  - Precision and Recall
  - F1-Score
- Confusion Matrix
- ROC and AUC

## Hyperparameter Tuning
- Grid Search
- Random Search
- Bayesian Optimization

## Scaling Up Training
- Parallel Training
- Distributed Training
- Training with GPUs

## Model Cards

## Conclusion