diff --git a/website/data/category_intros/machine-learning.md b/website/data/category_intros/machine-learning.md index ac52f790..9f469c14 100644 --- a/website/data/category_intros/machine-learning.md +++ b/website/data/category_intros/machine-learning.md @@ -13,6 +13,7 @@ How to choose: - Many series on classical models like ARIMA and ETS: StatsForecast - Forecasting next to classification and other time-series tasks: sktime - Forecasts with no model to train: TimesFM +- Tuning a model's hyperparameters: Optuna scikit-learn does [supervised and unsupervised learning](https://scikit-learn.org/stable/getting_started.html), plus the tools around it: preprocessing, model selection, and evaluation. Every estimator learns with `fit`. Put your preprocessing and model in one Pipeline, and run cross-validation and searches on that, since the pipeline [keeps test data out of training](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage). You may not need a separate boosting library: its histogram-based gradient boosting is [inspired by LightGBM](https://scikit-learn.org/stable/modules/ensemble.html) and supports missing values and categorical data natively. Deep learning is another matter: it [doesn't fit scikit-learn's design constraints](https://scikit-learn.org/stable/faq.html#why-is-there-no-support-for-deep-or-reinforcement-learning-will-there-be-such-support-in-the-future), so it's out of scope. @@ -34,4 +35,6 @@ sktime gives forecasting, classification, clustering, and anomaly detection [one TimesFM is [a pretrained foundation model from Google Research](https://github.com/google-research/timesfm) for time-series forecasting. It forecasts [zero-shot](https://research.google/blog/a-decoder-only-foundation-model-for-time-series-forecasting/), with no training on your data: load a checkpoint and call `predict` with a horizon. Check the license of the weights you download: some are [for non-commercial, non-production use only](https://github.com/google-research/timesfm#license-notice-for-pretrained-weights). +Optuna is [an automatic hyperparameter optimization framework, particularly designed for machine learning](https://optuna.readthedocs.io/en/latest/). Write an `objective` function that takes a `trial`, picks each hyperparameter with methods like `suggest_float`, trains the model, and returns its score. Then [create a study and call `optimize`](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html) with the number of trials. The search space is [plain Python, with conditionals and loops](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html), so you can tune an SVM's `C` only in trials that picked an SVM. To stop unpromising trials early, [call `report` and `should_prune`](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners) after each training step. + To evaluate a model on time-series data, test it on observations from after its training data. scikit-learn's [TimeSeriesSplit](https://scikit-learn.org/stable/modules/cross_validation.html#time-series-split) does that in cross-validation. For more, see [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning#python).