From 49daf78664de94eca40ee7eaebb433ce902397e7 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:13:28 +0800 Subject: [PATCH 1/5] style: audit WebSocket General held two jobs, so it splits into Servers and Clients (websockets, autobahn-python) and Framework Integrations (channels, flask-socketio), next to WebRTC. Co-Authored-By: Claude --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 27eb53b8..800f0030 100644 --- a/README.md +++ b/README.md @@ -323,11 +323,12 @@ _ASGI and WSGI compatible web servers._ _Libraries for working with WebSocket._ -- General +- Servers and Clients - [websockets](https://github.com/python-websockets/websockets) - A library for building WebSocket servers and clients with a focus on correctness and simplicity. + - [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and [asyncio](https://docs.python.org/3/library/asyncio.html). +- Framework Integrations - [channels](https://github.com/django/channels) - Brings WebSocket, long-poll HTTP, and other async support to Django. - [flask-socketio](https://github.com/miguelgrinberg/Flask-SocketIO) - Socket.IO integration for Flask applications. - - [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and [asyncio](https://docs.python.org/3/library/asyncio.html). - WebRTC - [aiortc](https://github.com/aiortc/aiortc) - A WebRTC and ORTC implementation for asyncio, with audio, video, and data channels. From cbbbb86d19a61927d47c405423bc7971f6448522 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:13:28 +0800 Subject: [PATCH 2/5] style: audit Machine Learning pgmpy does causal and probabilistic reasoning with graphical models, closer to statistics than to the scikit-learn ecosystem in General, so it moves to Science > Statistics after statsmodels. Co-Authored-By: Claude --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 800f0030..ed9177a3 100644 --- a/README.md +++ b/README.md @@ -213,7 +213,6 @@ _Libraries for machine learning. Also see [awesome-machine-learning](https://git - General - [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for machine learning with extensive documentation and community support. - [imbalanced-learn](https://github.com/scikit-learn-contrib/imbalanced-learn) - Resampling techniques for imbalanced datasets, compatible with scikit-learn. - - [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for causal and probabilistic reasoning with graphical models. - [feature-engine](https://github.com/feature-engine/feature_engine) - sklearn-compatible API with the widest toolset for feature engineering and selection. - Gradient Boosting - [xgboost](https://github.com/dmlc/xgboost) - A scalable, portable, and distributed gradient boosting library. @@ -599,6 +598,7 @@ _Libraries for scientific computing. Also see [Python-for-Scientists](https://gi - [sympy](https://github.com/sympy/sympy) - A Python library for symbolic mathematics. - Statistics - [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. + - [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for causal and probabilistic reasoning with graphical models. - Biology and Chemistry - [rdkit](https://github.com/rdkit/rdkit) - Cheminformatics and machine learning software. - [biopython](https://github.com/biopython/biopython) - Biopython is a set of freely available tools for biological computation. From 594ab2bf8bbb6fc5db91acc99adc0cdc3e75bb3e Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:13:31 +0800 Subject: [PATCH 3/5] docs: add aiortc to WebSocket intro Co-Authored-By: Claude --- website/data/category_intros/websocket.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/website/data/category_intros/websocket.md b/website/data/category_intros/websocket.md index ed95128c..43c14df8 100644 --- a/website/data/category_intros/websocket.md +++ b/website/data/category_intros/websocket.md @@ -7,6 +7,7 @@ How to choose: - Plain WebSocket servers and clients: websockets - A few real-time features next to a Django project: websockets as a separate server - RPC and pub/sub over WAMP, or a Twisted app: Autobahn|Python +- Real-time audio, video, and data channels over WebRTC: aiortc Channels [extends Django beyond HTTP](https://channels.readthedocs.io/en/latest/) to handle WebSockets, and it [integrates with Django's auth and sessions](https://channels.readthedocs.io/en/latest/introduction.html). Write [sync consumers by default](https://channels.readthedocs.io/en/latest/topics/consumers.html#basic-layout). Switch to async ones only when async handling helps and every library you call is async-native. Serve everything with Daphne, or [keep HTTP on your WSGI server](https://channels.readthedocs.io/en/latest/deploying.html#http-and-websocket) and send only WebSockets to Daphne. @@ -16,6 +17,8 @@ websockets runs on asyncio by default, which is [ideal for servers with many con Autobahn|Python implements [both WebSocket and WAMP, on Twisted or asyncio](https://autobahn.readthedocs.io/en/latest/). WAMP adds [RPC and pub/sub over WebSocket](https://github.com/crossbario/autobahn-python), and every WAMP client [needs a WAMP router to talk to](https://autobahn.readthedocs.io/en/latest/wamp/programming.html#wamp-programming-1). Write components with functions and decorators, [the recommended approach](https://autobahn.readthedocs.io/en/latest/wamp/programming.html#creating-components-1). +aiortc implements [WebRTC and ORTC on asyncio](https://aiortc.readthedocs.io/en/latest/), with an API that closely follows the JavaScript one: promises become coroutines. Since it's plain Python, one server can [handle both signaling and data channels](https://aiortc.readthedocs.io/en/latest/#why-should-i-use-aiortc), or run computer vision on incoming video frames. + Always [secure WebSocket connections with TLS](https://websockets.readthedocs.io/en/stable/topics/security.html#encryption) in production. Any site can open a WebSocket to yours, with your users' cookies attached. If you serve private data, [restrict the allowed origins](https://channels.readthedocs.io/en/latest/topics/security.html#websockets): Channels has `AllowedHostsOriginValidator`, websockets has the [`origins` argument](https://websockets.readthedocs.io/en/stable/reference/asyncio/server.html#websockets.asyncio.server.serve), and Flask-SocketIO [allows only the same origin by default](https://flask-socketio.readthedocs.io/en/latest/deployment.html#cross-origin-controls). Broadcasting across processes needs a message broker such as Redis. Channels' [production channel layer](https://channels.readthedocs.io/en/latest/topics/channel_layers.html#redis-channel-layer) runs on it, Flask-SocketIO [takes it as a message queue](https://flask-socketio.readthedocs.io/en/latest/deployment.html#using-multiple-workers), with sticky sessions at the load balancer, and websockets' docs [suggest it for pub/sub](https://websockets.readthedocs.io/en/stable/faq/server.html#how-do-i-send-a-message-to-a-channel-a-topic-or-some-users). From 86bbbb5d9602c92d2b3b773ad3563a308db36f53 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:13:31 +0800 Subject: [PATCH 4/5] docs: swap pgmpy for MLflow in Machine Learning intro Co-Authored-By: Claude --- website/data/category_intros/machine-learning.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/data/category_intros/machine-learning.md b/website/data/category_intros/machine-learning.md index 01d18bd7..6652b4e9 100644 --- a/website/data/category_intros/machine-learning.md +++ b/website/data/category_intros/machine-learning.md @@ -4,7 +4,6 @@ How to choose: - Classification, regression, and clustering on tabular data: scikit-learn - Boosted trees with no extra dependency: scikit-learn -- Causal graphs and Bayesian networks: pgmpy - Feature engineering that keeps your DataFrame's column names: Feature-engine - Boosted trees trained across a cluster: XGBoost - Boosted trees on large data, fast and light on memory: LightGBM @@ -14,11 +13,10 @@ How to choose: - Forecasting next to classification and other time-series tasks: sktime - Forecasts with no model to train: TimesFM - Tuning a model's hyperparameters: Optuna +- Runs, parameters, and metrics of every training job, tracked: MLflow scikit-learn does [supervised and unsupervised learning](https://scikit-learn.org/stable/getting_started.html), plus the tools around it: preprocessing, model selection, and evaluation. Every estimator learns with `fit`. Put your preprocessing and model in one Pipeline, and run cross-validation and searches on that, since the pipeline [keeps test data out of training](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage). You may not need a separate boosting library: its histogram-based gradient boosting is [inspired by LightGBM](https://scikit-learn.org/stable/modules/ensemble.html) and supports missing values and categorical data natively. Deep learning is another matter: it [doesn't fit scikit-learn's design constraints](https://scikit-learn.org/stable/faq.html#why-is-there-no-support-for-deep-or-reinforcement-learning-will-there-be-such-support-in-the-future), so it's out of scope. -pgmpy does [causal and probabilistic reasoning with graphical models](https://pgmpy.org/). Its [quickstart](https://pgmpy.org/started/quickstart.html) follows the tasks: learn a graph's structure from data, fit the distributions for a graph you know, then query the model for probabilities or causal effects. - Feature-engine is [designed to work with dataframes](https://feature-engine.trainindata.com/en/latest/): a DataFrame goes in, and the same DataFrame comes out, with no column order or name changes. Instead of wrapping transformers in a ColumnTransformer, you tell each one which variables to transform. Its transformers [work just like any scikit-learn transformer](https://feature-engine.trainindata.com/en/latest/quickstart/index.html), so they fit in a Pipeline. XGBoost is [an optimized distributed gradient boosting library](https://xgboost.readthedocs.io/en/stable/). Its Python package has [a native interface, a scikit-learn interface, and a Dask interface](https://xgboost.readthedocs.io/en/stable/python/python_intro.html). @@ -37,4 +35,6 @@ TimesFM is [a pretrained foundation model from Google Research](https://github.c Optuna is [an automatic hyperparameter optimization framework, particularly designed for machine learning](https://optuna.readthedocs.io/en/latest/). Write an `objective` function that takes a `trial`, picks each hyperparameter with methods like `suggest_float`, trains the model, and returns its score. Then [create a study and call `optimize`](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/001_first.html) with the number of trials. The search space is [plain Python, with conditionals and loops](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/002_configurations.html), so you can tune an SVM's `C` only in trials that picked an SVM. To stop unpromising trials early, [call `report` after each training step and raise `optuna.TrialPruned()` when `should_prune()` returns true](https://optuna.readthedocs.io/en/latest/tutorial/10_key_features/003_efficient_optimization_algorithms.html#activating-pruners). +MLflow groups [runs and models into experiments](https://mlflow.org/docs/latest/ml/tracking/), one per task. Call [`mlflow.autolog()`](https://mlflow.org/docs/latest/ml/tracking/autolog/) before your training code, and it logs metrics, parameters, and models from libraries like scikit-learn, XGBoost, and PyTorch with no explicit log statements. Log anything else with `mlflow.log_param()` and `mlflow.log_metric()`, then run `mlflow server` to compare runs in its UI. + To evaluate a model on time-series data, test it on observations from after its training data. scikit-learn's [TimeSeriesSplit](https://scikit-learn.org/stable/modules/cross_validation.html#time-series-split) does that in cross-validation. For more, see [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning#python). From 4e1d12ffe83f6777ca8f9c76edf957e254ea5647 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:13:31 +0800 Subject: [PATCH 5/5] docs: add pgmpy to Science intro Co-Authored-By: Claude --- website/data/category_intros/science.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/website/data/category_intros/science.md b/website/data/category_intros/science.md index 87f70c05..1443df01 100644 --- a/website/data/category_intros/science.md +++ b/website/data/category_intros/science.md @@ -6,7 +6,7 @@ How to choose: - Optimization, integration, interpolation, and linear algebra: SciPy - Numeric loops that won't vectorize: Numba - Exact, symbolic math: SymPy -- Regression, time series, and hypothesis tests: statsmodels +- Regression, time series, and hypothesis tests: statsmodels; causal graphs and Bayesian networks: pgmpy - Biological sequences and file formats: Biopython; molecules: RDKit - Physical units: Pint; astronomy: Astropy; seismology: ObsPy - Bayesian models: PyMC @@ -25,6 +25,8 @@ SymPy does symbolic math: expressions stay [exact, not approximate](https://docs statsmodels [estimates statistical models, runs hypothesis tests, and explores data](https://www.statsmodels.org/stable/index.html), and [most of its results are verified](https://www.statsmodels.org/stable/about.html#testing) against another statistical package. SciPy's stats docs [send regression, linear models, and time series analysis to statsmodels](https://docs.scipy.org/doc/scipy/reference/stats.html), while `scipy.stats` keeps the probability distributions and statistical tests. Write your model as an [R-style formula](https://www.statsmodels.org/stable/example_formulas.html) on a pandas DataFrame, like `smf.ols("y ~ x", data=df).fit()`, and read the fit with `summary()`. +pgmpy does [causal and probabilistic reasoning with graphical models](https://pgmpy.org/). Its [quickstart](https://pgmpy.org/started/quickstart.html) follows the tasks: learn a graph's structure from data, fit the distributions for a graph you know, then query the model for probabilities or causal effects. + Biopython [parses bioinformatics file formats](https://biopython.org/docs/latest/Tutorial/chapter_introduction.html) like FASTA and GenBank, queries online services like NCBI, and gives you a standard sequence class. Read sequence files with `Bio.SeqIO.parse()`, or [`Bio.SeqIO.read()` when a file holds one record](https://biopython.org/docs/latest/Tutorial/chapter_seqio.html#parsing-or-reading-sequences). When you query NCBI through `Bio.Entrez`, [pass your email](https://biopython.org/docs/latest/Tutorial/chapter_entrez.html#entrez-guidelines) so NCBI can contact you if there's a problem. RDKit is a [cheminformatics toolkit with its core in C++](https://www.rdkit.org/docs/Overview.html), for 2D and 3D molecular operations and descriptors for machine learning. To compare molecules, [create a fingerprint generator](https://www.rdkit.org/docs/GettingStartedInPython.html#fingerprinting-and-molecular-similarity) for the fingerprint type you want, the docs' most consistent way to get fingerprints. Pint handles [physical quantities](https://pint.readthedocs.io/en/stable/getting/overview.html): a value times a unit, in any numeric type. In a package, [create one `UnitRegistry` in a single place](https://pint.readthedocs.io/en/stable/getting/pint-in-your-projects.html#having-a-shared-registry) and import it everywhere, since quantities from different registries don't mix. Astropy is [a common core package for astronomy](https://www.astropy.org/), with affiliated packages around it. [Import the subpackage you need](https://docs.astropy.org/en/stable/importing_astropy.html#importing-astropy-and-sub-packages), like `from astropy import units as u`, and never import with `*`. ObsPy is [a framework for processing seismological data](https://docs.obspy.org/): `read()` loads [SAC, MiniSEED, and other formats into a Stream](https://docs.obspy.org/tutorial/code_snippets/reading_seismograms.html) of Traces, and its [FDSN client](https://docs.obspy.org/packages/obspy.clients.fdsn.html#basic-fdsn-client-usage) fetches data from data centers.