Lessons / Part 8. Machine learning on small data in plain terms
46 What a flexible model adds on a small table
A straight line and a mean predictor scored under the same folds show whether the forest earned its score.
What people usually ask
Use machine learning to find the relationship in my data. Table attached.
What comes back
A random forest with a score of 0.91 in one cell, and nothing that says whether a straight line through the same points would have scored the same.
Ask this instead
I have pellets.csv: 36 tablet press runs, columns binder_pct, pressure_MPa, moisture_pct and crush_strength_N. Work in a notebook, one step per cell, a print after each.
1. Scatter plot of crush_strength_N against each input, three panels.
2. LinearRegression on all three inputs. Print the equation with its coefficients and R2.
3. RandomForestRegressor on the same inputs, defaults. Use the same KFold(5, shuffle=True, random_state=0) for both models.
4. A DummyRegressor that predicts the mean, scored the same way. Report one table: model, cross validated R2 mean, RMSE mean, for all three.
5. Partial dependence plots for the forest, one per input, so the shape of each relationship can be read.
6. Print the min and max of every input. Any prediction outside that box is refused, not reported.
Do not drop rows. Do not tune anything on the held out folds. If the forest does not beat the line by more than the spread across folds, say so in one line.
Attach: The run table as CSV, with the column names used in the prompt.
Why it works: The line and the mean predictor sit in the same table under the same folds, so the forest’s score has something to be compared with.
Drafted with Fable 5.1 from my notes; chosen and edited by me.