Skip to content
tishad.net
SearchCtrl K

Lessons / Part 8. Machine learning on small data in plain terms

50 A hyperparameter search in plain terms

A resumable study with a trial budget, settings read as text first, and the quoted score from an outer loop the search never saw.

What people usually ask

Optimize the hyperparameters of my model.

What comes back

A grid over every parameter, scored on the same rows the final number is quoted from, with nothing saved when the kernel dies.

Ask this instead

I have gran.csv: 48 granulation runs, inputs liquid_pct, impeller_rpm, time_min, target d50_um, and settings.csv with columns name, value, one row per hyperparameter.
1. Load settings.csv with dtype=str. Print each name, its value and type(value). Cast each one on purpose and print the type again, so a size written as "10,10" never becomes the number 1010.
2. An Optuna study: sampler TPESampler(seed=0), storage="sqlite:///study.db", study_name="gran_rf", load_if_exists=True, direction="minimize". Objective: mean RMSE of a RandomForestRegressor over an inner KFold(5) on the training rows only. Search n_estimators 50 to 500, max_depth 2 to 12, min_samples_leaf 1 to 8. Budget: 60 trials.
3. Print each trial's parameters and value as it completes. If the run stops, rerunning the cell resumes from the database.
4. Report the best parameters beside the scikit-learn defaults, with both RMSEs.
5. The score that gets quoted comes from an outer KFold(5) the search never saw: run the search inside each outer fold and report the outer RMSE mean and spread.
Do not touch the outer test rows during the search. Do not drop rows.

Attach: The run table and the settings file, both as CSV.

Why it works: The budget and the database make the search finite and resumable, and the outer loop keeps the quoted score apart from the rows the search tuned on.

Drafted with Fable 5.1 from my notes; chosen and edited by me.