53 A bootstrap interval in a few lines

What people usually ask

Give me a confidence interval on the model score.

What comes back

The same single score restated with the words “high confidence” and no second number anywhere.

Ask this instead

I have extraction_runs.csv: 34 runs, columns temp_C, time_min,
solvent_ratio and yield_pct. The model is the random forest
pipeline in fit.ipynb, 5 fold cross validation, R2 0.71.
Bootstrap that score. Resample the 34 rows with replacement 1000
times. Refit the same pipeline on each resample and score it on
the rows that resample left out. Collect the 1000 scores.
Report the 2.5th and 97.5th percentiles as the interval, with the
median, in one line. Plot a histogram of the scores with the
original 0.71 marked. Fix the random seed and print it.
Print the count of resamples that could not be scored and skip
them; do not fill them in.
Add one sentence: an interval from 34 rows tends to come out
narrower than the truth, and by how much cannot be known from
this table.
Do not run any condition that is not in the table and do not
predict outside the range of any column.

Attach: extraction_runs.csv and fit.ipynb.

Why it works: The interval is defined by a procedure the reader can rerun, and its known weakness at this size is written next to it instead of being left out.