51 Leave one out validation, and what its score leaves out

What people usually ask

What is the R2 of my model with leave one out?

What comes back

A number averaged over folds of one row each, where R2 is undefined, printed as if it were a score.

Ask this instead

I have cat.csv: 24 catalyst activity tests, columns run_id, batch_id, temp_C, whsv and conversion_pct. Runs with the same batch_id are replicates of one catalyst batch.
Model: Ridge in a Pipeline with StandardScaler.
1. Use cross_val_predict with LeaveOneGroupOut and groups=batch_id, so replicates leave together and never sit on both sides of a fold. Collect the held out predictions into one column, pred_conv, beside run_id.
2. Compute R2 and RMSE once, on that column against conversion_pct. Do not average per fold scores.
3. Print the min and max of temp_C and whsv. State that the score covers points inside that range only and says nothing about any point outside it.
4. If the Ridge alpha was chosen using these same folds, say so, and do not quote the score as performance; it is a tuning score. If it was fixed in advance, say what it is.
Do not drop rows. Do not split replicates to get more folds.

Attach: The run table with run ids and a batch or replicate column.

Why it works: One column of held out predictions gives one score that is defined, and grouped folds stop a replicate from telling the model the answer to its twin.