57 What a SHAP value says and what it does not

What people usually ask

Use SHAP to explain which variables drive the prediction.

What comes back

A beeswarm plot and a paragraph that reads each colour band as a cause of the measured outcome.

Ask this instead

I have drying_runs.csv (31 runs: inlet_temp, air_flow, bed_depth,
feed_moisture, target final_moisture) and the fitted gradient
boosting pipeline in model.ipynb.
First print the cross validated R2 and RMSE of the model next to
a mean predictor. If the model is not clearly better than the
mean, stop; there is nothing to explain.
Then compute SHAP values on the training rows and show the
beeswarm plot.
Under the plot, write what the values are: for one run, the
shares sum to that run's model prediction minus the mean
prediction. They do not sum to the measured final_moisture.
Print the correlation matrix of the four inputs and name every
pair above 0.6. For those pairs say that the plot cannot tell
their shares apart.
Beside the plot, list the four readings it does not support:
that a high ranked input causes the outcome; that changing it in
the dryer would move final_moisture by the shown amount; that a
low ranked input does not matter in the process; that the
ranking holds outside the training range.

Attach: drying_runs.csv and model.ipynb with the fitted model.

Why it works: The plot arrives with its definition and its limits beside it, so it is read as a description of the model and not of the dryer.