Step 5 — Evaluate Performance

Phase: Machine Learning

“A model is only useful if we can trust it. Evaluation is how we decide whether the model is ready for production — or back to the drawing board.”

5a. Metrics

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

import numpy as np

def evaluate(model, X_test, y_test, name):

    preds = model.predict(X_test)

    mae   = mean_absolute_error(y_test, preds)

    rmse  = np.sqrt(mean_squared_error(y_test, preds))

    r2    = r2_score(y_test, preds)

    print(f'{name:25s}  MAE=${mae:,.0f}  RMSE=${rmse:,.0f}  R2={r2:.3f}’)

 

evaluate(lr,  X_test, y_test, ‘Linear Regression’)

evaluate(rf,  X_test, y_test, ‘Random Forest’)

evaluate(gbr, X_test, y_test, ‘Gradient Boosting’)

 

Typical results:

💡 Translating for your CEO: Our Random Forest model is wrong by about $540 on average. Given that diamonds range from $326 to $18,823, that’s a relative error of around 8–10% — good enough to power a pricing tool for mid-range inventory.

5b. Feature Importance

importances = pd.Series(rf.feature_importances_, index=features).sort_values()

importances.plot(kind=’barh’)

plt.title(‘Feature Importance — Random Forest’)

 

Expected ranking: carat will be by far the most important feature, followed by clarity_enc, color_enc, and cut_enc. This confirms our EDA finding and validates the model.

5c. Residual Plot

preds     = rf.predict(X_test)

residuals = y_test – preds

 

plt.scatter(preds, residuals, alpha=0.2)

plt.axhline(0, color=’red’, linewidth=1)

plt.xlabel(‘Predicted Price’)

plt.ylabel(‘Residual (Actual – Predicted)’)

plt.title(‘Residual Plot’)

Ideally, residuals should be randomly scattered around zero. Any pattern (like a fan shape widening at high prices) tells you the model is systematically wrong somewhere.