Ensemble models¶
gwlearn implements geographically weighted versions of ensemble models for both classification and regression tasks. For classification, there’s a caveat: given it is unlikely that all categories are present in all local models, fitting a non-binary would lead to inconsistent local models. Hence gwlearn currently supports only binary classification.
This notebook outlines how to fit the models and extract their attributes. For details on prediction, see the Prediction guide.
import geopandas as gpd
from geodatasets import get_path
from sklearn import metrics
from gwlearn.ensemble import (
GWGradientBoostingClassifier,
GWGradientBoostingRegressor,
GWRandomForestClassifier,
GWRandomForestRegressor,
)
Get sample data
gdf = gpd.read_file(get_path("geoda.south")).to_crs(5070)
gdf["point"] = gdf.representative_point()
gdf = gdf.set_geometry("point")
y = gdf["FH90"] > gdf["FH90"].median() # binary target for classification
X = gdf.iloc[:, 9:15]
Random Forest Classifier¶
The implementation of geographically-weighted random forest classifier follows the logic of linear models, where each local neighborhood defined by a set bandwidth is used to fit a single local model.
gwrf = GWRandomForestClassifier(
bandwidth=150,
fixed=False,
)
gwrf.fit(
X,
y,
geometry=gdf.geometry,
)
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
GWRandomForestClassifier(bandwidth=150)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| bandwidth | 150 | |
| fixed | False | |
| kernel | 'bisquare' | |
| include_focal | False | |
| graph | None | |
| n_jobs | -1 | |
| fit_global_model | True | |
| strict | False | |
| keep_models | False | |
| temp_folder | None | |
| batch_size | None | |
| min_proportion | 0.2 | |
| undersample | False | |
| leave_out | None | |
| random_state | None | |
| coplanar | 'raise' | |
| verbose | False |
Fitted attributes
| Name | Type | Value |
|---|---|---|
| feature_importances_ | DataFrame | HR6...s x 6 columns] |
| feature_names_in_ | ndarray[object](6,) | ['HR60','HR70','HR80','HR90','HC60','HC70'] |
| local_class_support_ | Series[int64](1412,) | 0 2 1 ..., dtype: int64 |
| oob_pred_pooled_ | ndarray[int64](177150,) | [1,0,0,...,1,0,0] |
| oob_y_pooled_ | ndarray[float64](177150, 1) | [[0.], [1.], [0.], ..., [1.], [0.], [0.]] |
| pred_ | Series[boolean](1412,) | 0 False...dtype: boolean |
| prediction_rate_ | float64 | 0.8364 |
| proba_ | DataFrame | False ...s x 2 columns] |
Focal score¶
The performance of these models can be measured in three ways. The first one is using the focal prediction. Unlike in linear models, where the focal observation is part of the local model training, in this case it is excluded to ensure it can be used to evaluate the model. Otherwise, it would be a part of the data the local model has seen and would report unrealistically high performance. If you want to include the focal observation in the training nevertheless, just set include_focal=True.
Focal accuracy can be measured as follows. Note that some local models are not fitted due to imbalance rules, and report NA that needs to be filtered out.
na_mask = gwrf.pred_.notna()
metrics.accuracy_score(y[na_mask], gwrf.pred_[na_mask])
0.7662997459779848
Pooled out-of-bag score¶
Another option is to pull the out-of-bag predictions from individual local models and pool them together. This uses more data to evaluate the model but given the local models are tuned for their focal location, some predictions on locations far from the focal point may be artifically worse than the actual model prediction would be.
metrics.accuracy_score(gwrf.oob_y_pooled_, gwrf.oob_pred_pooled_)
0.7329494778436353
Local score¶
The final option is to take the local out-of-bag predicitons and measure performance per each local model. To do that, you can use a method local_metric().
local_accuracy = gwrf.local_metric(metrics.accuracy_score)
local_accuracy
array([0.62666667, 0.66 , 0.62666667, ..., 0.58 , 0.70666667,
0.68 ], shape=(1412,))
gdf.set_geometry("geometry").plot(
local_accuracy, legend=True, missing_kwds=dict(color="lightgray")
).set_axis_off()
Feature importance¶
Feature importances are reported for each local model.
gwrf.feature_importances_
| HR60 | HR70 | HR80 | HR90 | HC60 | HC70 | |
|---|---|---|---|---|---|---|
| 0 | 0.173866 | 0.173257 | 0.210367 | 0.210047 | 0.103940 | 0.128523 |
| 1 | 0.149851 | 0.166215 | 0.199482 | 0.249666 | 0.114214 | 0.120572 |
| 2 | 0.179945 | 0.153896 | 0.184103 | 0.255872 | 0.117738 | 0.108447 |
| 3 | 0.161548 | 0.171059 | 0.185708 | 0.234038 | 0.131072 | 0.116575 |
| 4 | 0.118702 | 0.158343 | 0.243771 | 0.216935 | 0.125143 | 0.137106 |
| ... | ... | ... | ... | ... | ... | ... |
| 1407 | 0.121638 | 0.178695 | 0.252081 | 0.271191 | 0.091792 | 0.084602 |
| 1408 | 0.185446 | 0.157277 | 0.201252 | 0.215212 | 0.125249 | 0.115565 |
| 1409 | 0.158372 | 0.173804 | 0.202224 | 0.215973 | 0.143715 | 0.105912 |
| 1410 | 0.150573 | 0.197277 | 0.212832 | 0.203509 | 0.103357 | 0.132451 |
| 1411 | 0.138652 | 0.174034 | 0.226490 | 0.218264 | 0.135243 | 0.107317 |
1412 rows × 6 columns
gdf.set_geometry("geometry").plot(
gwrf.feature_importances_["HC60"], legend=True
).set_axis_off()
You can compare all of this to values extracted from a global model, fitted alongside.
gwrf.global_model.feature_importances_
array([0.14757778, 0.15126896, 0.22430459, 0.2478163 , 0.11743271,
0.11159968])
gwrf.feature_importances_.mean()
HR60 0.153245
HR70 0.158751
HR80 0.202698
HR90 0.222850
HC60 0.141146
HC70 0.121308
dtype: float64
Random Forest Regressor¶
The GWRandomForestRegressor provides the same functionality as the classifier version but for continuous target variables. It uses local Random Forest models to capture spatial heterogeneity in regression relationships.
y_reg = gdf["FH90"]
gwrf_reg = GWRandomForestRegressor(
bandwidth=150,
fixed=False,
random_state=0,
)
gwrf_reg.fit(
X,
y_reg,
geometry=gdf.representative_point(),
)
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
/home/runner/micromamba/envs/py314-latest/lib/python3.14/site-packages/sklearn/ensemble/_forest.py:589: UserWarning: Some inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.
warn(
GWRandomForestRegressor(bandwidth=150, random_state=0)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| bandwidth | 150 | |
| random_state | 0 | |
| fixed | False | |
| kernel | 'bisquare' | |
| include_focal | False | |
| graph | None | |
| n_jobs | -1 | |
| fit_global_model | True | |
| strict | False | |
| keep_models | False | |
| temp_folder | None | |
| batch_size | None | |
| coplanar | 'raise' | |
| verbose | False |
Fitted attributes
| Name | Type | Value |
|---|---|---|
| RSS_ | Series[float64](1412,) | focal 0 ...dtype: float64 |
| TSS_ | Series[float64](1412,) | 0 277....dtype: float64 |
| feature_importances_ | DataFrame | HR6...s x 6 columns] |
| local_r2_ | Series[float64](1412,) | 0 0.081...dtype: float64 |
| oob_pred_pooled_ | ndarray[float64](211800,) | [11.63,13.76, 0. ,...,24.68,12.92,14.26] |
| oob_y_pooled_ | ndarray[float64](211800, 1) | [[11.24], [17.57], [13.56], ..., [17.06], [12.11], [ 9.83]] |
| pred_ | Series[float64](1412,) | 0 12.00...dtype: float64 |
| resid_ | Series[float64](1412,) | 0 0.601...dtype: float64 |
| y_bar_ | Series[float64](1412,) | 0 12.92...dtype: float64 |
The regressor provides local R² values showing how well the model fits in each neighborhood:
gwrf_reg.local_r2_.head()
0 0.081027
1 -0.003088
2 -0.085531
3 -0.095847
4 0.638388
dtype: float64
gdf.set_geometry("geometry").plot(gwrf_reg.local_r2_, legend=True).set_axis_off()
Out-of-bag performance¶
Like the classifier, the regressor supports out-of-bag predictions that can be used to evaluate model performance:
from sklearn.metrics import r2_score
r2_score(gwrf_reg.oob_y_pooled_, gwrf_reg.oob_pred_pooled_)
0.454453697844472
Feature importance¶
Feature importances show the contribution of each feature in local models:
gwrf_reg.feature_importances_.head()
| HR60 | HR70 | HR80 | HR90 | HC60 | HC70 | |
|---|---|---|---|---|---|---|
| 0 | 0.119112 | 0.176089 | 0.221701 | 0.266632 | 0.051806 | 0.164660 |
| 1 | 0.126635 | 0.159728 | 0.189345 | 0.262279 | 0.066052 | 0.195962 |
| 2 | 0.159609 | 0.153870 | 0.168146 | 0.339030 | 0.053191 | 0.126155 |
| 3 | 0.134806 | 0.135256 | 0.181486 | 0.298535 | 0.056838 | 0.193079 |
| 4 | 0.036047 | 0.166505 | 0.237873 | 0.285831 | 0.106199 | 0.167545 |
gdf.set_geometry("geometry").plot(
gwrf_reg.feature_importances_["HC60"], legend=True
).set_axis_off()
Gradient Boosting Classifier¶
If you prefer to use gradient boosting, there is a minimal implementation of geographically weighted gradient boosting classifier, following the same model described for the random forest above.
gwgb = GWGradientBoostingClassifier(
bandwidth=150,
fixed=False,
)
gwgb.fit(
X,
y,
geometry=gdf.geometry,
)
GWGradientBoostingClassifier(bandwidth=150)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| bandwidth | 150 | |
| fixed | False | |
| kernel | 'bisquare' | |
| include_focal | False | |
| graph | None | |
| n_jobs | -1 | |
| fit_global_model | True | |
| strict | False | |
| keep_models | False | |
| temp_folder | None | |
| batch_size | None | |
| min_proportion | 0.2 | |
| undersample | False | |
| random_state | None | |
| coplanar | 'raise' | |
| verbose | False |
Fitted attributes
| Name | Type | Value |
|---|---|---|
| feature_importances_ | DataFrame | HR6...s x 6 columns] |
| feature_names_in_ | ndarray[object](6,) | ['HR60','HR70','HR80','HR90','HC60','HC70'] |
| local_class_support_ | Series[int64](1412,) | 0 2 1 ..., dtype: int64 |
| pred_ | Series[boolean](1412,) | 0 False...dtype: boolean |
| prediction_rate_ | float64 | 0.8364 |
| proba_ | DataFrame | Fals...s x 2 columns] |
Given the nature of the model, the outputs are a bit more limited. You can still extract focal predictions.
nan_mask = gwgb.pred_.notna()
metrics.accuracy_score(y[nan_mask], gwgb.pred_[nan_mask])
0.7519051651143099
And local feature importances.
gwgb.feature_importances_
| HR60 | HR70 | HR80 | HR90 | HC60 | HC70 | |
|---|---|---|---|---|---|---|
| 0 | 0.145283 | 0.255636 | 0.188970 | 0.220171 | 0.092706 | 0.097233 |
| 1 | 0.186062 | 0.182374 | 0.196313 | 0.236683 | 0.082929 | 0.115639 |
| 2 | 0.246788 | 0.129120 | 0.173432 | 0.290358 | 0.072368 | 0.087935 |
| 3 | 0.165445 | 0.193522 | 0.170751 | 0.270211 | 0.100012 | 0.100059 |
| 4 | 0.070898 | 0.185680 | 0.255261 | 0.241550 | 0.095212 | 0.151400 |
| ... | ... | ... | ... | ... | ... | ... |
| 1407 | 0.120036 | 0.173668 | 0.383373 | 0.254418 | 0.031432 | 0.037072 |
| 1408 | 0.230903 | 0.139015 | 0.288464 | 0.206338 | 0.089193 | 0.046087 |
| 1409 | 0.143109 | 0.296104 | 0.257873 | 0.176419 | 0.077498 | 0.048996 |
| 1410 | 0.121825 | 0.229833 | 0.263717 | 0.230628 | 0.030570 | 0.123426 |
| 1411 | 0.073566 | 0.189405 | 0.326127 | 0.244002 | 0.109388 | 0.057512 |
1412 rows × 6 columns
Leave out samples¶
However, the pooled data are not available. In this case, you can use the leave_out keyword to leave out a fraction (when float) or a set number (when int) of random observations from each local model. For these, the local model does prediction and returns as left_out_proba_ and left_out_y_ arrays.
gwgb_leave = GWGradientBoostingClassifier(
bandwidth=150,
fixed=False,
leave_out=0.2,
)
gwgb_leave.fit(
X,
y,
geometry=gdf.geometry,
)
GWGradientBoostingClassifier(bandwidth=150)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| bandwidth | 150 | |
| fixed | False | |
| kernel | 'bisquare' | |
| include_focal | False | |
| graph | None | |
| n_jobs | -1 | |
| fit_global_model | True | |
| strict | False | |
| keep_models | False | |
| temp_folder | None | |
| batch_size | None | |
| min_proportion | 0.2 | |
| undersample | False | |
| random_state | None | |
| coplanar | 'raise' | |
| verbose | False |
Fitted attributes
| Name | Type | Value |
|---|---|---|
| feature_importances_ | DataFrame | HR6...s x 6 columns] |
| feature_names_in_ | ndarray[object](6,) | ['HR60','HR70','HR80','HR90','HC60','HC70'] |
| left_out_proba_ | ndarray[float64](35430, 2) | [[0.6 ,0.4 ], [0.11,0.89], [0.98,0.02], ..., [0.67,0.33], [0.06,0.94], [0.17,0.83]] |
| left_out_w_ | ndarray[float64](35430,) | [0.04,0.34,0.77,...,0.42,0.97,0.84] |
| left_out_y_ | ndarray[bool](35430,) | [False,False, True,..., True, True,False] |
| local_class_support_ | Series[int64](1412,) | 0 2 1 ..., dtype: int64 |
| pred_ | Series[boolean](1412,) | 0 False...dtype: boolean |
| prediction_rate_ | float64 | 0.8364 |
| proba_ | DataFrame | Fals...s x 2 columns] |
metrics.accuracy_score(
gwgb_leave.left_out_y_, gwgb_leave.left_out_proba_.argmax(axis=1)
)
0.7190234264747389
Gradient Boosting Regressor¶
In the GWGradientBoostingRegressor, gradient boosting is applied for regression, taking into consideration the spatial heterogeneity. Gradient boosting is a type of boosting that builds a model by adding trees on a sequentially iterative basis; every new tree attempts to predict the errors made by the preceding ensemble. This makes the gradient boosting regressor appropriately suited for modeling complex non-linear relationships.
Unlike Random Forest, out-of-bag predictions are not implemented for Gradient Boosting in gwlearn. If you want to evaluate your model without a separate test dataset, you can use the leave_out option (as shown in the classifier section) or consider using GWRandomForestRegressor, which provides built-in OOB scoring.
gwgb_reg = GWGradientBoostingRegressor(
bandwidth=150,
fixed=False,
random_state=0,
)
gwgb_reg.fit(
X,
y_reg,
geometry=gdf.representative_point(),
)
GWGradientBoostingRegressor(bandwidth=150, random_state=0)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| bandwidth | 150 | |
| random_state | 0 | |
| fixed | False | |
| kernel | 'bisquare' | |
| include_focal | False | |
| graph | None | |
| n_jobs | -1 | |
| fit_global_model | True | |
| strict | False | |
| keep_models | False | |
| temp_folder | None | |
| batch_size | None | |
| coplanar | 'raise' | |
| verbose | False |
Fitted attributes
| Name | Type | Value |
|---|---|---|
| RSS_ | Series[float64](1412,) | focal 0 ...dtype: float64 |
| TSS_ | Series[float64](1412,) | 0 277....dtype: float64 |
| feature_importances_ | DataFrame | HR6...s x 6 columns] |
| local_r2_ | Series[float64](1412,) | 0 -0.069...dtype: float64 |
| pred_ | Series[float64](1412,) | 0 13.37...dtype: float64 |
| resid_ | Series[float64](1412,) | 0 -0.773...dtype: float64 |
| y_bar_ | Series[float64](1412,) | 0 12.92...dtype: float64 |
The regressor provides local R² values showing how well the model fits in each neighborhood:
gwgb_reg.local_r2_.head()
0 -0.069304
1 -0.167922
2 -0.285144
3 -0.271371
4 0.602021
dtype: float64
gdf.set_geometry("geometry").plot(gwgb_reg.local_r2_, legend=True).set_axis_off()
Feature importance¶
Gradient boosting computes feature importances based on how much each feature reduces the loss function across all trees in the ensemble. These importances tend to be more focused on the most predictive features compared to Random Forest:
gwgb_reg.feature_importances_.head()
| HR60 | HR70 | HR80 | HR90 | HC60 | HC70 | |
|---|---|---|---|---|---|---|
| 0 | 0.084235 | 0.135194 | 0.223552 | 0.293745 | 0.062522 | 0.200751 |
| 1 | 0.099926 | 0.131104 | 0.158024 | 0.319933 | 0.049705 | 0.241308 |
| 2 | 0.087537 | 0.142124 | 0.209735 | 0.413748 | 0.030275 | 0.116581 |
| 3 | 0.101652 | 0.148784 | 0.175880 | 0.372212 | 0.025944 | 0.175529 |
| 4 | 0.028434 | 0.050476 | 0.352884 | 0.460266 | 0.030526 | 0.077413 |
gdf.set_geometry("geometry").plot(
gwgb_reg.feature_importances_["HC60"], legend=True
).set_axis_off()
Comparison with global model¶
You can compare the local feature importances to values from the global model:
gwgb_reg.global_model.feature_importances_
array([0.06160495, 0.10269876, 0.20532801, 0.45881746, 0.11148472,
0.06006611])
gwgb_reg.feature_importances_.mean()
HR60 0.127054
HR70 0.121575
HR80 0.222042
HR90 0.302435
HC60 0.123367
HC70 0.103527
dtype: float64