Q3jdZddlmZddlZddlmZddlmZddl m Z m Z ddl m Z ddlmZdd lmZdd lmZmZdd lmZdd lmZmZmZdd lmZmZmZddlm Z ddl!m"Z"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(ddl)m*Z*ddl+m,Z,m-Z-dgZ.dZ/dZ0 d,dZ1e&e"ddge"ddge"ddggddgde#e2gddgddgddge%hdge3ge$e#ddd ge%hd!ge%hd"ge4dgd# d$%dddd&d'd(dd&d)d* d+Z5y)-zBPartial dependence plots for regression and classification models.)IterableN)sparse) mquantiles) is_classifier is_regressor)RandomForestRegressor)BaseGradientBoosting)BaseHistGradientBoosting)_check_feature_names_get_feature_index)DecisionTreeRegressor)Bunch_safe_indexing check_array)_determine_key_type_get_column_indices _safe_assign)check_matplotlib_support) HasMethodsIntegralInterval StrOptionsvalidate_params)_get_response_values) cartesian)_check_sample_weightcheck_is_fittedpartial_dependencect|trt|dk7r tdt d|Ds td|d|dk\r td|dkr tdd }|j Dcic]\}}|||}}}t d |jDr4d jd |j D}td |g} t|D]\} } | |vr|| } n tjt|| d} | s| jd|kr| } n`tt|| d|d}tj |d|dr tdtj"|d|d|d} | j%| t'| | fScc}}w#t$r}td| d|d}~wwxYw)a!Generate a grid of points based on the percentiles of X. The grid is a cartesian product between the columns of ``values``. The ith column of ``values`` consists in ``grid_resolution`` equally-spaced points between the percentiles of the jth column of X. If ``grid_resolution`` is bigger than the number of unique values in the j-th column of X or if the feature is a categorical feature (by inspecting `is_categorical`) , then those unique values will be used instead. Parameters ---------- X : array-like of shape (n_samples, n_target_features) The data. percentiles : tuple of float The percentiles which are used to construct the extreme values of the grid. Must be in [0, 1]. is_categorical : list of bool For each feature, tells whether it is categorical or not. If a feature is categorical, then the values used will be the unique ones (i.e. categories) instead of the percentiles. grid_resolution : int The number of equally spaced points to be placed on the grid for each feature. custom_values: dict Mapping from column index of X to an array-like of values where the partial dependence should be calculated for that feature Returns ------- grid : ndarray of shape (n_points, n_target_features) A value for each feature at each point in the grid. ``n_points`` is always ``<= grid_resolution ** X.shape[1]``. values : list of 1d ndarrays The values with which the grid has been created. The size of each array ``values[j]`` is either ``grid_resolution``, the number of unique values in ``X[:, j]``, if j is not in ``custom_range``. If j is in ``custom_range``, then it is the length of ``custom_range[j]``. z/'percentiles' must be a sequence of 2 elements.c3<K|]}d|cxkxrdkncyw)rN).0xs S/DATA/.local/lib/python3.12/site-packages/sklearn/inspection/_partial_dependence.py z_grid_from_X..Zs0KqqA{{{Kz''percentiles' values must be in [0, 1].rr"z9percentiles[0] must be strictly less than percentiles[1].z2'grid_resolution' must be strictly greater than 1.cdtd|Drtnd}tj||S)Nc3<K|]}t|tywN) isinstancestrr$vs r&r'z?_grid_from_X.._convert_custom_values..dsA&QjC0&r()dtype)anyobjectnpasarray)valuesr0s r&_convert_custom_valuesz,_grid_from_X.._convert_custom_valuesbs'A&AAtzz&..c3:K|]}|jdk7yw)r"Nndimr.s r&r'z_grid_from_X..hs 7 61166Q; 6s, c3fK|])\}}|jdk7rd|d|jd+yw)r"zFeature z: z dimensionsNr9)r$kr/s r&r'z_grid_from_X..is:! -1vv{qcAFF8; /-s/1zBThe custom grid for some features is not a one-dimensional array. axisz The column #z contains mixed data types. Finding unique categories fail due to sorting. It usually means that the column contains `np.nan` values together with `str` categories. Such use case is not yet supported in scikit-learn.N)probr?ztpercentiles are too close to each other, unable to build the grid. Please choose percentiles that are further apart.T)numendpoint)r,rlen ValueErrorallitemsr1r5join enumerater3uniquer TypeErrorshaperallcloselinspaceappendr)X percentilesis_categoricalgrid_resolution custom_valuesr6r=r/ error_stringr5featureis_catr?uniquesexcemp_percentiless r& _grid_from_XrZ+s4Z k8 ,K0@A0EJKK 0K0 0BCC1~Q'TUU!MNN/ ?L>Q>Q>ST>SdaQ.q11>SMT 7 4 4 6 77yy! %++-!   Pn   F%^4 m # )D ))N1gA$FGq)O;#-"1gA6[q#;;q1?13EF$2 {{#A&#A&'!   dO5R V f $$sU. !"7),AA   sG !G  G'G""G'cl|j||}|jdk(r|jdd}|S)a} Calculate partial dependence via the recursion method. The recursion method is in particular enabled for tree-based estimators. For each `grid` value, a weighted tree traversal is performed: if a split node involves an input feature of interest, the corresponding left or right branch is followed; otherwise both branches are followed, each branch being weighted by the fraction of training samples that entered that branch. Finally, the partial dependence is given by a weighted average of all the visited leaves values. This method is more efficient in terms of speed than the `'brute'` method (:func:`~sklearn.inspection._partial_dependence._partial_dependence_brute`). However, here, the partial dependence computation is done explicitly with the `X` used during training of `est`. Parameters ---------- est : BaseEstimator A fitted estimator object implementing :term:`predict` or :term:`decision_function`. Multioutput-multiclass classifiers are not supported. Note that `'recursion'` is only supported for some tree-based estimators (namely :class:`~sklearn.ensemble.GradientBoostingClassifier`, :class:`~sklearn.ensemble.GradientBoostingRegressor`, :class:`~sklearn.ensemble.HistGradientBoostingClassifier`, :class:`~sklearn.ensemble.HistGradientBoostingRegressor`, :class:`~sklearn.tree.DecisionTreeRegressor`, :class:`~sklearn.ensemble.RandomForestRegressor`, ). grid : array-like of shape (n_points, n_target_features) The grid of feature values for which the partial dependence is calculated. Note that `n_points` is the number of points in the grid and `n_target_features` is the number of features you are doing partial dependence at. features : array-like of {int, str} The feature (e.g. `[0]`) or pair of interacting features (e.g. `[(0, 1)]`) for which the partial dependency should be computed. Returns ------- averaged_predictions : array-like of shape (n_targets, n_points) The averaged predictions for the given `grid` of features values. Note that `n_targets` is the number of targets (e.g. 1 for binary classification, `n_tasks` for multi-output regression, and `n_classes` for multiclass classification) and `n_points` is the number of points in the `grid`. r")%_compute_partial_dependence_recursionr:reshape)estgridfeaturesaveraged_predictionss r&_partial_dependence_recursionrcsAbDDT8T  A% 4;;ArB r7cg}g}|dk(rt|rdnddg}|j}|D]o} t|D]\} } t|| | | t |||\} } |j | |j t j| d|q|jd}t j|j}t|r"|jd k(r|j|d }n4t|r)|jdd k(r|d }|j|d }t j|j}|jd k(r|jd d }||fS) a&Calculate partial dependence via the brute force method. The brute method explicitly averages the predictions of an estimator over a grid of feature values. For each `grid` value, all the samples from `X` have their variables of interest replaced by that specific `grid` value. The predictions are then made and averaged across the samples. This method is slower than the `'recursion'` (:func:`~sklearn.inspection._partial_dependence._partial_dependence_recursion`) version for estimators with this second option. However, with the `'brute'` force method, the average will be done with the given `X` and not the `X` used during training, as it is done in the `'recursion'` version. Therefore the average can always accept `sample_weight` (even when the estimator was fitted without). Parameters ---------- est : BaseEstimator A fitted estimator object implementing :term:`predict`, :term:`predict_proba`, or :term:`decision_function`. Multioutput-multiclass classifiers are not supported. grid : array-like of shape (n_points, n_target_features) The grid of feature values for which the partial dependence is calculated. Note that `n_points` is the number of points in the grid and `n_target_features` is the number of features you are doing partial dependence at. features : array-like of {int, str} The feature (e.g. `[0]`) or pair of interacting features (e.g. `[(0, 1)]`) for which the partial dependency should be computed. X : array-like of shape (n_samples, n_features) `X` is used to generate values for the complement features. That is, for each value in `grid`, the method will average the prediction of each sample from `X` having that grid value for `features`. response_method : {'auto', 'predict_proba', 'decision_function'}, default='auto' Specifies whether to use :term:`predict_proba` or :term:`decision_function` as the target response. For regressors this parameter is ignored and the response is always the output of :term:`predict`. By default, :term:`predict_proba` is tried first and we revert to :term:`decision_function` if it doesn't exist. sample_weight : array-like of shape (n_samples,), default=None Sample weights are used to calculate weighted means when averaging the model output. If `None`, then samples are equally weighted. Note that `sample_weight` does not change the individual predictions. Returns ------- averaged_predictions : array-like of shape (n_targets, n_points) The averaged predictions for the given `grid` of features values. Note that `n_targets` is the number of targets (e.g. 1 for binary classification, `n_tasks` for multi-output regression, and `n_classes` for multiclass classification) and `n_points` is the number of points in the `grid`. predictions : array-like The predictions for the given `grid` of features values over the samples from `X`. For non-multioutput regression and binary classification the shape is `(n_instances, n_points)` and for multi-output regression and multiclass classification the shape is `(n_targets, n_instances, n_points)`, where `n_targets` is the number of targets (`n_tasks` for multi-output regression, and `n_classes` for multiclass classification), `n_instances` is the number of instances in `X`, and `n_points` is the number of points in the `grid`. autopredict predict_probadecision_function)column_indexer)response_methodr)r?weightsr r\r")rcopyrHrrrNr3averagerKarrayTr:r^r)r_r`rarOrj sample_weight predictionsrbX_eval new_valuesivariablepred_ n_sampless r&_partial_dependence_bruterysoPK& %c*IBU0V VVXF $X.KAx Ax H/'sFOTa4 ##BJJt!]$ST  I((;'))KC[--2!)))R8 s  1 1! 4 9"!n !)))R8 88$89;;  A% 4;;ArB  ,,r7fitrfrgrhz array-likez sparse matrix>rergrhr"left)closed>rebrute recursion>bothrm individual) estimatorrOrarpcategorical_features feature_namesrjrPrRmethodkindrST)prefer_skip_nested_validationre)g?gffffff?drm) rprrrjrPrRrSrrc  t|t|st|s tdt|r2t |j dt jr tdt|ds'tj|st|dt}t|r|dk7r td| d k7r| d k(r td d } | d k(r | td | dk(rD|d } n?t |tr|jd } n t |tt t"frd } nd } | d k(rft |ttt t"fs+d} tdj%dj'| |dk(rd}|dk7rtd|d| t)||}t+|ddk(rHt j,t j.|drtd|j0ddz dt j2t5||t j6dj9} t;||}|j0d}|dgt=| z}nt j2|}|j>dk(r td|j@jBdk(r>|j>|k7rtd|j>d|d | Dcgc]}|| }}n]|j@jBd!vr,|Dcgc]}tE||"}}| Dcgc]}||v}}ntd#|j@d$| xsi} t |tFtHfr|g}tK| ||D]<\}}}|r tM||d%j@jBd&vs/td'|d(tM|| d%}tO|Dcic]\}}|| vr|| jQ|}}}tS|||||\}}| d k(rPtU||| |||\}}|jVd)|j0dg|Dcgc]}|j0dc}}n tY||| }|jVd)g|Dcgc]}|j0dc}}t[|*}| d k(r||d <|S| d+k(r|d+<|S||d <|d+<|Scc}wcc}wcc}wcc}}wcc}wcc}w),a#Partial dependence of ``features``. Partial dependence of a feature (or a set of features) corresponds to the average response of an estimator for each possible value of the feature. Read more in :ref:`sphx_glr_auto_examples_inspection_plot_partial_dependence.py` and the :ref:`User Guide `. .. warning:: For :class:`~sklearn.ensemble.GradientBoostingClassifier` and :class:`~sklearn.ensemble.GradientBoostingRegressor`, the `'recursion'` method (used by default) will not account for the `init` predictor of the boosting process. In practice, this will produce the same values as `'brute'` up to a constant offset in the target response, provided that `init` is a constant estimator (which is the default). However, if `init` is not a constant estimator, the partial dependence values are incorrect for `'recursion'` because the offset will be sample-dependent. It is preferable to use the `'brute'` method. Note that this only applies to :class:`~sklearn.ensemble.GradientBoostingClassifier` and :class:`~sklearn.ensemble.GradientBoostingRegressor`, not to :class:`~sklearn.ensemble.HistGradientBoostingClassifier` and :class:`~sklearn.ensemble.HistGradientBoostingRegressor`. Parameters ---------- estimator : BaseEstimator A fitted estimator object implementing :term:`predict`, :term:`predict_proba`, or :term:`decision_function`. Multioutput-multiclass classifiers are not supported. X : {array-like, sparse matrix or dataframe} of shape (n_samples, n_features) ``X`` is used to generate a grid of values for the target ``features`` (where the partial dependence will be evaluated), and also to generate values for the complement features when the `method` is 'brute'. features : array-like of {int, str, bool} or int or str The feature (e.g. `[0]`) or pair of interacting features (e.g. `[(0, 1)]`) for which the partial dependency should be computed. sample_weight : array-like of shape (n_samples,), default=None Sample weights are used to calculate weighted means when averaging the model output. If `None`, then samples are equally weighted. If `sample_weight` is not `None`, then `method` will be set to `'brute'`. Note that `sample_weight` is ignored for `kind='individual'`. .. versionadded:: 1.3 categorical_features : array-like of shape (n_features,) or shape (n_categorical_features,), dtype={bool, int, str}, default=None Indicates the categorical features. - `None`: no feature will be considered categorical; - boolean array-like: boolean mask of shape `(n_features,)` indicating which features are categorical. Thus, this array has the same shape has `X.shape[1]`; - integer or string array-like: integer indices or strings indicating categorical features. .. versionadded:: 1.2 feature_names : array-like of shape (n_features,), dtype=str, default=None Name of each feature; `feature_names[i]` holds the name of the feature with index `i`. By default, the name of the feature corresponds to their numerical index for NumPy array and their column name for pandas dataframe. .. versionadded:: 1.2 response_method : {'auto', 'predict_proba', 'decision_function'}, default='auto' Specifies whether to use :term:`predict_proba` or :term:`decision_function` as the target response. For regressors this parameter is ignored and the response is always the output of :term:`predict`. By default, :term:`predict_proba` is tried first and we revert to :term:`decision_function` if it doesn't exist. If ``method`` is 'recursion', the response is always the output of :term:`decision_function`. percentiles : tuple of float, default=(0.05, 0.95) The lower and upper percentile used to create the extreme values for the grid. Must be in [0, 1]. This parameter is overridden by `custom_values` if that parameter is set. grid_resolution : int, default=100 The number of equally spaced points on the grid, for each target feature. This parameter is overridden by `custom_values` if that parameter is set. custom_values : dict A dictionary mapping the index of an element of `features` to an array of values where the partial dependence should be calculated for that feature. Setting a range of values for a feature overrides `grid_resolution` and `percentiles`. See :ref:`how to use partial_dependence ` for an example of how this parameter can be used. .. versionadded:: 1.7 method : {'auto', 'recursion', 'brute'}, default='auto' The method used to calculate the averaged predictions: - `'recursion'` is only supported for some tree-based estimators (namely :class:`~sklearn.ensemble.GradientBoostingClassifier`, :class:`~sklearn.ensemble.GradientBoostingRegressor`, :class:`~sklearn.ensemble.HistGradientBoostingClassifier`, :class:`~sklearn.ensemble.HistGradientBoostingRegressor`, :class:`~sklearn.tree.DecisionTreeRegressor`, :class:`~sklearn.ensemble.RandomForestRegressor`, ) when `kind='average'`. This is more efficient in terms of speed. With this method, the target response of a classifier is always the decision function, not the predicted probabilities. Since the `'recursion'` method implicitly computes the average of the Individual Conditional Expectation (ICE) by design, it is not compatible with ICE and thus `kind` must be `'average'`. - `'brute'` is supported for any estimator, but is more computationally intensive. - `'auto'`: the `'recursion'` is used for estimators that support it, and `'brute'` is used otherwise. If `sample_weight` is not `None`, then `'brute'` is used regardless of the estimator. Please see :ref:`this note ` for differences between the `'brute'` and `'recursion'` method. kind : {'average', 'individual', 'both'}, default='average' Whether to return the partial dependence averaged across all the samples in the dataset or one value per sample or both. See Returns below. Note that the fast `method='recursion'` option is only available for `kind='average'` and `sample_weights=None`. Computing individual dependencies and doing weighted averages requires using the slower `method='brute'`. .. versionadded:: 0.24 Returns ------- predictions : :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. individual : ndarray of shape (n_outputs, n_instances, len(values[0]), len(values[1]), ...) The predictions for all the points in the grid for all samples in X. This is also known as Individual Conditional Expectation (ICE). Only available when `kind='individual'` or `kind='both'`. average : ndarray of shape (n_outputs, len(values[0]), len(values[1]), ...) The predictions for all the points in the grid, averaged over all samples in X (or over the training data if `method` is 'recursion'). Only available when `kind='average'` or `kind='both'`. grid_values : seq of 1d ndarrays The values with which the grid has been created. The generated grid is a cartesian product of the arrays in `grid_values` where `len(grid_values) == len(features)`. The size of each array `grid_values[j]` is either `grid_resolution`, or the number of unique values in `X[:, j]`, whichever is smaller. .. versionadded:: 1.3 `n_outputs` corresponds to the number of classes in a multi-class setting, or to the number of tasks for multi-output regression. For classical regression and binary classification `n_outputs==1`. `n_values_feature_j` corresponds to the size `grid_values[j]`. See Also -------- PartialDependenceDisplay.from_estimator : Plot Partial Dependence. PartialDependenceDisplay : Partial Dependence visualization. Examples -------- >>> X = [[0, 0, 2], [1, 0, 0]] >>> y = [0, 1] >>> from sklearn.ensemble import GradientBoostingClassifier >>> gb = GradientBoostingClassifier(random_state=0).fit(X, y) >>> partial_dependence(gb, features=[0], X=X, percentiles=(0, 1), ... grid_resolution=2) # doctest: +SKIP (array([[-4.52, 4.52]]), [array([ 0., 1.])]) z5'estimator' must be a fitted regressor or classifier.rz3Multiclass-multioutput estimators are not supported __array__z allow-nan)ensure_all_finiter0rezKThe response_method parameter is ignored for regressors and must be 'auto'.rmr~zCThe 'recursion' method only applies when 'kind' is set to 'average'r}zFThe 'recursion' method can only be applied when sample_weight is None.)GradientBoostingClassifierGradientBoostingRegressorHistGradientBoostingClassifierHistGradientBoostingRegressorrr rz[Only the following estimators support the 'recursion' method: {}. Try using method='brute'.r;rhzRWith the 'recursion' method, the response_method must be 'decision_function'. Got .F) accept_sliceintzall features must be in [0, r"]C)r0orderzPassing an empty list (`[]`) to `categorical_features` is not supported. Use `None` instead to indicate that there are no categorical features.bzeWhen `categorical_features` is a boolean array-like, the array should be of shape (n_features,). Got z elements while `X` contains z features.)rtOU)rzXExpected `categorical_features` to be an array-like of boolean, integer, or string. Got z instead.r>iuz The column a contains integer data. Partial dependence plots are not supported for integer data: this can lead to implicit rounding with NumPy arrays or even errors with newer pandas versions. Please convert numerical features to floating point dtypes ahead of time to avoid problems.r\) grid_valuesr).rrrrDr,classes_r3ndarrayhasattrrissparserr2r initr r rformatrGrrr1lessrKr4rintpravelr rCsizer0rr r-rziprrHgetrZryr^rcr)rrOrarprrrjrPrRrSrrsupported_classes_recursionfeatures_indices n_featuresrQidxcatcategorical_features_idx feature_idxrUrVX_subsetindexcustom_values_for_X_subsetr`r5rbrqval pdp_resultss r&rr]sNI ) $ Y(?PQQYJy/A/A!/Dbjj$QNOO A{ #vq'9 [ GI?f#<    y [ U  !: T    $F  #7 8Y^^=S F   %'<>S T !FF  $(%%   + '88>II9:9  f $1O 1 1,,;+ 8%8EA 66"''(A& ';AGGAJN;K1MN NzzAx(s eg)M:MJ#3'7#88!zz*>?  $ $ )(  % % * *c 1#((J6 G+0011N!l*. DTTCSC237CSNT ! ' ' , , ?0(/C#3mD/ %( .s$ SS " S S=S 9S r+)6__doc__collections.abcrnumpyr3scipyrscipy.stats.mstatsr sklearn.baserrsklearn.ensemblersklearn.ensemble._gbr :sklearn.ensemble._hist_gradient_boosting.gradient_boostingr sklearn.inspection._pd_utilsr r sklearn.treer sklearn.utilsrrrsklearn.utils._indexingrrr$sklearn.utils._optional_dependenciesrsklearn.utils._param_validationrrrrrsklearn.utils._responsersklearn.utils.extmathrsklearn.utils.validationrr__all__rZrcryr-tupledictrr#r7r&rs\H %)425R.<< J9+J  u%p7 v<@}-@ y) * / 0 23 4 O ,!8S1&-!-t 4&-&'UVWw$Xq$vFG<=>=>?!$#''4  S+*Sr7