Q3jafdZddlZddlmZddlmZmZmZddl m Z ddl m Z m Z dZGdd Zy) zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) get_namespaceget_namespace_and_devicemove_to)_align_api_if_sparse)safe_sparse_dot squared_normc |jd}tj|r=tt |j tj |df||f|zdS|dddf|z}|j |zS)z/Compute the sandwich product X.T @ diag(W) @ X.rshapeT) dense_outputN)r rissparserrT dia_array)XW n_samplesWXs N/DATA/.local/lib/python3.12/site-packages/sklearn/linear_model/_linear_loss.py sandwich_dotrsz I q#   !Q 9/EFJ!   q$wZ!^ssRxc~eZdZdZdZd dZdZdZdZ ddZ dd Z dd Z dd Z dd Z y)LinearModelLossa> General class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the average of per sample losses and includes a term for L2 regularization:: loss = 1 / s_sum * sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None and s_sum=sum_i s_i. Gradient and hessian, for simplicity without intercept, are:: gradient = 1 / s_sum * X.T @ loss.gradient + l2_reg_strength * coef hessian = 1 / s_sum * X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_classes * n_features:] = coef[(n_dof-1):] intercept.shape = (n_classes,) else: intercept = coef[-1] Shape of gradient follows shape of coef. gradient.shape = coef.shape But hessian (to make our lives simpler) are always 2-d: if base_loss.is_multiclass: hessian.shape = (n_classes * n_dof, n_classes * n_dof) else: hessian.shape = (n_dof, n_dof) Note: if coef has shape (n_classes * n_dof,), the classes are expected to be contiguous, i.e. the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool c ||_||_yN) base_loss fit_intercept)selfrrs r__init__zLinearModelLoss.__init__ss"*rNc |jd}|jj}|jr|dz}n|}|jjrt j ||f|d}|St j ||}|S)aAllocate coef of correct shape with zeros. Parameters: ----------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. dtype : data-type, default=None Overrides the data type of coef. With dtype=None, coef will have the same dtype as X. Returns ------- coef : ndarray of shape (n_dof,) or (n_classes, n_dof) Coefficients of a linear model. F)r dtypeorder)r r#)r r n_classesr is_multiclassnpzeros)rrr# n_featuresr%n_dofcoefs rinit_zero_coefzLinearModelLoss.init_zero_coefwsy WWQZ NN,,   NEE >> ' '889e"4EMD 88%u5D rc<|jjs"|jr|d}|dd}||fSd}|}||fS|jdk(r*|j |jj dfd}n|}|jr|dddf}|ddddf}||fSd}||fS)aHelper function to get coefficients and intercept. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. Nr!r"r$)rr&rndimreshaper%)rr+ interceptweightss rweight_interceptz LinearModelLoss.weight_intercepts$~~++!! H s)  !!  !!yyA~,,(@(@"'ES,Q!!#ArEN !!SbS&/ !!  !!rc,|j|\}}t|\}}}|j||j|}|j||j|} |jj s ||z| z} n||j z| z} ||| fS)aiHelper function to get coefficients, intercept and raw_prediction. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. raw_prediction : ndarray of shape (n_samples,) or (n_samples, n_classes) )r#device)r5rasarrayr#rr&r) rr+rr4r3xp_device_ weights_xp intercept_xpraw_predictions rweight_intercept_rawz$LinearModelLoss.weight_intercept_raws,"22481!4AwZZqwwwZG zz)1777zK ~~++^l:N- 11rcb|jdk(r||zn t|}td|z|zS)z5Compute L2 penalty term l2_reg_strength/2 *||w||_2^2.r!g?)r1r float)rr4l2_reg_strengthnorm2_ws r l2_penaltyzLinearModelLoss.l2_penaltys3'.||q'8'G#l7>SS?*W455rct|jd}||j||\} } }n|j|\} } |jj ||||} t |||\} } ||n| j |}t| j | |z } |dkDr| |j| |z } | S)aCompute the loss as weighted average over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. ry_truer> sample_weight n_threads) r r?r5rlossrsumrArD)rr+ryrHrBrIr>rr4r3rJr9r:sw_sums rrJzLinearModelLoss.losssNGGAJ  !151J1J4QR1S .GY!%!6!6t!< GY~~"")' #  aM2A+3 9NRVVD\F*+ Q  DOOG_= =D rc<|j|jjc\}} } | t|jz} ||j ||\} } }n|j |\} } |jj||||\}}t|||\}}||n|j|}t|j||z }||j| |z }||z}|jjsotj|| j}|j |z}t#|td|| zz|d| |jr|j||d<||fStj$| | f| jd}|j |z}t#|td|| zz|ddd| f<|jr*t#|j|d td|dddf<|j&d k(r|j)d }||fS) a\Computes the sum of loss and gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. gradient : ndarray of shape coef.shape The gradient of the loss. NrFr#cpu)r9r7r.r"r#r$raxisr!r0)r rr%intrr?r5 loss_gradientrrKrArDr&r' empty_liker#rremptyr1ravel)rr+rrLrHrBrIr>rr)r%r*r4r3rJgrad_pointwiser9r:rMgradX_gradgrad_Xs rrUzLinearModelLoss.loss_gradient#sT./WWdnn6N6N*JS!3!344  !151J1J4QR1S .GY!%!6!6t!< GY#~~;;)' < n aM2A+3 9NRVVD\F*+ 99& ~~++==W]];DSS>)F2e47PP * !!66.1R$Tz88Y.gmm3OD#%%)F2e47PP KZK !!%FF>F2r%QU yyA~zzz,TzrcF|j|jjc\}} } | t|jz} ||j ||\} } }n|j |\} } |jj||||}||ntj|}||z}|jjsZtj|| j}|j|z|| zz|d| |jr|j|d<|Stj| | f| jd}|j|z|| zz|ddd| f<|jr|jd|dddf<|jd k(r|j!d S|S) aComputes the gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. NrFrOr.r"rQrrRr!r0)r rr%rTrr?r5gradientr'rKr&rVr#rrWr1rX)rr+rrLrHrBrIr>rr)r%r*r4r3rYrMrZs rr^zLinearModelLoss.gradient}sN./WWdnn6N6N*JS!3!344  !151J1J4QR1S .GY!%!6!6t!< GY00)' 1  ,3 9N& ~~++==W]];D !n 47P PD* !!)--/RK88Y.gmm3OD#1#3#3a#7/G:S#SDKZK !!,00a08QU yyA~zzz,, rc $ |j|jjc\} } } | t|jz} | |j ||\}}} n|j |\}}|| ntj|}|#tj||jd}nx|j|jk7r&td|jd|jd|jjr!|jjs td|}|j}|$tj ||f|j}n~|j||fk7rtd ||fd |jd|jjr7|jj"s!|jjs td |}|jjs:|jj%|| || \}}||z}||z}tj&|d k|dkD}tj(|}|j*|z||zz|d| |jr|j|d<|r|||fSt-|||d| d| f<|d kDrA|jj"rdnd}|j/d|d| | z| dzxx|z cc<|jr|j*|z}||dddf<||dddf<|j|d<nj|jj1|| || \}}||z}|j/| | fd}|j*|z||zz|ddd| f<|jr|jd |dddf<|j2dk(r|j5d}|||z }nd|z }t7| D]]}|dd|fd|dd|fz z|z}t-||||| | z| || | z| f<|jrV|j*|z}|||| | z| | | z|zf<||| | z|z|| | z| f<|j|| | z|z| | z|zf<t7|dz| D]}|dd|f |dd|fz|z}t-||||| | z| || | z| f<|jrV|j*|z}|||| | z| | | z|zf<||| | z|z|| | z| f<|j|| | z|z| | z|zf<||d| |d| f||d| |d| f<`|d kDrJ|jj"rdnd}|j/d|d| dz| z| z| | zdzxx|z cc<d}|||fS)a~Computes gradient and hessian w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. gradient_out : None or ndarray of shape coef.shape A location into which the gradient is stored. If None, a new array might be created. hessian_out : None or ndarray of shape (n_dof, n_dof) or (n_classes * n_dof, n_classes * n_dof) A location into which the hessian is stored. If None, a new array might be created. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessian : ndarray of shape (n_dof, n_dof) or (n_classes, n_dof, n_dof, n_classes) Hessian matrix. hessian_warning : bool True if pointwise hessian has more than 25% of its elements non-positive. Nr"rQz4gradient_out is required to have shape coef.shape = z; got .z"gradient_out must be F-contiguous.rOz'hessian_out is required to have shape (z); got hessian_out.shape=zhessian_out must be contiguous.rFr)r4g?r.Cr0r!)r.r.rRg?F)r rr%rTrr?r5r'rKrVr# ValueErrorr&flags f_contiguoussizerW c_contiguousgradient_hessianaverageabsrrr2gradient_probar1rXrange)rr+rrLrHrBrI gradient_out hessian_outr>rr)r%r*r4r3rMrZnhessrYhess_pointwisehessian_warningr$Xhprobaswkhls rrhz LinearModelLoss.gradient_hessiansn./WWdnn6N6N*JS!3!344  !151J1J4QR1S .GY!%!6!6t!< GY+3 9N  ==W]]#FD   4:: -FtzzlS#))*!- ^^ ) ),2D2D2Q2QAB BD II  88QF'--8D   1a& (9!Q$@&$$&a) ^^ ) )!!..{7H7H7U7U>? ?D~~++-1^^-L-L-+# .M. *NN f $N f $N  >Q. FM  VVN3N !n 47P PD* !!)--/RT?22-9!^-LD*kzk) *" $zz66C Ru -.Re1CPQ .RS#S!!SS>) "SbS"W "R"W -113V %)NN$A$A-+# %B% !NE f $N<<E 2#<>D#1#3#3a#7/G:S#SDKZK !!,00a08QU yyA~zzz,L("V+6\9%!Q$K1uQT{?3b8!A& J.: J.:<%%qBI 2Y>!J.24!J.2I 2Y>@ Z/!3Y5Ka5OOPq1ui0Aq!t uQT{2R7A%Q*I 2Y>I 2Y>@))SS1W J 6B% 2Q68% 2Q6 J 6BD EEGY3a7Z9ORS9SST8.hesspsmmA&??1%'(ssb1[j>.A'BC $')yy':':ACCQ{ ^;T'UC $KZ Oa n$DD %% $"6$$q*~5 ae8KKCG rr"rQcT |jdfd} jr|dddf}|ddddf}nd}|jz|z}| |zjdddtj fz}| z} | ddtj fz}t j fjd}|jz z |zz|ddd f< jr|jd z |dddf<jdk(r|jdS|S)Nr.r"r0rr!rRrQ) r2rrrKr'newaxisrWr#r1rX)r} s_intercepttmp hess_prodrr+rBr%r*r)rtrHrrMr4s rrz7LinearModelLoss.gradient_hessian_product..hesspDs6IIy"oSI9%%"#ArE(K!SbS& A"#K!##g + ((a(0BJJ??u  ,=BJJ77CHHi%7w}}TWX -0UUQY&,@?UVCV,V ![j[.)%%'*wwAw'?Iae$99>$???55$$rr!r0)r rr%rTrr?r'rKr&rhrVr#rrrrrsqueezer8 atleast_1drkrWr1rX)rr+rrLrHrBrIrr3r>rYrqrZrrrrr%r*r)rtrMr4s``` `` @@@@@@@@@rgradient_hessian_productz(LinearModelLoss.gradient_hessian_products@./WWdnn6N6N*JS!3!344-1-F-FtQ-O*N+3 9N~~++-1^^-L-L-+# .M. *NN f $N f $N==W]];D !n 47P PD* !!)--/R),,.Kq!$$na%8I@VW $ArzzM2Q6!!BJJrvv1v~$>?v.  \U{w%)NN$A$A-+# %B% !NE f $N88Y.gmm3OD#1#3#3a#7/G:S#SDKZK !!,00a08QU . % %.yyA~zzz,e33U{rr)Nr/r!N)Nr/r!NNN)Nr/r!)__name__ __module__ __qualname____doc__rr,r5r?rDrJrUr^rhrrrrr/sAF+8%"N&2P6:BX~G\+DNOWrr)rnumpyr'scipyrsklearn.utils._array_apirrrsklearn.utils._sparsersklearn.utils.extmathrr rrrrrrs5 7?6o o r