-i 0SrSSKrSSKrSSKrSSKJrJr SSKJ r "SS\ 5r S Sjr S S jr S S jrSS jrg)a Our own implementation of the Newton algorithm Unlike the scipy.optimize version, this version of the Newton conjugate gradient solver uses only one function call to retrieve the func value, the gradient value and a callable for the Hessian matvec product. If the function call is very expensive (e.g. for logistic regression with large design matrix), this approach gives very significant speedups. N)line_search_wolfe1line_search_wolfe2)ConvergenceWarningc\rSrSrSrg)_LineSearchErrorN)__name__ __module__ __qualname____firstlineno____static_attributes__r I/var/www/html/venv/lib/python3.13/site-packages/sklearn/utils/optimize.pyrrsrrc .US:n S[R"[R"U5R5R-n U (a$[ S5 [ SU 35 [ S5 [ XX#XEU40UD6n U (aU ScSOSn [ S U -S -5 U SGcURS [55n U"X#-/U Q76n[R"XZ-5nX- n[R"U5U:*nU (a([ S [R"U5S USU35 U(a[RRUSS9nU"X#-/U Q76n[RRUSS9nUU:nU (a[ SUSUSU35 U(aSU SS-U SS-UUU4n U ScCU (a [ S5 [XX#XEU40UD6n U (aU ScSOSn [ SU -S -5 U Sc [5eU $)z Same as line_search_wolfe1, but fall back to line_search_wolfe2 if suitable step length is not found, and raise an exception if a suitable step length is not found. Raises ------ _LineSearchError If no suitable step size is found. rz Line Searchz eps=16 * finfo.eps=z try line search wolfe1rznot z wolfe1 line search was successfulargsz2 check loss |improvement| <= eps * |loss_old|:  <=  )ordz1 check sum(|gradient|) < sum(|gradient_old|): z < ?z' last resort: try line search wolfe2z wolfe2 line search was )npfinfoasarraydtypeepsprintrgettupleabsscipylinalgnormrr)ffprimexkpkgfkold_fval old_old_fvalverbosekwargs is_verboser ret_not_rfval tiny_lossloss_improvementchecksum_abs_grad_oldgrad sum_abs_grads r_line_search_wolfe12r;s:AJ rxx 8,22377 7C o 'u-. *+ Q| Vv VCa&.b +e3lBC 1v~zz&%'* 4 FF8>* ?'(I5  FF+,-T)AeWF  $||00!0< "')D)D <<,,Tq,9L #33EG#nC(8'95'CFQJFQJ  1v~  ; < rsl >D  !!fnF"E /%7,F G 1v~  Jrc tS[R"[R5R-n[R"[ U5UR S9n[R"U5nU*nSn [R"Xw5n U n US:n X::GaX[R"[R"U55U::aDU (a<[SU S[R"[R"U55SU35 OU"U5n [R"X5nSUs=::aX[-::a'O O$U (a[SU SUS U S USX[-3 5 OUS:a<U S:aU (a[SU S U35 OmXjU- U-- nU (a [S 5 OPX- nXoU-- nXU -- n[R"Xw5nUU - nU*UU--nUUS-U --n U S -n Un X::aGMXU (aAX:a<[SU S - S[R"[R"U5535 U$)a Solve iteratively the linear system 'fhess_p . xsupi = fgrad' with a conjugate gradient descent. Parameters ---------- fhess_p : callable Function that takes the gradient as a parameter and returns the matrix product of the Hessian and gradient. fgrad : ndarray of shape (n_features,) or (n_features + 1,) Gradient vector. maxiter : int Number of CG iterations. tol : float Stopping criterion. Returns ------- xsupi : ndarray of shape (n_features,) or (n_features + 1,) Estimated solution. r)rrrz Inner CG solver iteration z+ stopped with sum(|residuals|) <= tol: rz2 stopped with tiny_|p| = eps * ||p||^2, eps = z, squared L2 norm ||p||^2 = z curvature <= tiny_|p|: z. stopped with negative curvature, curvature = z; Inner CG solver iteration 0 fell back to steepest descentrz+ Inner CG solver stopped reaching maxiter=z with sum(|residuals|) = ) rrfloat64r zeroslenrcopydotsumr$r!)fhess_pfgradmaxitertolr/r xsupiripsupiidri0 psupi_norm2r1Apcurvalphaidri1betais r_cgrRqsA2 rxx #'' 'C HHSZu{{ 3E B CE A 66">DKAJ , 66"&&*  $21#6446FF266":4F3GtC5R  U^vve   )) )21#6;;>%@11< >226tC=F>cDUS:XaUb[URU5nO URnURS:wa]USUSURSURS3nUbXB:Xa USUS3- nUS - nUbUSU-- n[R "U[ S S 9 U$[e) aCheck the OptimizeResult for successful convergence Parameters ---------- solver : str Solver name. Currently only `lbfgs` is supported. result : OptimizeResult Result of the scipy.optimize.minimize function. max_iter : int, default=None Expected maximum number of iterations. extra_warning_msg : str, default=None Extra warning message. Returns ------- n_iter : int Number of iterations. lbfgsrz failed to converge after z iteration(s) (status=z):  zH Increase the number of iterations to improve the convergence (max_iter=z).zr You might also want to scale the data as shown in: https://scikit-learn.org/stable/modules/preprocessing.htmlr) stacklevel)rWnitstatusmessagerYrZrNotImplementedError)solverresultmax_iterextra_warning_msgn_iter_i warning_msgs r_check_optimize_resultryKs.  6::x0HzzH ==A (4XJ?!==/>>""& #(<--5Jb:  % K !,t&777 MM+'9a H O"!r)r)r g-C6?dTTr)NN)__doc__rYnumpyrr%scipy.optimize._linesearchrr exceptionsr RuntimeErrorrr;rRrjryr rrrs] " M+ | =>OdV|    ~B:r