-iSrSSKrSSKJrJr SSKJr SSKr SSK J r SSK J r SSKJrJrJr SS KJr SS KJr SS KJr "S S 5r"SS5rSSjrg)z This module contains the TreeGrower class. TreeGrower builds a regression tree fitting a Newton-Raphson step, based on the gradients and hessians of the training data. N)heappopheappush) default_timer)_openmp_effective_n_threads)!set_raw_bitset_from_binned_bitset)PREDICTOR_RECORD_DTYPEX_BITSET_INNER_DTYPEMonotonicConstraint)HistogramBuilder) TreePredictor)Splitterc2\rSrSrSrSS.SjrSrSrSrg) TreeNodeaTree Node class used in TreeGrower. This isn't used for prediction purposes, only for training (see TreePredictor). Parameters ---------- depth : int The depth of the node, i.e. its distance from the root. sample_indices : ndarray of shape (n_samples_at_node,), dtype=np.uint32 The indices of the samples at the node. partition_start : int start position of the node's sample_indices in splitter.partition. partition_stop : int stop position of the node's sample_indices in splitter.partition. sum_gradients : float The sum of the gradients of the samples at the node. sum_hessians : float The sum of the hessians of the samples at the node. Attributes ---------- depth : int The depth of the node, i.e. its distance from the root. sample_indices : ndarray of shape (n_samples_at_node,), dtype=np.uint32 The indices of the samples at the node. sum_gradients : float The sum of the gradients of the samples at the node. sum_hessians : float The sum of the hessians of the samples at the node. split_info : SplitInfo or None The result of the split evaluation. is_leaf : bool True if node is a leaf left_child : TreeNode or None The left child of the node. None for leaves. right_child : TreeNode or None The right child of the node. None for leaves. value : float or None The value of the leaf, as computed in finalize_leaf(). None for non-leaf nodes. partition_start : int start position of the node's sample_indices in splitter.partition. partition_stop : int stop position of the node's sample_indices in splitter.partition. allowed_features : None or ndarray, dtype=int Indices of features allowed to split for children. interaction_cst_indices : None or list of ints Indices of the interaction sets that have to be applied on splits of child nodes. The fewer sets the stronger the constraint as fewer sets contain fewer features. children_lower_bound : float children_upper_bound : float N)valuec*XlX lURSUlXPlX`lXplSUlSUlSUl UR[S5[S55 SUl SUl SUlSUlX0lX@lg)NrFz-infz+inf)depthsample_indicesshape n_samples sum_gradients sum_hessiansris_leafallowed_featuresinteraction_cst_indicesset_children_boundsfloat split_info left_child right_child histogramspartition_startpartition_stop)selfrrr#r$rrrs b/var/www/html/venv/lib/python3.13/site-packages/sklearn/ensemble/_hist_gradient_boosting/grower.py__init__TreeNode.__init__Vs ,'--a0*(  $'+$   vf > /,cXlX lg)z`. interaction_cst : list of sets of integers, default=None List of interaction constraints. l2_regularization : float, default=0. The L2 regularization parameter penalizing leaves with small hessians. Use ``0`` for no regularization (default). feature_fraction_per_split : float, default=1 Proportion of randomly chosen features in each and every node split. This is a form of regularization, smaller values make the trees weaker learners and might prevent overfitting. rng : Generator Numpy random Generator used for feature subsampling. shrinkage : float, default=1. The shrinkage parameter to apply to the leaves values, also known as learning rate. n_threads : int, default=None Number of OpenMP threads to use. `_openmp_effective_n_threads` is called to determine the effective number of threads use, which takes cgroups CPU quotes into account. See the docstring of `_openmp_effective_n_threads` for details. Attributes ---------- histogram_builder : HistogramBuilder splitter : Splitter root : TreeNode finalized_leaves : list of TreeNode splittable_nodes : list of TreeNode missing_values_bin_idx : int Equals n_bins - 1 n_categorical_splits : int n_features : int n_nodes : int total_find_split_time : float Time spent finding the best splits total_compute_hist_time : float Time spent computing histograms total_apply_split_time : float Time spent splitting nodes with_monotonic_cst : bool Whether there are monotonic constraints that apply. False iff monotonic_cst is None. NgMbP?Fg?c&URUUU5 [U5nU cU S- n [U [R5(a5[ R "U /URS-[ RS9n O#[ R"U [ RS9n [U [5(aU /URS-n [ R"U [ RS9n U c@[ R"URS[R[ RS9n O#[ R"U [ RS9n [ R "U [R:g5UlU c1[ R$"URS[ RS9n O#[ R"U [ RS9n [ R "[ R&"U S:HU [R:g55(a [)S5eURSS:Hn[+XX#UU5UlU S- n[/UU UU U U UUUUUUUUS9UlXlX@lXPlX`lXplXlUUlXl Xl!Xl"Xl#Xl$UUl%URSUl&UUl'/Ul(/Ul)SUl*SUl+SUl,SUl-UR]5 SUl/g) Nrdtype)r fill_valuerD)rrDz7Categorical features cannot have monotonic constraints.r)X_binnedn_bins_non_missingmissing_values_bin_idxhas_missing_valuesis_categorical monotonic_cstl2_regularizationmin_hessian_to_splitmin_samples_leafmin_gain_to_splithessians_are_constantfeature_fraction_per_splitrng n_threadsr@)0_validate_parametersr isinstancenumbersIntegralnparrayruint32asarraybooluint8fullr NO_CSTint8anywith_monotonic_cstzeros logical_and ValueErrorr histogram_builderrsplitterrFmax_leaf_nodes max_depthrNrOrGrHrIrJrKinteraction_cstrL shrinkage n_featuresrSsplittable_nodesfinalized_leavestotal_find_split_timetotal_compute_hist_timetotal_apply_split_timen_categorical_splits_initialize_rootn_nodes)r%rF gradientshessiansrhrirNrOrMn_binsrGrIrJrKrjrLrQrRrkrSrPrHs r&r'TreeGrower.__init__s, !!    0 :  %!'!  ('*:*: ; ;!##$x~~a'88 " "$,>bii!P  ($ / /"4!5q8I!I ZZ(:"((K  GGnnQ'.55ggM JJ}BGGDM"$&&:M:T:T)T"U  !XXHNN1,=RXXNNZZbhhGN 66 NN!#]6I6P6P%P   VW W (q 1Q 6!1 i3H)" "(! 1#91)'/!5-/"7'A  ! ," 0!2"4&<#"4,*.!2""..+" " "%("'*$&)#$%!  r)c"UR[R:wa [S5eURR (d [ S5eUS:a[ SRU55eUS:a[ SRU55eg)zVValidate parameters passed to __init__. Also validate parameters passed to splitter. zX_binned must be of type uint8.zMX_binned should be passed as Fortran contiguous array for maximum efficiency.rz&min_gain_to_split={} must be positive.z)min_hessian_to_split={} must be positive.N)rDrXr]NotImplementedErrorflags f_contiguousreformat)r%rFrOrMs r&rTTreeGrower._validate_parametersfs >>RXX %%&GH H~~**0  q 8??@QR  ! #;BBCWX  $r)cUR(a#UR5 UR(aM#UR5 g)z#Grow the tree, from root to leaves.N)rm split_next_apply_shrinkage)r%s r&growTreeGrower.grows1## OO ### r)chURH"nU=RUR-slM$ g)avMultiply leaves values by shrinkage parameter. This must be done at the very end of the growing process. If this were done during the growing process e.g. in finalize_leaf(), then a leaf would be shrunk but its sibling would potentially not be (if it's a non-leaf), which would lead to a wrong computation of the 'middle' value needed to enforce the monotonic constraints. N)rnrrk)r%leafs r&rTreeGrower._apply_shrinkages&))D JJ$.. (J*r)c "[5nURbT[5R"UR6n[R "U[R [U5S9nUSnOSnSnURRURRU5nU=R[5U- - sl [5nURRSnSn[R"XC5nUSR!5nURR"(aURR$SU-n OUSR!5n ['UURRSUUU SS9UlUR(R*SUR,-:aUR/UR(5 gXRR0:aUR/UR(5 gURb=[3[UR55UR(lX R(lX@R(lUR;UR(5 U=R<[5U- - slg)z/Initialize root node and finalize it if needed.NrDcountrrrrrr#r$rrr)timerjsetunionrXfromiterrZlenrfcompute_histograms_bruterg partitionrprFrr[sumrPrvrrootrrN_finalize_leafrMrangerrr"_compute_best_split_and_pushro) r%ticrarbitrary_featurer"rrhistogram_arrayrrs r&rsTreeGrower._initialize_rootsf    +"u{{D,@,@A !{{  =M9N  !1 3 #  !  ++DD MM # #   $$ 4$fMM''* **Z%BC'8<<>  ! ! 7 711::1= IL*>:>>@L==22$'%   99  T%:%:!: :    *  --<< <    *     +05c$:N:N6O0PDII -)9II &)  ))$))4 ""dfsl2"r)c rURRURURURUR UR URURURS9Ul URRS::aURU5 g[URU5 g)a5Compute the best possible split (SplitInfo) of a given node. Also push it in the heap of splittable nodes if gain isn't zero. The gain of a node is 0 if either all the leaves are pure (best gain = 0), or if no split would satisfy the constraints, (min_hessians_to_split, min_gain_to_split, min_samples_leaf) )rr"rrr lower_bound upper_boundrrN)rgfind_node_splitrr"rrrr+r,rrr1rrrmr%nodes r&r'TreeGrower._compute_best_split_and_pushs--77nn,,****1111!228   ??  1 $    % T**D 1r)c  [UR5n[5nURR UR UR 5unnnU=R[5U- - slURS-n[UR5[UR5-nUS- n[UUURURU-UR RUR RUR RS9n[UUUR UR UR R"UR R$UR R&S9n XlXlUR,b@UR/U5uUlUlUR2U lUR0U lUR4UR R6(d(UR8U R8:UR lU=R<S- slU=R>UR R@- slURBbDXpRB:Xa5UREU5 UREU 5 URG5 X4$URHb4X`RH:Xa%UREU5 UREU 5 X4$UR8URJS-:aUREU5 U R8URJS-:aUREU 5 URL(aURNUR R6[PRR:XaURT=pURV=pOURXU RX-S- nURNUR R6[PRZ:XaURTUpXRVpOXRVpURTUpUR]X5 U R]X5 UR^(+nU R^(+nU(dU(Ga`UR R`SnU R R`SnUU:aUnU nOU nUn[5nURbReUR UR05Ul3URbRiURfURfUR05Ul3SUl3U=Rj[5U- - sl5[5nU(aURmU5 U(aURmU 5 U=Rn[5U- - sl7X4HnUR^(dMU?3M U?3X4$)zSplit the node with highest potential gain. Returns ------- left : TreeNode The resulting left child. right : TreeNode The resulting right child. rrrNr)8rrmrrg split_indicesrrrqrrrnrr#sum_gradient_leftsum_hessian_left value_leftr$sum_gradient_rightsum_hessian_right value_rightr!r rj_compute_interactionsrrrI feature_idxrmissing_go_to_leftrtrrrJrhr_finalize_splittable_nodesrirNrbrKr r_r+r,rPOSrrrrfrr"compute_histograms_subtractionrprro)r%rrsample_indices_leftsample_indices_rightright_child_posr n_leaf_nodesleft_child_noderight_child_node lower_left lower_right upper_left upper_rightmidshould_split_leftshould_split_rightn_samples_leftn_samples_rightsmallest_child largest_childchilds r&rTreeGrower.split_nextst,,-f MM ' '9L9L M    ##tv|3# Q4001C8M8M4NN  ". 00///A//;;99//,, $/+::..//<<:://-- ,)    + **40 07 77  41@0P0P  -&&t'B'BC  )),<,F,FF OO .   !!T__%C%CC!    *|?R?R/R    0    0 1  + + -"4 4 >> %%>>*A    0    0 1"4 4  $ $t'<'#>?&--.,0+D+DD +/+D+DD [&,,/?/E/EEJ&&t'B'BC*../.2-F-F /24M4M-02K2K /3/H/H#  / / G  0 0 J!0 7 77!1!9!99  2-;;AA!DN.==CCAFO/!0 0 !1 / &C(,(>(>(W(W--~/N/N)N %&&EEOO"--"33  $#DO  ( (DFSL 8 (&C 11/B!112BC  & &$&3, 6 &*<===(= O00r)cN[5n/nURH[nURRURU;dM,UR U5 UR URU5 M] [R"U[R[U5S9U4$)aCompute features allowed by interactions to be inherited by child nodes. Example: Assume constraints [{0, 1}, {1, 2}]. 1 <- Both constraint groups could be applied from now on / \ 1 2 <- Left split still fulfills both constraint groups. / \ / \ Right split at feature 2 has only group {1, 2} from now on. LightGBM uses the same logic for overlapping groups. See https://github.com/microsoft/LightGBM/issues/4481 for details. Parameters: ---------- node : TreeNode A node that might have children. Based on its feature_idx, the interaction constraints for possible child nodes are computed. Returns ------- allowed_features : ndarray, dtype=uint32 Indices of features allowed to split for children. interaction_cst_indices : list of ints Indices of the interaction sets that have to be applied on splits of child nodes. The fewer sets the stronger the constraint as fewer sets contain fewer features. r) rrrrrjappendupdaterXrrZr)r%rrris r&r TreeGrower._compute_interactionss>5"$--A**d.B.B1.EE'..q1 ''(<(Helper used in make_predictor to set the TreePredictor fields.rrr1rTrrr bin_thresholdrrJ num_threshold bitset_idxleft)rrGnext_free_node_idxnext_free_bitset_idxright)rrrr1rrrbin_idxrrJrXinfleft_cat_bitsetrrr r!) rrr grower_noderrGrrrrrr categoriess r&rrs  .D))DM%%DM)"--22V V %%DMY!A%';;;''J%11:3E3E%#!+!>!>D '66D /rs]#(E6 ($sAsAlS  S  zKr)