-iSrSSKrSSKJrJr SSKJrJr SSKJ r J r J r SSK r SSKJr SSKJr SS KJrJrJrJrJrJr SS KJr SS KJrJr SS KJr SS K J!r!J"r"J#r#J$r$J%r% SSK&J'r'J(r( SSK)J*r*J+r+ SSK,J-r-J.r.J/r/ SSK0J1r1 SSK2J3r3J4r4 SSK5J6r6J7r7 SSK8J9r9J:r:J;r;JJ?r?J@r@ /SQrA\ R"\ R5RrESrFSrGSrHS.SjrI"SS\\?\S9rJSrK"S S!\\J\S9rL"S"S#\\J\S9rM"S$S%\L5rN"S&S'\M5rO"S(S)\L5rP"S*S+\M5rQ"S,S-\\J5rRg)/a2 Forest of trees-based ensemble methods. Those methods include random forests and extremely randomized trees. The module structure is the following: - The ``BaseForest`` base class implements a common ``fit`` method for all the estimators in the module. The ``fit`` method of the base ``Forest`` class calls the ``fit`` method of each sub-estimator on random samples (with replacement, a.k.a. bootstrap) of the training set. The init of the sub-estimator is further delegated to the ``BaseEnsemble`` constructor. - The ``ForestClassifier`` and ``ForestRegressor`` base classes further implement the prediction logic by computing an average of the predicted outcomes of the sub-estimators. - The ``RandomForestClassifier`` and ``RandomForestRegressor`` derived classes provide the user with concrete implementations of the forest ensemble method using classical, deterministic ``DecisionTreeClassifier`` and ``DecisionTreeRegressor`` as sub-estimator implementations. - The ``ExtraTreesClassifier`` and ``ExtraTreesRegressor`` derived classes provide the user with concrete implementations of the forest ensemble method using the extremely randomized trees ``ExtraTreeClassifier`` and ``ExtraTreeRegressor`` as sub-estimator implementations. Single and multi-output problems are both handled. N)ABCMetaabstractmethod)IntegralReal)catch_warnings simplefilterwarn)hstack)issparse)ClassifierMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context is_classifier)DataConversionWarning)accuracy_scorer2_score) OneHotEncoder)BaseDecisionTreeDecisionTreeClassifierDecisionTreeRegressorExtraTreeClassifierExtraTreeRegressor)DOUBLEDTYPE)check_random_statecompute_sample_weight)Interval RealNotInt StrOptions)get_tags)check_classification_targetstype_of_target)Paralleldelayed)_check_feature_names_in_check_sample_weight _num_samplescheck_is_fitted validate_data) BaseEnsemble_partition_estimators)ExtraTreesClassifierExtraTreesRegressorRandomForestClassifierRandomForestRegressorRandomTreesEmbeddingcUcU$[U[5(a#X:aSn[URX55eU$[U[5(a[ [ X-5S5$g)a6 Get the number of samples in a bootstrap sample. Parameters ---------- n_samples : int Number of samples in the dataset. max_samples : int or float The maximum number of samples to draw from the total available: - if float, this indicates a fraction of the total and should be the interval `(0.0, 1.0]`; - if int, this indicates the exact number of samples; - if None, this indicates the total number of samples. Returns ------- n_samples_bootstrap : int The total number of samples to draw for the bootstrap sample. Nz6`max_samples` must be <= n_samples={} but got value {}r-) isinstancer ValueErrorformatrmaxround) n_samples max_samplesmsgs K/var/www/html/venv/lib/python3.13/site-packages/sklearn/ensemble/_forest.py_get_n_samples_bootstrapr?[si(+x((  "JCSZZ ?@ @+t$$501155%cZ[U5nURSX[RS9nU$)z9 Private function used to _parallel_build_trees function.rdtype)rrandintnpint32) random_stater;n_samples_bootstraprandom_instancesample_indicess r>_generate_sample_indicesrK|s6)6O$,, 9-N r@c[XU5n[R"X1S9nUS:Hn[R"U5nXenU$)z9 Private function used to forest._set_oob_score function. minlengthr)rKrEbincountarange)rGr;rHrJ sample_countsunsampled_mask indices_rangeunsampled_indicess r>_generate_unsampled_indicesrUsK.!4NKKDM"a'NIIi(M%5 r@c 6US:a[SUS-U4-5 U(aURSn Uc%[R"U 4[RS9n OUR 5n [ URX5n [R"XS9nX-n US:Xa1[5 [S[5 U [S X=S 9-n SSS5 OUS :Xa U [S X=S 9-n URUUU S U S9 U$URUUUS U S9 U$!,(df  N8=f)z8 Private function used to fit a single tree in parallel.r-zbuilding tree %d of %drNrBrM subsampleignoreauto)indicesbalanced_subsamplebalancedF) sample_weight check_inputmissing_values_in_feature_mask)printshaperEonesfloat64copyrKrGrOrrDeprecationWarningr_fit)tree bootstrapXyr]tree_idxn_treesverbose class_weightrHr_r;curr_sample_weightrZrQs r>_parallel_build_treesrps7{ &(Q,)@@AGGAJ  !#)RZZ!H !.!3!3!5 *   y  GA + ; &!X'9:"&;FA&WW""!1 1 "7 A"W W  ,+I  K '+I   K-"!s &D  DcB^\rSrSr%Sr\"\SSSS9/S/S\/\S/S/S /S/S\"\S S S S9\"\SSSS9/S .r \ \ S'\ S!\ "5SSSSSSSSS. U4Sjjj5rSrSr\"SS9S"Sj5r\ S"Sj5rSrSrSr\S5rSr\S5rU4SjrS rU=r$)# BaseForestzq Base class for forests of trees. Warning: This class should not be used directly. Use derived classes instead. r-NleftclosedbooleanrGrm?right) n_estimatorsrh oob_scoren_jobsrGrm warm_startr<_parameter_constraintsFr estimator_paramsrhr|r}rGrmr~rnr<c >[T U]UUUS9 X@lXPlX`lXplXlXlXlXl g)N) estimatorr{r) super__init__rhr|r}rGrmr~rnr< selfrr{rrhr|r}rGrmr~rnr< __class__s r>rBaseForest.__init__sM %-  #" ( $(&r@c^URT5m[URURSS9"U4SjUR55n[ R "U5R$)a Apply trees in the forest to X, return leaf indices. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csr_matrix``. Returns ------- X_leaves : ndarray of shape (n_samples, n_estimators) For each datapoint x in X and for each tree in the forest, return the index of the leaf x ends up in. threadsr}rmpreferc3X># UHn[UR5"TSS9v M! g7fFr^N)r'apply.0rgris r> #BaseForest.apply..s# Q@P'$** aU 3@P'*)_validate_X_predictr&r}rm estimators_rEarrayT)rriresultss ` r>rBaseForest.applys^"  $ $Q ';;LL  R@P@P Q R xx """r@c^URT5m[URURSS9"U4SjUR55nS/nUR UVs/sHoDR SPM sn5 [R"U5R5n[U5R5U4$s snf)a Return the decision path in the forest. .. versionadded:: 0.18 Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csr_matrix``. Returns ------- indicator : sparse matrix of shape (n_samples, n_nodes) Return a node indicator matrix where non zero elements indicates that the samples goes through the nodes. The matrix is of CSR format. n_nodes_ptr : ndarray of shape (n_estimators + 1,) The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]] gives the indicator value for the i-th estimator. rrc3X># UHn[UR5"TSS9v M! g7fr)r' decision_pathrs r>r+BaseForest.decision_path..<s* ( D&& 'u =(rrr-) rr&r}rmrextendrarErcumsum sparse_hstacktocsr)rri indicatorsn_nodesi n_nodes_ptrs ` r>rBaseForest.decision_paths0  $ $Q ';;LL  ((  #J7Jq J78hhw'..0 Z(..0+==8sB<Tprefer_skip_nested_validationc " ^^^^^ ^ ^ [T5(a [S5e[TTTSS[SS9umm[ TR 5"TR S9nURTTRRS9m Tb [TT5m[T5(aTR5 [R"T5mTRS :Xa"TRS S :Xa[!S ["S S 9 TRS :Xa[R$"TS 5mTR S:XaN[R&"TS:5(a [S5e[R("T5S::a [S5eTRuTlTlTR/T5umn[1TSS5[2:wdTR4R6(d[R8"T[2S9mUb TbTU-mOUmTR:(dTR<b [S5eTR:(a"[?TRSTR<S9m OSm T Tl TRC5 TR:(dTRD(a [S5e[GTRH5nTRJ(a[MTS5(d/Tl'TRP[STRN5- nUS:a-[STRP[STRN54-5eUS:Xa [!S5 OTRJ(a@[STRN5S:a'URU[V[STRN5S9 [YU5Vs/sHnTR[SUS9PM snm []TR^TR`SS9"UU U UUU U4Sj[cT 555m TRNReT 5 TRD(aUS:d[MTS5(d~[gT5n U S :Xd[iT5(aU S!:Xa[S"U S#35e[kTRD5(aTRmTTTRDS$9 OTRmTT5 [MTS%5(a8TR,S :Xa(TRnSTl7TRpSTl8T$s snf)&a Build a forest of trees from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csc_matrix``. y : array-like of shape (n_samples,) or (n_samples, n_outputs) The target values (class labels in classification, real numbers in regression). sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Splits that would create child nodes with net zero or negative weight are ignored while searching for a split in each node. In the case of classification, splits are also ignored if they would result in any single class carrying a negative weight in either child node. Returns ------- self : object Fitted estimator. z3sparse multilabel-indicator for y is not supported.TcscF) multi_output accept_sparserCensure_all_finite criterion)estimator_nameNr r-zA column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().) stacklevel)r-poissonrzLSome value(s) of y are negative which is not allowed for Poisson regression.zLSum of y is not strictly positive which is necessary for Poisson regression.rCrBzl`max_sample` cannot be set if `bootstrap=False`. Either switch to `bootstrap=True` or set `max_sample=None`.)r;r<z6Out of bag estimation only available if bootstrap=TruerzTn_estimators=%d must be larger or equal to len(estimators_)=%d when warm_start==TruezJWarm-start fitting without increasing n_estimators does not fit new trees.size)appendrGrrc3># UHKup[[5"UTRTT TU[T5TRTR TTS9 v MM g7f))rmrnrHr_N)r'rprhlenrmrn) rrtrir_rHr]rtreesrjs r>r!BaseForest.fit..s\-DA-.NN!J LL!%!2!2(;3Q -sAA oob_score_unknownzmulticlass-multioutputz@The type of target cannot be used to compute OOB estimates. Got zv while only the following are supported: continuous, continuous-multioutput, binary, multiclass, multilabel-indicator.)scoring_functionclasses_)9r r7r,rtyperr'_compute_missing_values_in_feature_maskr__name__r) sort_indicesrE atleast_1dndimrar rreshapeanysum _n_samples n_outputs__validate_y_class_weightgetattrrflags contiguousascontiguousarrayrhr<r?_n_samples_bootstrap_validate_estimatorr|rrGr~hasattrrr{rrDMAX_INTrange_make_estimatorr&r}rm enumeraterr%rcallable_set_oob_score_and_attributes n_classes_r) rrirjr]rexpanded_class_weightrGn_more_estimatorsry_typer_rHrs ```` @@@r>fitBaseForest.fitGs: A;;RS S  # 1(4>>B  = =$.."9"9 >  '  $0BM A;; NN  MM!  66Q;1771:? ?&  66Q; 1g&A >>Y &vva!e}} :vvayA~ ; ,-77(#'#@#@#C  1gt $ .agg6H6H$$Qf5A ,( -0E E 5 ~~$"2"2">%  ^^":''!*$2B2B# #' $7!   "~~$..UV V)$*;*;< gdM&B&B!D  --D4D4D0EE q <$$c$*:*:&;<=  ! # !  3t'7'7#81#<$$W3t7G7G3H$I011A$$E $M1E{{  &e, E,    # #E * >>  !|)D)D#A&F"d##2J(J !&&,X.88''22q4>>3221a8 4 $ $A)="ooa0DO MM!,DM Cs%T cg)aCompute and set the OOB score and attributes. Parameters ---------- X : array-like of shape (n_samples, n_features) The data matrix. y : ndarray of shape (n_samples, n_outputs) The target matrix. scoring_function : callable, default=None Scoring function for OOB score. Default depends on whether this is a regression (R2 score) or classification problem (accuracy score). Nrrirjrs r>r(BaseForest._set_oob_score_and_attributessr@c[U5(aUR5nURSnURn[ U5(a#[ US5(aX0R SU4nOUSU4n[R"U[RS9n[R"X44[RS9n[UUR5nURHPn [U RUU5n UR!XU SS245n XjS4==U - ss'XzSS24==S- ss'MR [#U5HGn US:HR%5(a['S[(5 SXwS:H'USU 4==USU /4-ss'MI U$) aSCompute and set the OOB score. Parameters ---------- X : array-like of shape (n_samples, n_features) The data matrix. y : ndarray of shape (n_samples, n_outputs) The target matrix. Returns ------- oob_pred : ndarray of shape (n_samples, n_classes, n_outputs) or (n_samples, 1, n_outputs) The OOB predictions. rrr-)rarCrBN.zvSome inputs do not have OOB scores. This probably means too few trees were used to compute any reliable OOB estimates.)r rrarrrrrEzerosrcint64r?r<rrUrG_get_oob_predictionsrrr UserWarning) rrirjr; n_outputsoob_pred_shapeoob_pred n_oob_predrHrrTy_predks r>_compute_oob_predictions#BaseForest._compute_oob_predictions.s" A;; AGGAJ OO   74#>#>();YGN (I6N88. CXXy4BHHE 6     ))I ;&&#!  ..yr#BaseForest._validate_y_class_weightps $wr@c f[U5 URSRU5(aSnOSn[UU[SSUS9n[ U5(a[UR R[R:wd(URR[R:wa [S5eU$)z@ Validate X whenever one tries to predict, apply, predict_proba.rz allow-nanTcsrF)rCrresetrz3No support for np.int64 index based sparse matrices) r+r_support_missing_valuesr,rr rZrCrEintcindptrr7)rrirs r>rBaseForest._validate_X_predictts    A  6 6q 9 9 +  $    /   A;;AIIOOrww6!((..BGG:SRS Sr@cP[U5 [URSS9"SUR55nU(d-[R "UR [RS9$[R"US[RS9nU[R"U5- $)a The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative. Returns ------- feature_importances_ : ndarray of shape (n_features,) The values of this array sum to 1, unless all trees are single node trees consisting of only the root node, in which case it will be an array of zeros. r)r}rc3# UH7nURRS:dM[[5"US5v M9 g7f)r-feature_importances_N)tree_ node_countr'r)rrgs r>r2BaseForest.feature_importances_..s<I (zz$$q( ;GG T#9 : :(s AArBr)axisrC) r+r&r}rrErn_features_in_rcmeanr)rall_importancess r>rBaseForest.feature_importances_s* "$++iHI ((I   88D//rzzB B''/L!888r@c## URHsnUR(d1[R"UR[R S9v MEUR n[X RUR5v Mu g7f)NrB) rrhrErPrrFrGrKr)rrgseeds r>_get_estimators_indices"BaseForest._get_estimators_indicess`$$D>>iirxx@@ ((///4+D+D%sBBcLUR5Vs/sHoPM sn$s snf)aThe subset of drawn samples for each base estimator. Returns a dynamically generated list of indices identifying the samples used for fitting each member of the ensemble, i.e., the in-bag samples. Note: the list is re-created at each call to the property in order to reduce the object memory footprint by not storing the sampling data. Thus fetching the property may be slower than expected. )r )rrJs r>estimators_samples_BaseForest.estimators_samples_s'6:5Q5Q5ST5S>5STTTs !c>[TU]5n[UR5"URS9n[ U5R RUR lU$)Nr)r__sklearn_tags__rrrr# input_tags allow_nan)rtagsrrs r>rBaseForest.__sklearn_tags__sKw')(4>>B $,Y$7$B$B$L$L! r@)rrrhrnrrr<rr}rr|rGrmr~dr)r __module__ __qualname____firstlineno____doc__r rrr!rdict__annotations__rtuplerrrrrrrrrpropertyrr r r__static_attributes__ __classcell__rs@r>rrrrs;"(AtFCD[*T"'(; k  Zc' : Xq$v 6  $D ' '''<#4&>P5T6Tl    @D* 9 9D  U Ur@rr) metaclasscU"USS9nU [U5S:XaUS==U- ss'O)[[U55HnX%==XE- ss'M SSS5 g!,(df  g=f)z This is a utility function for joblib's Parallel. It can't go locally in ForestClassifier or ForestRegressor, because joblib complains that it cannot pickle it when placed there. Frr-rN)rr)predictrioutlock predictionrs r>_accumulate_predictionr(sV.J s8q= Fj F3s8_*-'% s AA A(c ^\rSrSrSr\S\"5SSSSSSSSS. U4Sjjj5r\S5r SU4S jjr S r S r S r S rU4SjrSrU=r$)ForestClassifieriz Base class for forest of trees-based classifiers. Warning: This class should not be used directly. Use derived classes instead. FNrrc 4>[T U]UUUUUUUUU U U S9 g)N rr{rrhr|r}rGrmr~rnr<rrrs r>rForestClassifier.__init__s8 %-%!%#  r@cURUSS9n[R"U5nURS:XaUS[R4nU$[R "USSS9nU$)aJCompute the OOB predictions for an individual tree. Parameters ---------- tree : DecisionTreeClassifier object A single decision tree classifier. X : ndarray of shape (n_samples, n_features) The OOB samples. Returns ------- y_pred : ndarray of shape (n_samples, n_classes, n_outputs) The OOB associated predictions. Frr .r)rstart) predict_probarEasarrayrnewaxisrollaxisrgrirs r>r%ForestClassifier._get_oob_predictions se ##A5#9F# ;;! CO,F  [[aq9F r@c >[TU]X5UlURRSS:XaURR SS9UlUc[ nU"U[ R"URSS95Ulg)a9Compute and set the OOB score and attributes. Parameters ---------- X : array-like of shape (n_samples, n_features) The data matrix. y : ndarray of shape (n_samples, n_outputs) The target matrix. scoring_function : callable, default=None Scoring function for OOB score. Defaults to `accuracy_score`. rr-rN) rroob_decision_function_rasqueezerrEargmaxrrrirjrrs r>r.ForestClassifier._set_oob_score_and_attributes%s{',g&Fq&L#  & & , ,R 0A 5*.*E*E*M*MSU*M*VD '  #- * ryy441= r@c~[U5 [R"U5nSnURb[R"U5n/Ul/Ul[R "UR[S9n[UR5Hkn[R"USS2U4SS9uodSS2U4'URRU5 UR RURS5 Mm UnURbSn[UR[5(aDURU;a[SUR-5eUR (a [#S5 URS:wdUR$(d+URS:XaS nO URn['UW5nX4$) NrBT)return_inverser)r\r[zVValid presets for class_weight include "balanced" and "balanced_subsample".Given "%s".aclass_weight presets "balanced" or "balanced_subsample" are not recommended for warm_start if the fitted data differs from the full dataset. In order to use "balanced" weights, use compute_class_weight ("balanced", classes, y). In place of y you can use a large enough sample of the full training set target to properly estimate the class frequency distributions. Pass the resulting weights as the class_weight parameter.r[r\)r$rErdrnrrrraintrruniquerr6strr7r~r rhr) rrjr y_originaly_store_unique_indicesr classes_k valid_presetsrns r>r)ForestClassifier._validate_y_class_weight=s}$Q' GGAJ $    (J !#!''!=t'A68ii!Q$7 3Iad3 MM  + OO " "9??1#5 6 ( #    (>M$++S11$$M9$&(,(9(9: ?? 2   $88$$(<<#-L#'#4#4L(=lJ(W%''r@cURU5nURS:Xa-URR[R "USS9SS9$USR SnURSRn[R"X0R4US9n[UR5H<nURUR[R "X&SS9SS9USS2U4'M> U$)aa Predict class for X. The predicted class of an input sample is a vote by the trees in the forest, weighted by their probability estimates. That is, the predicted class is the one with highest mean probability estimate across the trees. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csr_matrix``. Returns ------- y : ndarray of shape (n_samples,) or (n_samples, n_outputs) The predicted classes. r-r9rrBN) r2rrtakerEr<rarCemptyr)rriprobar; class_type predictionsrs r>r$ForestClassifier.predictrs*""1% ??a ==%%biiA&>Q%G Gaq)Iq)//J((I#?zRK4??+$(MM!$4$9$9IIehQ/a%:% AqD!,  r@cL^^^[U5 URT5m[URUR5un n[ R "UR5Vs/sH5n[ R"TRSU4[ RS9PM7 snm[R"5m[X RSS9"UUU4SjUR55 THnU[!UR5-nM [!T5S:XaTS$T$s snf)a Predict class probabilities for X. The predicted class probabilities of an input sample are computed as the mean predicted class probabilities of the trees in the forest. The class probability of a single tree is the fraction of samples of the same class in a leaf. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csr_matrix``. Returns ------- p : ndarray of shape (n_samples, n_classes), or a list of such arrays The class probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. rrB sharedmemr}rmrequirec3h># UH'n[[5"URTTT5v M) g7fr)r'r(r2)reri all_probar&s r>r1ForestClassifier.predict_proba..s1K % * +AOOQ 4 P P%s/2r-)r+rr/r{r}rErrrrarc threadingLockr&rmrr)rrir}_jrLrVr&s ` @@r>r2ForestClassifier.predict_probas,   $ $Q '-T->-> L 1 ]]4??3 3 HHaggaj!_BJJ 73 ~~ kJK %%K  E S))* *E y>Q Q<  ! s&predict_log_proba"ForestClassifier.predict_log_probas\*""1% ??a 66%= 4??+66%(+,Lr@ch>[TU]5nSURlSURlU$NT)rrclassifier_tags multi_labelrsparserrrs r>r!ForestClassifier.__sklearn_tags__s/w')+/(!% r@)rrr:rrr)rrrrrrrr staticmethodrrrr$r2r_rrr r!s@r>r*r*s    :6 03(j%N.`@r@r*c ^\rSrSrSr\S\"5SSSSSSSS.U4Sjjj5rSr\ S 5r SU4S jjr S r U4S jr S rU=r$)ForestRegressoriz Base class for forest of trees-based regressors. Warning: This class should not be used directly. Use derived classes instead. FNr)rrhr|r}rGrmr~r<c 2>[T U]UUUUUUUUU U S9 g)N) r{rrhr|r}rGrmr~r<r-) rrr{rrhr|r}rGrmr~r<rs r>rForestRegressor.__init__s5  %-%!#  r@c@^^^[U5 URT5m[URUR5un nUR S:a=[ R"TRSUR 4[ RS9mO0[ R"TRS[ RS9m[R"5m[X RSS9"UUU4SjUR55 T[UR5-mT$)a Predict regression target for X. The predicted regression target of an input sample is computed as the mean predicted regression targets of the trees in the forest. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Internally, its dtype will be converted to ``dtype=np.float32``. If a sparse matrix is provided, it will be converted into a sparse ``csr_matrix``. Returns ------- y : ndarray of shape (n_samples,) or (n_samples, n_outputs) The predicted values. r-rrBrQrRc3j># UH(n[[5"URTT/T5v M* g7fr)r'r(r$)rrUrir&y_hats r>r*ForestRegressor.predict..6s3K % * +AIIq5'4 H H%s03)r+rr/r{r}rrErrarcrXrYr&rmrr)rrir}rZr&ros ` @@r>r$ForestRegressor.predicts&   $ $Q '-T->-> L 1 ??Q HHaggaj$//:"**MEHHaggajr$ForestRegressor._get_oob_predictions?s^ aU3 ;;! Arzz2::56F Arzz1,-F r@c>[TU]X5RSS9UlURRSS:XaURRSS9UlUc[ nU"X R5Ulg)a3Compute and set the OOB score and attributes. Parameters ---------- X : array-like of shape (n_samples, n_features) The data matrix. y : ndarray of shape (n_samples, n_outputs) The target matrix. scoring_function : callable, default=None Scoring function for OOB score. Defaults to `r2_score`. r-r9rN)rrr;oob_prediction_rarrr=s r>r-ForestRegressor._set_oob_score_and_attributesXsy %w?EMMSTMU    % %b )Q .#'#7#7#?#?R#?#HD  #' *1.B.BCr@cr[R"U[SS9n[R"U[RSS9n[R"UR S[R SS9nURHnURRXU5 M! U[UR5-nU$)aFast partial dependence computation. Parameters ---------- grid : ndarray of shape (n_samples, n_target_features), dtype=DTYPE The grid points on which the partial dependence should be evaluated. target_features : ndarray of shape (n_target_features), dtype=np.intp The set of target features for which the partial dependence should be evaluated. Returns ------- averaged_predictions : ndarray of shape (n_samples,) The value of the partial dependence function on each grid point. C)rCorderr)rarCry) rEr3rintprrarcrrcompute_partial_dependencer)rgridtarget_featuresaveraged_predictionsrgs r>%_compute_partial_dependence_recursion5ForestRegressor._compute_partial_dependence_recursionns"zz$e37**_BGG3O!xx**Q-rzz $$D JJ 1 1'; % D$4$4 55##r@cF>[TU]5nSURlU$rbrrrrerfs r>r ForestRegressor.__sklearn_tags__!w')!% r@)rurrr)rrrrrrrrr$rhrrrrrr r!s@r>rjrjsw    6)V0D, $Dr@rjc^\rSrSr%Sr0\R E\R ES\"SS15\ \ S/0Er\ \ S'\RS5 SS SS S S S SS SSSSSSSS SSS.U4Sjjjr SrU=r$)r2ia4 A random forest classifier. A random forest is a meta estimator that fits a number of decision tree classifiers on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. Trees in the forest use the best split strategy, i.e. equivalent to passing `splitter="best"` to the underlying :class:`~sklearn.tree.DecisionTreeClassifier`. The sub-sample size is controlled with the `max_samples` parameter if `bootstrap=True` (default), otherwise the whole dataset is used to build each tree. For a comparison between tree-based ensemble models see the example :ref:`sphx_glr_auto_examples_ensemble_plot_forest_hist_grad_boosting_comparison.py`. This estimator has native support for missing values (NaNs). During training, the tree grower learns at each split point whether samples with missing values should go to the left or right child, based on the potential gain. When predicting, samples with missing values are assigned to the left or right child consequently. If no missing values were encountered for a given feature during training, then samples with missing values are mapped to whichever child has the most samples. Read more in the :ref:`User Guide `. Parameters ---------- n_estimators : int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22. criterion : {"gini", "entropy", "log_loss"}, default="gini" The function to measure the quality of a split. Supported criteria are "gini" for the Gini impurity and "log_loss" and "entropy" both for the Shannon information gain, see :ref:`tree_mathematical_formulation`. Note: This parameter is tree-specific. max_depth : int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. min_samples_split : int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions. min_samples_leaf : int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. min_weight_fraction_leaf : float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. max_features : {"sqrt", "log2", None}, int or float, default="sqrt" The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None, then `max_features=n_features`. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to `"sqrt"`. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features. max_leaf_nodes : int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes. min_impurity_decrease : float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19 bootstrap : bool, default=True Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree. oob_score : bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.accuracy_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`. n_jobs : int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls both the randomness of the bootstrapping of the samples used when building trees (if ``bootstrap=True``) and the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``). See :term:`Glossary ` for details. verbose : int, default=0 Controls the verbosity when fitting and predicting. warm_start : bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details. class_weight : {"balanced", "balanced_subsample"}, dict or list of dicts, default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. For multi-output problems, a list of dicts can be provided in the same order as the columns of y. Note that for multioutput (including multilabel) weights should be defined for each class of every column in its own dict. For example, for four-class multilabel classification weights should be [{0: 1, 1: 1}, {0: 1, 1: 5}, {0: 1, 1: 1}, {0: 1, 1: 1}] instead of [{1:1}, {2:5}, {3:1}, {4:1}]. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` The "balanced_subsample" mode is the same as "balanced" except that weights are computed based on the bootstrap sample for every tree grown. For multi-output, the weights of each column of y will be multiplied. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified. ccp_alpha : non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22 max_samples : int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples. - If int, then draw `max_samples` samples. - If float, then draw `max(round(n_samples * max_samples), 1)` samples. Thus, `max_samples` should be in the interval `(0.0, 1.0]`. .. versionadded:: 0.22 monotonic_cst : array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multiclass classifications (i.e. when `n_classes > 2`), - multioutput classifications (i.e. when `n_outputs_ > 1`), - classifications trained on data with missing values. The constraints hold over the probability of the positive class. Read more in the :ref:`User Guide `. .. versionadded:: 1.4 Attributes ---------- estimator_ : :class:`~sklearn.tree.DecisionTreeClassifier` The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. estimators_ : list of DecisionTreeClassifier The collection of fitted sub-estimators. classes_ : ndarray of shape (n_classes,) or a list of such arrays The classes labels (single output problem), or a list of arrays of class labels (multi-output problem). n_classes_ : int or list The number of classes (single output problem), or a list containing the number of classes for each output (multi-output problem). n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_outputs_ : int The number of outputs when ``fit`` is performed. feature_importances_ : ndarray of shape (n_features,) The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_decision_function_ : ndarray of shape (n_samples, n_classes) or (n_samples, n_classes, n_outputs) Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, `oob_decision_function_` might contain NaN. This attribute exists only when ``oob_score`` is True. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4 See Also -------- sklearn.tree.DecisionTreeClassifier : A decision tree classifier. sklearn.ensemble.ExtraTreesClassifier : Ensemble of extremely randomized tree classifiers. sklearn.ensemble.HistGradientBoostingClassifier : A Histogram-based Gradient Boosting Classification Tree, very fast for big datasets (n_samples >= 10_000). Notes ----- The default values for the parameters controlling the size of the trees (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. The features are always randomly permuted at each split. Therefore, the best found split may vary, even with the same training data, ``max_features=n_features`` and ``bootstrap=False``, if the improvement of the criterion is identical for several splits enumerated during the search of the best split. To obtain a deterministic behaviour during fitting, ``random_state`` has to be fixed. References ---------- .. [1] L. Breiman, "Random Forests", Machine Learning, 45(1), 5-32, 2001. Examples -------- >>> from sklearn.ensemble import RandomForestClassifier >>> from sklearn.datasets import make_classification >>> X, y = make_classification(n_samples=1000, n_features=4, ... n_informative=2, n_redundant=0, ... random_state=0, shuffle=False) >>> clf = RandomForestClassifier(max_depth=2, random_state=0) >>> clf.fit(X, y) RandomForestClassifier(...) >>> print(clf.predict([[0, 0, 0, 0]])) [1] rnr[r\Nrsplitterginir r-rxsqrtTFrr max_depthmin_samples_splitmin_samples_leafmin_weight_fraction_leaf max_featuresmax_leaf_nodesmin_impurity_decreaserhr|r}rGrmr~rn ccp_alphar< monotonic_cstc>[TU][5USU U U U UUUUS9 X lX0lX@lXPlX`lXplXl Xl UUl UUl gN rrrrrrrrrGrrr,) rrrrrrrrrrrrrrr{rrrrrrrrrhr|r}rGrmr~rnrr<rrs r>rRandomForestClassifier.__init__s~. ,.%  %!%#/  4#"!2 0(@%(,%:"*"r@ rrrrrrrrrrrrrrrrr*rrr"rlistrpoprrr r!s@r>r2r2sAF $  1 1 $ 7 7 $  ,j9 :      $D z*:#!$!+:#:#r@r2c^\rSrSr%Sr0\R E\R Er\\ S'\RS5 SSSSSS S SS S S SSS S S SSS.U4Sjjjr Sr U=r $)r3i#a0 A random forest regressor. A random forest is a meta estimator that fits a number of decision tree regressors on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. Trees in the forest use the best split strategy, i.e. equivalent to passing `splitter="best"` to the underlying :class:`~sklearn.tree.DecisionTreeRegressor`. The sub-sample size is controlled with the `max_samples` parameter if `bootstrap=True` (default), otherwise the whole dataset is used to build each tree. This estimator has native support for missing values (NaNs). During training, the tree grower learns at each split point whether samples with missing values should go to the left or right child, based on the potential gain. When predicting, samples with missing values are assigned to the left or right child consequently. If no missing values were encountered for a given feature during training, then samples with missing values are mapped to whichever child has the most samples. For a comparison between tree-based ensemble models see the example :ref:`sphx_glr_auto_examples_ensemble_plot_forest_hist_grad_boosting_comparison.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_estimators : int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22. criterion : {"squared_error", "absolute_error", "friedman_mse", "poisson"}, default="squared_error" The function to measure the quality of a split. Supported criteria are "squared_error" for the mean squared error, which is equal to variance reduction as feature selection criterion and minimizes the L2 loss using the mean of each terminal node, "friedman_mse", which uses mean squared error with Friedman's improvement score for potential splits, "absolute_error" for the mean absolute error, which minimizes the L1 loss using the median of each terminal node, and "poisson" which uses reduction in Poisson deviance to find splits. Training using "absolute_error" is significantly slower than when using "squared_error". .. versionadded:: 0.18 Mean Absolute Error (MAE) criterion. .. versionadded:: 1.0 Poisson criterion. max_depth : int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. min_samples_split : int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions. min_samples_leaf : int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. min_weight_fraction_leaf : float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. max_features : {"sqrt", "log2", None}, int or float, default=1.0 The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None or 1.0, then `max_features=n_features`. .. note:: The default of 1.0 is equivalent to bagged trees and more randomness can be achieved by setting smaller values, e.g. 0.3. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to 1.0. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features. max_leaf_nodes : int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes. min_impurity_decrease : float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19 bootstrap : bool, default=True Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree. oob_score : bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.r2_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`. n_jobs : int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls both the randomness of the bootstrapping of the samples used when building trees (if ``bootstrap=True``) and the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``). See :term:`Glossary ` for details. verbose : int, default=0 Controls the verbosity when fitting and predicting. warm_start : bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details. ccp_alpha : non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22 max_samples : int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples. - If int, then draw `max_samples` samples. - If float, then draw `max(round(n_samples * max_samples), 1)` samples. Thus, `max_samples` should be in the interval `(0.0, 1.0]`. .. versionadded:: 0.22 monotonic_cst : array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonically increasing - 0: no constraint - -1: monotonically decreasing If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multioutput regressions (i.e. when `n_outputs_ > 1`), - regressions trained on data with missing values. Read more in the :ref:`User Guide `. .. versionadded:: 1.4 Attributes ---------- estimator_ : :class:`~sklearn.tree.DecisionTreeRegressor` The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. estimators_ : list of DecisionTreeRegressor The collection of fitted sub-estimators. feature_importances_ : ndarray of shape (n_features,) The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_outputs_ : int The number of outputs when ``fit`` is performed. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_prediction_ : ndarray of shape (n_samples,) or (n_samples, n_outputs) Prediction computed with out-of-bag estimate on the training set. This attribute exists only when ``oob_score`` is True. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4 See Also -------- sklearn.tree.DecisionTreeRegressor : A decision tree regressor. sklearn.ensemble.ExtraTreesRegressor : Ensemble of extremely randomized tree regressors. sklearn.ensemble.HistGradientBoostingRegressor : A Histogram-based Gradient Boosting Regression Tree, very fast for big datasets (n_samples >= 10_000). Notes ----- The default values for the parameters controlling the size of the trees (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. The features are always randomly permuted at each split. Therefore, the best found split may vary, even with the same training data, ``max_features=n_features`` and ``bootstrap=False``, if the improvement of the criterion is identical for several splits enumerated during the search of the best split. To obtain a deterministic behaviour during fitting, ``random_state`` has to be fixed. The default value ``max_features=1.0`` uses ``n_features`` rather than ``n_features / 3``. The latter was originally suggested in [1], whereas the former was more recently justified empirically in [2]. References ---------- .. [1] L. Breiman, "Random Forests", Machine Learning, 45(1), 5-32, 2001. .. [2] P. Geurts, D. Ernst., and L. Wehenkel, "Extremely randomized trees", Machine Learning, 63(1), 3-42, 2006. Examples -------- >>> from sklearn.ensemble import RandomForestRegressor >>> from sklearn.datasets import make_regression >>> X, y = make_regression(n_features=4, n_informative=2, ... random_state=0, shuffle=False) >>> regr = RandomForestRegressor(max_depth=2, random_state=0) >>> regr.fit(X, y) RandomForestRegressor(...) >>> print(regr.predict([[0, 0, 0, 0]])) [-8.32987858] rr squared_errorNr r-rxryTFrrrrrrrrrrhr|r}rGrmr~rr<rc >[TU][5USU U U U UUUS9 X lX0lX@lXPlX`lXplXl Xl UUl UUl gNr rr{rrhr|r}rGrmr~r<) rrrrrrrrrrrrrrr{rrrrrrrrrhr|r}rGrmr~rr<rrs r>rRandomForestRegressor.__init__[s{, +-%  %!#-  2#"!2 0(@%(,%:""*r@rrrrrrrrjrrrrrrrr r!s@r>r3r3#sob $  0 0$  6 6$Dz*8+"!$!)8+8+r@r3c^\rSrSr%Sr0\R E\R ES\"SS15\ \ S/0Er\ \ S'\RS5 SS SS S S S SS SSSSSSSS SSS.U4Sjjjr SrU=r$)r0ia0 An extra-trees classifier. This class implements a meta estimator that fits a number of randomized decision trees (a.k.a. extra-trees) on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. This estimator has native support for missing values (NaNs) for random splits. During training, a random threshold will be chosen to split the non-missing values on. Then the non-missing values will be sent to the left and right child based on the randomly selected threshold, while the missing values will also be randomly sent to the left or right child. This is repeated for every feature considered at each split. The best split among these is chosen. Read more in the :ref:`User Guide `. Parameters ---------- n_estimators : int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22. criterion : {"gini", "entropy", "log_loss"}, default="gini" The function to measure the quality of a split. Supported criteria are "gini" for the Gini impurity and "log_loss" and "entropy" both for the Shannon information gain, see :ref:`tree_mathematical_formulation`. Note: This parameter is tree-specific. max_depth : int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. min_samples_split : int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions. min_samples_leaf : int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. min_weight_fraction_leaf : float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. max_features : {"sqrt", "log2", None}, int or float, default="sqrt" The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None, then `max_features=n_features`. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to `"sqrt"`. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features. max_leaf_nodes : int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes. min_impurity_decrease : float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19 bootstrap : bool, default=False Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree. oob_score : bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.accuracy_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`. n_jobs : int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls 3 sources of randomness: - the bootstrapping of the samples used when building trees (if ``bootstrap=True``) - the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``) - the draw of the splits for each of the `max_features` See :term:`Glossary ` for details. verbose : int, default=0 Controls the verbosity when fitting and predicting. warm_start : bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details. class_weight : {"balanced", "balanced_subsample"}, dict or list of dicts, default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. For multi-output problems, a list of dicts can be provided in the same order as the columns of y. Note that for multioutput (including multilabel) weights should be defined for each class of every column in its own dict. For example, for four-class multilabel classification weights should be [{0: 1, 1: 1}, {0: 1, 1: 5}, {0: 1, 1: 1}, {0: 1, 1: 1}] instead of [{1:1}, {2:5}, {3:1}, {4:1}]. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` The "balanced_subsample" mode is the same as "balanced" except that weights are computed based on the bootstrap sample for every tree grown. For multi-output, the weights of each column of y will be multiplied. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified. ccp_alpha : non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22 max_samples : int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples. - If int, then draw `max_samples` samples. - If float, then draw `max_samples * X.shape[0]` samples. Thus, `max_samples` should be in the interval `(0.0, 1.0]`. .. versionadded:: 0.22 monotonic_cst : array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonically increasing - 0: no constraint - -1: monotonically decreasing If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multiclass classifications (i.e. when `n_classes > 2`), - multioutput classifications (i.e. when `n_outputs_ > 1`), - classifications trained on data with missing values. The constraints hold over the probability of the positive class. Read more in the :ref:`User Guide `. .. versionadded:: 1.4 Attributes ---------- estimator_ : :class:`~sklearn.tree.ExtraTreeClassifier` The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. estimators_ : list of DecisionTreeClassifier The collection of fitted sub-estimators. classes_ : ndarray of shape (n_classes,) or a list of such arrays The classes labels (single output problem), or a list of arrays of class labels (multi-output problem). n_classes_ : int or list The number of classes (single output problem), or a list containing the number of classes for each output (multi-output problem). feature_importances_ : ndarray of shape (n_features,) The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_outputs_ : int The number of outputs when ``fit`` is performed. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_decision_function_ : ndarray of shape (n_samples, n_classes) or (n_samples, n_classes, n_outputs) Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, `oob_decision_function_` might contain NaN. This attribute exists only when ``oob_score`` is True. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4 See Also -------- ExtraTreesRegressor : An extra-trees regressor with random splits. RandomForestClassifier : A random forest classifier with optimal splits. RandomForestRegressor : Ensemble regressor using trees with optimal splits. Notes ----- The default values for the parameters controlling the size of the trees (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. References ---------- .. [1] P. Geurts, D. Ernst., and L. Wehenkel, "Extremely randomized trees", Machine Learning, 63(1), 3-42, 2006. Examples -------- >>> from sklearn.ensemble import ExtraTreesClassifier >>> from sklearn.datasets import make_classification >>> X, y = make_classification(n_features=4, random_state=0) >>> clf = ExtraTreesClassifier(n_estimators=100, random_state=0) >>> clf.fit(X, y) ExtraTreesClassifier(random_state=0) >>> clf.predict([[0, 0, 0, 0]]) array([1]) rnr[r\Nrrrr r-rxrFrrc>[TU][5USU U U U UUUUS9 X lX0lX@lXPlX`lXplXl Xl UUl UUl gr) rrrrrrrrrrrrrrs r>rExtraTreesClassifier.__init__s~. )+%  %!%#/  4#"!2 0(@%(,%:""*r@rrrr!s@r>r0r0stl $  1 1 $ 7 7 $  ,j9 :      $D z*:+!$!+:+:+r@r0c^\rSrSr%Sr0\R E\R Er\\ S'\RS5 SSSSSS S SS S S SSS S S SSS .U4Sjjjr Sr U=r $)r1i a0, An extra-trees regressor. This class implements a meta estimator that fits a number of randomized decision trees (a.k.a. extra-trees) on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. This estimator has native support for missing values (NaNs) for random splits. During training, a random threshold will be chosen to split the non-missing values on. Then the non-missing values will be sent to the left and right child based on the randomly selected threshold, while the missing values will also be randomly sent to the left or right child. This is repeated for every feature considered at each split. The best split among these is chosen. Read more in the :ref:`User Guide `. Parameters ---------- n_estimators : int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22. criterion : {"squared_error", "absolute_error", "friedman_mse", "poisson"}, default="squared_error" The function to measure the quality of a split. Supported criteria are "squared_error" for the mean squared error, which is equal to variance reduction as feature selection criterion and minimizes the L2 loss using the mean of each terminal node, "friedman_mse", which uses mean squared error with Friedman's improvement score for potential splits, "absolute_error" for the mean absolute error, which minimizes the L1 loss using the median of each terminal node, and "poisson" which uses reduction in Poisson deviance to find splits. Training using "absolute_error" is significantly slower than when using "squared_error". .. versionadded:: 0.18 Mean Absolute Error (MAE) criterion. max_depth : int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. min_samples_split : int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions. min_samples_leaf : int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. min_weight_fraction_leaf : float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. max_features : {"sqrt", "log2", None}, int or float, default=1.0 The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None or 1.0, then `max_features=n_features`. .. note:: The default of 1.0 is equivalent to bagged trees and more randomness can be achieved by setting smaller values, e.g. 0.3. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to 1.0. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features. max_leaf_nodes : int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes. min_impurity_decrease : float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19 bootstrap : bool, default=False Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree. oob_score : bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.r2_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`. n_jobs : int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls 3 sources of randomness: - the bootstrapping of the samples used when building trees (if ``bootstrap=True``) - the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``) - the draw of the splits for each of the `max_features` See :term:`Glossary ` for details. verbose : int, default=0 Controls the verbosity when fitting and predicting. warm_start : bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details. ccp_alpha : non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22 max_samples : int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples. - If int, then draw `max_samples` samples. - If float, then draw `max_samples * X.shape[0]` samples. Thus, `max_samples` should be in the interval `(0.0, 1.0]`. .. versionadded:: 0.22 monotonic_cst : array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonically increasing - 0: no constraint - -1: monotonically decreasing If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multioutput regressions (i.e. when `n_outputs_ > 1`), - regressions trained on data with missing values. Read more in the :ref:`User Guide `. .. versionadded:: 1.4 Attributes ---------- estimator_ : :class:`~sklearn.tree.ExtraTreeRegressor` The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. estimators_ : list of DecisionTreeRegressor The collection of fitted sub-estimators. feature_importances_ : ndarray of shape (n_features,) The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_outputs_ : int The number of outputs. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_prediction_ : ndarray of shape (n_samples,) or (n_samples, n_outputs) Prediction computed with out-of-bag estimate on the training set. This attribute exists only when ``oob_score`` is True. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4 See Also -------- ExtraTreesClassifier : An extra-trees classifier with random splits. RandomForestClassifier : A random forest classifier with optimal splits. RandomForestRegressor : Ensemble regressor using trees with optimal splits. Notes ----- The default values for the parameters controlling the size of the trees (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values. References ---------- .. [1] P. Geurts, D. Ernst., and L. Wehenkel, "Extremely randomized trees", Machine Learning, 63(1), 3-42, 2006. Examples -------- >>> from sklearn.datasets import load_diabetes >>> from sklearn.model_selection import train_test_split >>> from sklearn.ensemble import ExtraTreesRegressor >>> X, y = load_diabetes(return_X_y=True) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, random_state=0) >>> reg = ExtraTreesRegressor(n_estimators=100, random_state=0).fit( ... X_train, y_train) >>> reg.score(X_test, y_test) 0.2727... rrrNr r-rxryFrrc >[TU][5USU U U U UUUS9 X lX0lX@lXPlX`lXplXl Xl UUl UUl gr) rrrrrrrrrrrrrrs r>rExtraTreesRegressor.__init__: s{, (*%  %!#-  2#"!2 0(@%(,%:""*r@rrrr!s@r>r1r1 s[z$  0 0$  6 6$Dz*8+"!$!)8+8+r@r1c 0^\rSrSr%Sr\"\SSSS9/\S/S/S/S .\RES S/0Er\ \ S 'S Hr \R\ 5 M S r SrSSSSSSSSSSSSS. U4SjjjrSSjrS Sjr\"SS9S U4Sjj5rSSjrSrU4SjrSrU=r$)!r4iu a An ensemble of totally random trees. An unsupervised transformation of a dataset to a high-dimensional sparse representation. A datapoint is coded according to which leaf of each tree it is sorted into. Using a one-hot encoding of the leaves, this leads to a binary coding with as many ones as there are trees in the forest. The dimensionality of the resulting representation is ``n_out <= n_estimators * max_leaf_nodes``. If ``max_leaf_nodes == None``, the number of leaf nodes is at most ``n_estimators * 2 ** max_depth``. For an example of applying Random Trees Embedding to non-linear classification, see :ref:`sphx_glr_auto_examples_ensemble_plot_random_forest_embedding.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_estimators : int, default=100 Number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22. max_depth : int, default=5 The maximum depth of each tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples. min_samples_split : int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` is the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions. min_samples_leaf : int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` is the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions. min_weight_fraction_leaf : float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided. max_leaf_nodes : int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes. min_impurity_decrease : float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19 sparse_output : bool, default=True Whether or not to return a sparse CSR matrix, as default behavior, or to return a dense array compatible with dense pipeline operators. n_jobs : int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`transform`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls the generation of the random `y` used to fit the trees and the draw of the splits for each feature at the trees' nodes. See :term:`Glossary ` for details. verbose : int, default=0 Controls the verbosity when fitting and predicting. warm_start : bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details. Attributes ---------- estimator_ : :class:`~sklearn.tree.ExtraTreeRegressor` instance The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. estimators_ : list of :class:`~sklearn.tree.ExtraTreeRegressor` instances The collection of fitted sub-estimators. feature_importances_ : ndarray of shape (n_features,) The feature importances (the higher, the more important the feature). n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_outputs_ : int The number of outputs when ``fit`` is performed. one_hot_encoder_ : OneHotEncoder instance One-hot encoder used to create the sparse embedding. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4 See Also -------- ExtraTreesClassifier : An extra-trees classifier. ExtraTreesRegressor : An extra-trees regressor. RandomForestClassifier : A random forest classifier. RandomForestRegressor : A random forest regressor. sklearn.tree.ExtraTreeClassifier: An extremely randomized tree classifier. sklearn.tree.ExtraTreeRegressor : An extremely randomized tree regressor. References ---------- .. [1] P. Geurts, D. Ernst., and L. Wehenkel, "Extremely randomized trees", Machine Learning, 63(1), 3-42, 2006. .. [2] Moosmann, F. and Triggs, B. and Jurie, F. "Fast discriminative visual codebooks using randomized clustering forests" NIPS 2007 Examples -------- >>> from sklearn.ensemble import RandomTreesEmbedding >>> X = [[0,0], [1,0], [0,1], [-1,0], [0,-1]] >>> random_trees = RandomTreesEmbedding( ... n_estimators=5, random_state=0, max_depth=1).fit(X) >>> X_sparse_embedding = random_trees.transform(X) >>> X_sparse_embedding.toarray() array([[0., 1., 1., 0., 1., 0., 0., 1., 1., 0.], [0., 1., 1., 0., 1., 0., 0., 1., 1., 0.], [0., 1., 0., 1., 0., 1., 0., 1., 0., 1.], [1., 0., 1., 0., 1., 0., 1., 0., 1., 0.], [0., 1., 1., 0., 1., 0., 0., 1., 1., 0.]]) r-Nrtrurmrw)r{r}rmr~ sparse_outputr)rrrrrr rxTrF) rrrrrrrr}rGrmr~c >[T U][5USSSU U U U SS9 X lX0lX@lXPlX`lXplXl g)N) rrrrrrrrrGFr) rrrrrrrrrr)rr{rrrrrrrr}rGrmr~rs r>rRandomTreesEmbedding.__init__< sg (*% %!)  .#!2 0(@%,%:"*r@c[S5e)Nz)OOB score not supported by tree embedding)NotImplementedErrorrs r>r2RandomTreesEmbedding._set_oob_score_and_attributesk s!"MNNr@c&URXUS9 U$)a1 Fit estimator. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input samples. Use ``dtype=np.float32`` for maximum efficiency. Sparse matrices are also supported, use sparse ``csc_matrix`` for maximum efficiency. y : Ignored Not used, present for API consistency by convention. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Splits that would create child nodes with net zero or negative weight are ignored while searching for a split in each node. In the case of classification, splits are also ignored if they would result in any single class carrying a negative weight in either child node. Returns ------- self : object Returns the instance itself. r]) fit_transform)rrirjr]s r>rRandomTreesEmbedding.fitn s6 1}= r@rc*>[UR5nUR[U5S9n[TU]XUS9 [ URS9UlURRURU55nURSUl U$)a  Fit estimator and transform dataset. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Input data used to build forests. Use ``dtype=np.float32`` for maximum efficiency. y : Ignored Not used, present for API consistency by convention. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Splits that would create child nodes with net zero or negative weight are ignored while searching for a split in each node. In the case of classification, splits are also ignored if they would result in any single class carrying a negative weight in either child node. Returns ------- X_transformed : sparse matrix of shape (n_samples, n_out) Transformed dataset. rr)rr-) rrGuniformr*rrrrone_hot_encoder_rrra_n_features_out)rrirjr]rndoutputrs r>r"RandomTreesEmbedding.fit_transform s4!!2!23 KK\!_K -  A 6 -Dget_feature_names_out*RandomTreesEmbedding.get_feature_names_out s( /0  d//0 0--99$?$D64& 1? 20  zz-v66  s3A>cl[U5 URRURU55$)a Transform dataset. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Input data to be transformed. Use ``dtype=np.float32`` for maximum efficiency. Sparse matrices are also supported, use sparse ``csr_matrix`` for maximum efficiency. Returns ------- X_transformed : sparse matrix of shape (n_samples, n_out) Transformed dataset. )r+r transformr)rris r>rRandomTreesEmbedding.transform s+ $$..tzz!}==r@cF>[TU]5nSURlU$rbrrfs r>r%RandomTreesEmbedding.__sklearn_tags__ rr@) rrrrrrrrrrr)NN)rrrrrr rrrrrparamrrrrrrrrrrrrr r!s@r>r4r4u svr"(AtFCDT"; k $  1 1 $ ) $DL""5)L IL-+!$!-+-+^O<5 6 D7@>&r@r4)rNNN)SrrXabcrrnumbersrrwarningsrrr numpyrE scipy.sparser rr baser rrrrr exceptionsrmetricsrr preprocessingrrgrrrrr tree._treerrutilsrrutils._param_validationr r!r" utils._tagsr#utils.multiclassr$r%utils.parallelr&r'utils.validationr(r)r*r+r,_baser.r/__all__iinforFr9rr?rKrUrprrr(r*rjr2r3r0r1r4rr@r>rs= J'"770!/.)'=FF"K.7  ((288  6B  , #'6rA!<7AH ( J gJZcnjGcLJ#-J#Z p+Op+f }++}+@ \+/\+~ p+Zpr@