-iN SSKJr SSKrSSKJr SSKJr SSKJ r SSK J r J r SSK Jr SS KJr SS KJrJrJr SS KJr SS KJrJr SS KJrJr SrSrS$SjrS%Sjr SSSSSSS.Sjr!\"SS/S/\"S15SS/\"\SSSS9/S/S/\S/S.SS 9SSSSSS!.S"j5r"\"SS/S/\"S15SS/\"\SSSS9/S/S/\S/S.SS 9SSSSSS!.S#j5r#g)&)IntegralN)issparse)digamma)mutual_info_score)KDTreeNearestNeighbors)scale)check_random_state)Interval StrOptionsvalidate_params)check_classification_targets)Paralleldelayed) check_array check_X_ycURnURS5nURS5n[R"X45n[ SUS9nUR U5 UR 5Sn[R"USS2S4S5n[USS9nURXSS S 9n[R"U5S - n[USS9nURXSS S 9n [R"U 5S - n [U5[U5-[R"[US -55- [R"[U S -55- n [SU 5$) aCompute mutual information between two continuous variables. Parameters ---------- x, y : ndarray, shape (n_samples,) Samples of two continuous random variables, must have an identical shape. n_neighbors : int Number of nearest neighbors to search for each point, see [1]_. Returns ------- mi : float Estimated mutual information in nat units. If it turned out to be negative it is replaced by 0. Notes ----- True mutual information can't be negative. If its estimate by a numerical method is negative, it means (providing the method is adequate) that the mutual information is close to 0 and replacing it by 0 is a reasonable strategy. References ---------- .. [1] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004.  chebyshev)metric n_neighborsrNr)rTF count_onlyreturn_distanceg?r)sizereshapenphstackr fit kneighbors nextafterr query_radiusarrayrmeanmax) xyr n_samplesxynnradiuskdnxnymis Y/var/www/html/venv/lib/python3.13/site-packages/sklearn/feature_selection/_mutual_info.py_compute_mi_ccr4s><I 'A 'A A6 B + FBFF2J ]]_Q F \\&B- +F + &B tU KB " B + &B tU KB " B   +   '''"q&/ " # '''"q&/ " # q":cURSnURS5n[R"U5n[R"U5n[R"U5n[ 5n[R "U5HnX:Hn [R "U 5n U S:ag[X*S- 5n URU S9 URX 5 UR5Sn [R"U SS2S4S5XI'XU 'XU 'M US:n [R "U 5nXYnXinX nXIn[U5n U RXSSS 9n[R"U5n[U5[R "[U55-[R "[U55- [R "[U55- n[#SU5$) aCCompute mutual information between continuous and discrete variables. Parameters ---------- c : ndarray, shape (n_samples,) Samples of a continuous random variable. d : ndarray, shape (n_samples,) Samples of a discrete random variable. n_neighbors : int Number of nearest neighbors to search for each point, see [1]_. Returns ------- mi : float Estimated mutual information in nat units. If it turned out to be negative it is replaced by 0. Notes ----- True mutual information can't be negative. If its estimate by a numerical method is negative, it means (providing the method is adequate) that the mutual information is close to 0 and replacing it by 0 is a reasonable strategy. References ---------- .. [1] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. rrr)rNrTFr)shaperr emptyr uniquesummin set_paramsr"r#r$rr%r&rr'r()cdrr+r. label_countsk_allr-labelmaskcountkrr/m_allr2s r3_compute_mi_cdrGSs@ I 'A XXi F88I&L HHY E  B1zt  19K+A MMaM ( FF17O "A<<!R%!4FL$K"T ! Dt I%L KE A \F B OOA$O NE HHUOE   '''%. ! " ''',' ( ) '''%. ! " q":r5cU(aU(a [X5$U(aU(d [XU5$U(dU(a [XU5$[XU5$)zCompute mutual information between two variables. This is a simple wrapper which selects a proper function to call based on whether `x` and `y` are discrete or not. )rrGr4)r)r* x_discrete y_discreters r3 _compute_mirLsE j && JaK00 JaK00aK00r5c#d# Uc[URS5n[U5(amUHfn[R"URS5nUR UUR US-pTUR XEX0RXE'Uv Mh gUHnUSS2U4v M g7f)a\Iterate over columns of a matrix. Parameters ---------- X : ndarray or csc_matrix, shape (n_samples, n_features) Matrix over which to iterate. columns : iterable or None, default=None Indices of columns to iterate over. If None, iterate over all columns. Yields ------ x : ndarray, shape (n_samples,) Columns of `X` in dense format. Nrr)ranger7rr zerosindptrdataindices)Xcolumnsir) start_ptrend_ptrs r3_iterate_columnsrXs  #{{A$A!"!ahhq1uow./ffY.GAii * +G  AAqD'MsB.B0autoFTdiscrete_featuresdiscrete_targetrcopy random_staten_jobsc .^^^[UTST(+S9unmURup[U[[45(a][U[5(aUS:Xa [ U5nO [ S5e[R"U [S9n U RU5 O:[USS9nURS:wa[R"U [S9n S X'OUn U )n [R"U 5(a[ U5(a [ S 5e[U5n [R"U 5(aUR[R US 9n[#US S 2U 4SSS 9US S 2U 4'[R$"S[R&"[R("US S 2U 45SS95n US S 2U 4==SU -U R+U[R,"U 54S9-- ss'T(d_[#TSS9mTS[R$"S[R&"[R("T555-U R+US9-- m[/US9"UUU4Sj[1[3U5U 555n[R4"U5$)a0Estimate mutual information between the features and the target. Parameters ---------- X : array-like or sparse matrix, shape (n_samples, n_features) Feature matrix. y : array-like of shape (n_samples,) Target vector. discrete_features : {'auto', bool, array-like}, default='auto' If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If 'auto', it is assigned to False for dense `X` and to True for sparse `X`. discrete_target : bool, default=False Whether to consider `y` as a discrete variable. n_neighbors : int, default=3 Number of neighbors to use for MI estimation for continuous variables, see [1]_ and [2]_. Higher values reduce variance of the estimation, but could introduce a bias. copy : bool, default=True Whether to make a copy of the given data. If set to False, the initial data will be overwritten. random_state : int, RandomState instance or None, default=None Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int, default=None The number of jobs to use for computing the mutual information. The parallelization is done on the columns of `X`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. .. versionadded:: 1.5 Returns ------- mi : ndarray, shape (n_features,) Estimated mutual information between each feature and the target in nat units. A negative value will be replaced by 0. References ---------- .. [1] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. .. [2] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. csc) accept_sparse y_numericrYz+Invalid string value for discrete_features.)dtypeF) ensure_2dboolTz1Sparse matrix `X` can't have continuous features.)r]N) with_meanr]rr)axisg|=)r)rg)r_c3Z># UH up[[5"UTUTT5v M" g7fN)rrL).0r)discrete_featurer\rr*s r3 _estimate_mi..=s1!#J A  Q#3_kRR#Js(+)rr7 isinstancestrrfr ValueErrorr r8fillrrdrOanyr astypefloat64r maximumr'absstandard_normalr:rziprXr&)rSr*r[r\rr]r^r_r+ n_features discrete_maskcontinuous_maskrngmeansr2s ` `` r3 _estimate_mirsEJ Qo:M NDAqGGI#c4[11 ' - - F*$,QK! !NOO48 ,-'(9UK  " "f ,HHZt`. Parameters ---------- X : array-like or sparse matrix, shape (n_samples, n_features) Feature matrix. y : array-like of shape (n_samples,) Target vector. discrete_features : {'auto', bool, array-like}, default='auto' If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If 'auto', it is assigned to False for dense `X` and to True for sparse `X`. n_neighbors : int, default=3 Number of neighbors to use for MI estimation for continuous variables, see [2]_ and [3]_. Higher values reduce variance of the estimation, but could introduce a bias. copy : bool, default=True Whether to make a copy of the given data. If set to False, the initial data will be overwritten. random_state : int, RandomState instance or None, default=None Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int, default=None The number of jobs to use for computing the mutual information. The parallelization is done on the columns of `X`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. .. versionadded:: 1.5 Returns ------- mi : ndarray, shape (n_features,) Estimated mutual information between each feature and the target in nat units. Notes ----- 1. The term "discrete features" is used instead of naming them "categorical", because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. 2. True mutual information can't be negative. If its estimate turns out to be negative, it is replaced by zero. References ---------- .. [1] `Mutual Information `_ on Wikipedia. .. [2] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. .. [3] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. .. [4] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy of a Random Vector", Probl. Peredachi Inf., 23:2 (1987), 9-16 Examples -------- >>> from sklearn.datasets import make_regression >>> from sklearn.feature_selection import mutual_info_regression >>> X, y = make_regression( ... n_samples=50, n_features=3, n_informative=1, noise=1e-4, random_state=42 ... ) >>> mutual_info_regression(X, y) array([0.117, 2.645, 0.0287]) FrZ)rrs r3mutual_info_regressionrEs*h   + !  r5c 8[U5 [UUUSUUUUS9$)aEstimate mutual information for a discrete target variable. Mutual information (MI) [1]_ between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2]_ and [3]_. Both methods are based on the idea originally proposed in [4]_. It can be used for univariate features selection, read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Feature matrix. y : array-like of shape (n_samples,) Target vector. discrete_features : 'auto', bool or array-like, default='auto' If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If 'auto', it is assigned to False for dense `X` and to True for sparse `X`. n_neighbors : int, default=3 Number of neighbors to use for MI estimation for continuous variables, see [2]_ and [3]_. Higher values reduce variance of the estimation, but could introduce a bias. copy : bool, default=True Whether to make a copy of the given data. If set to False, the initial data will be overwritten. random_state : int, RandomState instance or None, default=None Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int, default=None The number of jobs to use for computing the mutual information. The parallelization is done on the columns of `X`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. .. versionadded:: 1.5 Returns ------- mi : ndarray, shape (n_features,) Estimated mutual information between each feature and the target in nat units. Notes ----- 1. The term "discrete features" is used instead of naming them "categorical", because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. 2. True mutual information can't be negative. If its estimate turns out to be negative, it is replaced by zero. References ---------- .. [1] `Mutual Information `_ on Wikipedia. .. [2] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. .. [3] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. .. [4] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16 Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.feature_selection import mutual_info_classif >>> X, y = make_classification( ... n_samples=100, n_features=10, n_informative=2, n_clusters_per_class=1, ... shuffle=False, random_state=42 ... ) >>> mutual_info_classif(X, y) array([0.589, 0.107, 0.196, 0.0968 , 0., 0. , 0. , 0. , 0. , 0.]) TrZ)rrrs r3mutual_info_classifrs2j!#   + !  r5)rHrj)$numbersrnumpyr scipy.sparser scipy.specialrmetrics.clusterr neighborsrr preprocessingr utilsr utils._param_validationr r rutils.multiclassrutils.parallelrrutils.validationrrr4rGrLrXrrrr5r3rsR!!/0!&KK;.5<~FR 1 D  xvO ,^(&2I|L 1d6BC '(T"#'    q qhO ,^(&2I|L 1d6BC '(T"#'    s sr5