-i$SrSSKJrJr SSKrSSKJr SSK J r SSK J r J r SSKJrJrJrJr SSKJrJrJr SS KJrJr SS KJrJr SS KJrJrJ r S S K!J"r# S SK!J$r$ S SK!J%r& \RN"\(5RRr*Sr+"SS\\\5r,g)a6 ============================================================= Online Latent Dirichlet Allocation with variational inference ============================================================= This implementation is modified from Matthew D. Hoffman's onlineldavb code Link: https://github.com/blei-lab/onlineldavb )IntegralRealN)effective_n_jobs)gammaln logsumexp) BaseEstimatorClassNamePrefixFeaturesOutMixinTransformerMixin _fit_context)check_random_state gen_batchesgen_even_slices)Interval StrOptions)Paralleldelayed)check_is_fittedcheck_non_negative validate_data)_dirichlet_expectation_1d)_dirichlet_expectation_2d) mean_changec b[R"U5nURupURSn U(a-URSSX45R UR SS9n O [ R"X4UR S9n [ R"[U 55n U(a)[ R"URUR S9OSn U(a$URnURnURnUR [ R:XaSOS n[Un[ Un[ R""UR 5R$n['U5GH<nU(aWWUUUS -nWUUUUS -nO'[ R("UUSS245SnUUU4nU USS24nU USS24R+5nUSS2U4n['SU5H_nUn[ R,"UU5U-nU[ R,"UU- UR.5-nU"UUU5 U"UU5U:dM_ O UU USS24'U(dM[ R,"UU5U-nU SS2U4==[ R0"UUU- 5- ss'GM? X4$) a:E-step: update document-topic distribution. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. exp_topic_word_distr : ndarray of shape (n_topics, n_features) Exponential value of expectation of log topic word distribution. In the literature, this is `exp(E[log(beta)])`. doc_topic_prior : float Prior of document topic distribution `theta`. max_doc_update_iter : int Max number of iterations for updating document topic distribution in the E-step. mean_change_tol : float Stopping tolerance for updating document topic distribution in E-step. cal_sstats : bool Parameter that indicate to calculate sufficient statistics or not. Set `cal_sstats` to `True` when we need to run M-step. random_state : RandomState instance or None Parameter that indicate how to initialize document topic distribution. Set `random_state` to None will initialize document topic distribution to a constant number. Returns ------- (doc_topic_distr, suff_stats) : `doc_topic_distr` is unnormalized topic distribution for each document. In the literature, this is `gamma`. we can calculate `E[log(theta)]` from it. `suff_stats` is expected sufficient statistics for the M-step. When `cal_sstats == False`, this will be None. rY@g{Gz?FcopydtypeNfloatdoubler)spissparseshapegammaastyper nponesexprzerosdataindicesindptrfloat32cy_mean_changecy_dirichlet_expectation_1dfinfoepsrangenonzerordotTouter)Xexp_topic_word_distrdoc_topic_priormax_doc_update_itermean_change_tol cal_sstats random_state is_sparse_x n_samples n_featuresn_topicsdoc_topic_distr exp_doc_topic suff_statsX_data X_indicesX_indptrctyperdirichlet_expectation_1dr3idx_didscnts doc_topic_dexp_doc_topic_dexp_topic_word_d_last_dnorm_phis M/var/www/html/venv/lib/python3.13/site-packages/sklearn/decomposition/_lda.py_update_doc_distributionrV,sb++a.KGGI#))!,H&,,UD9:OPWW GG%X ''9"7qwwGFF4_EFM@J%++177;tII 88 ww"**,G(E 'K:5A ((177   Cy! HUOhuqy.ABC(5/HUQY,?@D**Quax[)!,CUCZ=D%eQh/ 'q1668/37q-.A Fvvo/?@3FH)BFF4(?DTDVDV,WWK $[/? S6;//A/%0q! :vvo/?@3FH q#v "((?D8O"L L C"F  ((cv^\rSrSr%Sr0S\"\SSSS9/_SS\"\SS S S9/_S S\"\SS S S9/_S \"S S15/_S\"\SS S S9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_S\"\SSSS9/_SS\/_SS/_SS/_r \ \ S'SS8jrS?S9jr\S:5rS;r U=r!$)@LatentDirichletAllocationa4Latent Dirichlet Allocation with online variational Bayes algorithm. The implementation is based on [1]_ and [2]_. .. versionadded:: 0.17 Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=10 Number of topics. .. versionchanged:: 0.19 ``n_topics`` was renamed to ``n_components`` doc_topic_prior : float, default=None Prior of document topic distribution `theta`. If the value is None, defaults to `1 / n_components`. In [1]_, this is called `alpha`. topic_word_prior : float, default=None Prior of topic word distribution `beta`. If the value is None, defaults to `1 / n_components`. In [1]_, this is called `eta`. learning_method : {'batch', 'online'}, default='batch' Method used to update `_component`. Only used in :meth:`fit` method. In general, if the data size is large, the online update will be much faster than the batch update. Valid options: - 'batch': Batch variational Bayes method. Use all training data in each EM update. Old `components_` will be overwritten in each iteration. - 'online': Online variational Bayes method. In each EM update, use mini-batch of training data to update the ``components_`` variable incrementally. The learning rate is controlled by the ``learning_decay`` and the ``learning_offset`` parameters. .. versionchanged:: 0.20 The default learning method is now ``"batch"``. learning_decay : float, default=0.7 It is a parameter that control learning rate in the online learning method. The value should be set between (0.5, 1.0] to guarantee asymptotic convergence. When the value is 0.0 and batch_size is ``n_samples``, the update method is same as batch learning. In the literature, this is called kappa. learning_offset : float, default=10.0 A (positive) parameter that downweights early iterations in online learning. It should be greater than 1.0. In the literature, this is called tau_0. max_iter : int, default=10 The maximum number of passes over the training data (aka epochs). It only impacts the behavior in the :meth:`fit` method, and not the :meth:`partial_fit` method. batch_size : int, default=128 Number of documents to use in each EM iteration. Only used in online learning. evaluate_every : int, default=-1 How often to evaluate perplexity. Only used in `fit` method. set it to 0 or negative number to not evaluate perplexity in training at all. Evaluating perplexity can help you check convergence in training process, but it will also increase total training time. Evaluating perplexity in every iteration might increase training time up to two-fold. total_samples : int, default=1e6 Total number of documents. Only used in the :meth:`partial_fit` method. perp_tol : float, default=1e-1 Perplexity tolerance. Only used when ``evaluate_every`` is greater than 0. mean_change_tol : float, default=1e-3 Stopping tolerance for updating document topic distribution in E-step. max_doc_update_iter : int, default=100 Max number of iterations for updating document topic distribution in the E-step. n_jobs : int, default=None The number of jobs to use in the E-step. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. verbose : int, default=0 Verbosity level. random_state : int, RandomState instance or None, default=None Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. Attributes ---------- components_ : ndarray of shape (n_components, n_features) Variational parameters for topic word distribution. Since the complete conditional for topic word distribution is a Dirichlet, ``components_[i, j]`` can be viewed as pseudocount that represents the number of times word `j` was assigned to topic `i`. It can also be viewed as distribution over the words for each topic after normalization: ``model.components_ / model.components_.sum(axis=1)[:, np.newaxis]``. exp_dirichlet_component_ : ndarray of shape (n_components, n_features) Exponential value of expectation of log topic word distribution. In the literature, this is `exp(E[log(beta)])`. n_batch_iter_ : int Number of iterations of the EM step. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int Number of passes over the dataset. bound_ : float Final perplexity score on training set. doc_topic_prior_ : float Prior of document topic distribution `theta`. If the value is None, it is `1 / n_components`. random_state_ : RandomState instance RandomState instance that is generated either from a seed, the random number generator or by `np.random`. topic_word_prior_ : float Prior of topic word distribution `beta`. If the value is None, it is `1 / n_components`. See Also -------- sklearn.discriminant_analysis.LinearDiscriminantAnalysis: A classifier with a linear decision boundary, generated by fitting class conditional densities to the data and using Bayes' rule. References ---------- .. [1] "Online Learning for Latent Dirichlet Allocation", Matthew D. Hoffman, David M. Blei, Francis Bach, 2010 https://github.com/blei-lab/onlineldavb .. [2] "Stochastic Variational Inference", Matthew D. Hoffman, David M. Blei, Chong Wang, John Paisley, 2013 Examples -------- >>> from sklearn.decomposition import LatentDirichletAllocation >>> from sklearn.datasets import make_multilabel_classification >>> # This produces a feature matrix of token counts, similar to what >>> # CountVectorizer would produce on text. >>> X, _ = make_multilabel_classification(random_state=0) >>> lda = LatentDirichletAllocation(n_components=5, ... random_state=0) >>> lda.fit(X) LatentDirichletAllocation(...) >>> # get topics for some given samples: >>> lda.transform(X[-2:]) array([[0.00360392, 0.25499205, 0.0036211 , 0.64236448, 0.09541846], [0.15297572, 0.00362644, 0.44412786, 0.39568399, 0.003586 ]]) n_componentsrNneither)closedr;rbothtopic_word_priorlearning_methodbatchonlinelearning_decaylearning_offset?leftmax_iter batch_sizeevaluate_every total_samplesperp_tolr=r<n_jobsverboser?_parameter_constraints gffffff?g$@g.Ag?gMbP?d)r;r_r`rcrdrgrhrirjrkr=r<rlrmr?cXlX lX0lX@lXPlX`lXplXlXlXl Xl Xl Xl Xl XlUUlgN)r[r;r_r`rcrdrgrhrirjrkr=r<rlrmr?)selfr[r;r_r`rcrdrgrhrirjrkr=r<rlrmr?s rU__init__"LatentDirichletAllocation.__init__ksa(). 0.,. $,* .#6   (rWc[UR5UlSUlSUlUR cSUR - UlOUR UlURcSUR - Ul OURUl SnSU- nURRX4UR U45RUSS9Ul [R"[UR55Ulg)zInitialize latent variables.rrNrerFr)r r? random_state_ n_batch_iter_n_iter_r;r[doc_topic_prior_r_topic_word_prior_r&r' components_r(r*rexp_dirichlet_component_)rurBr init_gammainit_vars rU_init_latent_vars+LatentDirichletAllocation._init_latent_varss00A0AB    '$'$*;*;$;D !$($8$8D !  (%(4+<+<%# UHOn[[5"TUSS24TRTRTRTR TT5v MQ g7frt)rrVrr|r<r=).0 idx_slicer9r>r?rus rU 4LatentDirichletAllocation._e_step..sc  A  , -)Q,--%%(($$  AsAAr)ryrrlrmaxrmrr%zipr(vstackr+r~r r) rur9r> random_initparallelrlresults doc_topics sstats_listrDrFsstatsr?s ``` @rU_e_step!LatentDirichletAllocation._e_steps>.9t))d "$++.  vs1dllQ>N7OPH  -QWWQZ@   #&w- ))J/ $"2"2"8"8@P@P@V@VWJ%$ & $77 7J ,,J,,rWcURUSSUS9upVU(aURU-UlO[R"UR UR -UR*5n[U5URS- nU=RSU- -slU=RUURX---- sl[R"[UR55Ul U=R S- slg)avEM update for 1 iteration. update `component_` by batch VB or online VB. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. total_samples : int Total number of documents. It is only used when batch_update is `False`. batch_update : bool Parameter that controls updating method. `True` for batch learning, `False` for online learning. parallel : joblib.Parallel, default=None Pre-initialized instance of joblib.Parallel Returns ------- doc_topic_distr : ndarray of shape (n_samples, n_components) Unnormalized document topic distribution. Tr>rrrrN) rr}r~r(powerrdrzrcr!r%r*rr) rur9rj batch_updaterrRrFweight doc_ratios rU_em_step"LatentDirichletAllocation._em_steps8 $D8%  #55 BD XX$$t'9'99D[TU]5nSURlSURlSS/UR lU$)NTr/float64)super__sklearn_tags__ input_tags positive_onlysparsetransformer_tagspreserves_dtype)rutags __class__s rUr*LatentDirichletAllocation.__sklearn_tags__%s@w')(,%!%1:I0F- rWcU(a [R[R/OURRn[ UUUSUS9n[ X5 U$)z~check X format check X format and make sure no negative value in X. Parameters ---------- X : array-like or sparse matrix csr)reset accept_sparser )r(rr/r~r rr)rur9reset_n_featureswhomr s rU_check_non_neg_array.LatentDirichletAllocation._check_non_neg_array,sN-=RZZ($BRBRBXBX   "   1#rWT)prefer_skip_nested_validationc J[US5(+nURXSS9nURupEURnU(aUR XQR S9 XPR RS:wa&[SXPR RS4-5e[UR5n[U[SURS- 5S9n[XF5H%n URXS S 24URS US 9 M' S S S 5 U$!,(df  U$=f) aOnline VB with Mini-Batch update. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. y : Ignored Not used, present here for API consistency by convention. Returns ------- self Partially fitted estimator. r~z%LatentDirichletAllocation.partial_fitrrrrzUThe provided data has %d dimensions while the model was trained with feature size %d.rrNFrjrr)hasattrrr%rhrr r~ ValueErrorrrlrrrmrrrj) rur9y first_timerArBrhrlrrs rU partial_fit%LatentDirichletAllocation.partial_fitCs'"!}55  % % 1X & !" __    " ":WW " = ))//2 2>//55a89:  "$++. VSDLL14D-E F((?  lO"&"4"4!&% @G G F s 5D D"c URUSSS9nURup4URnURnURnUR nUR XARS9 Sn [UR5n [U [SURS- 5S9n [U5GHn US :Xa+[X85Hn URXSS24US U S 9 M OURXSU S 9 US:a}U S-U-S:XaqUR!US S U S 9upUR#XS S 9nUR(a[%SU S-UU4-5 U (a['U U- 5UR(:a OAUn O$UR(a[%SU S-U4-5 U=R*S- slGM SSS5 UR!US S W S 9upUR#XS S 9UlU$!,(df  N9=f)a|Learn model for the data X with variational Bayes method. When `learning_method` is 'online', use mini-batch update. Otherwise, use batch update. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. y : Ignored Not used, present here for API consistency by convention. Returns ------- self Fitted estimator. TzLatentDirichletAllocation.fitrrNrrrrbFrr sub_samplingz/iteration: %d of max_iter: %d, perplexity: %.4fziteration: %d of max_iter: %d)rr%rgrir`rhrr rrlrrrmr4rrr_perplexity_precomp_distrprintabsrkr{bound_)rur9rrArBrgrir`rh last_boundrlrirdoc_topics_distrrRbounds rUfitLatentDirichletAllocation.fitss((  % % +J & !" ==,,..__  z9 !$++. VSDLL14D-E F(8_"h.%0%G  lO*3).%- &&HMMPX" "A%1q5N*Ba*G*.,,e+7+'$!::%;E||M 1uh67 "c*u*<&= &M!&J\\9QUHr)r)rur9rDrRs rU_unnormalized_transform1LatentDirichletAllocation._unnormalized_transforms!"\\!5\QrW normalizec[U5 URUSSS9nURU5nU(a&X3RSS9SS2[R 4-nU$)aTransform data X according to the fitted model. .. versionchanged:: 0.18 `doc_topic_distr` is now normalized. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. normalize : bool, default=True Whether to normalize the document topic distribution. Returns ------- doc_topic_distr : ndarray of shape (n_samples, n_components) Document topic distribution for X. Fz#LatentDirichletAllocation.transformrraxisN)rrrsumr(newaxis)rur9rrDs rU transform#LatentDirichletAllocation.transformsg&   % % ,Q & 66q9  222:1bjj=I IOrWc>URX5RXS9$)a* Fit to data, then transform it. Fits transformer to `X` and `y` and returns a transformed version of `X`. Parameters ---------- X : array-like of shape (n_samples, n_features) Input samples. y : array-like of shape (n_samples,) or (n_samples, n_outputs), default=None Target values (None for unsupervised transformations). normalize : bool, default=True Whether to normalize the document topic distribution in `transform`. Returns ------- X_new : ndarray array of shape (n_samples, n_components) Transformed array. r)rr)rur9rrs rU fit_transform'LatentDirichletAllocation.fit_transforms .xx~'''??rWc4Sn[R"U5nURupgURRSnSn [ U5n [ UR5n UR n UR n U(a$URnURnURn[SU5HnU(aWWUUUS-nWUUUUS-nO'[R"UUSS245SnUUU4nU USS2[R4U SS2U4-n[USS9nU [R"UU5- n M X"XXR 5- n U(a[#UR$5U- nU U-n X"XRX5- n U $)aEstimate the variational bound. Estimate the variational bound over "all documents" using only the documents passed in as X. Since log-likelihood of each word cannot be computed directly, we use this bound to estimate it. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Document word matrix. doc_topic_distr : ndarray of shape (n_samples, n_components) Document topic distribution. In the literature, this is called gamma. sub_sampling : bool, default=False Compensate for subsampling of documents. It is used in calculate bound in online learning. Returns ------- score : float c "[R"X- U-5nU[R"[U5[U5- 5- nU[R"[X-5[[R"US55- 5- nU$)Nr)r(rr)priordistrdirichlet_distrsizescores rU_loglikelihood?LatentDirichletAllocation._approx_bound.._loglikelihood&skFFEM_<=E RVVGENWU^;< ? $ HT3VDE $ Xh4?@$ x!T)DE$ 8HdDKL$ (4DCD$ XdAtF;<$ HT1d6BC$ 1d6 JK$ 4"$ I;$ (!$D*#)%#)J35** :@-D4l.5-6-^5P6Pd")-8@D@2M^2*,XL8))rWrY)-rnumbersrrnumpyr( scipy.sparserr#joblibr scipy.specialrrbaser r r r utilsr rrutils._param_validationrrutils.parallelrrutils.validationrrr_online_lda_fastrr1rrr0r2r!r3EPSrVrYrWrUr sy##, ED:.QQ hhuou)p[ )#%5}[ )rW