JP6433858B2 - Probability density function estimation device, continuous value prediction device, method, and program - Google Patents

Description

  The present invention relates to a probability density function estimation device, continuous value prediction device, method, and program, and more particularly to a probability density function estimation device, continuous value prediction device, method, and program for predicting continuous values generated by an individual. About.

  In general, we consider a situation in which an individual who behaves stochastically generates a continuous value and observes that continuous data as data. As this situation, for example, a consumer (= individual) purchases a certain amount (= continuous value), a consumer (= individual) purchases at a certain time interval (= continuous value), a traveler (= The individual moves to the next place of stay at a certain time interval (= continuous value), the traveler (= individual) moves to a position separated by a certain distance (= continuous value), in the document (= individual) A certain word appears at a certain time (= continuous value), a device (= individual) fails at a certain time (= continuous value), and a nerve cell (= individual) has a nerve at a certain time interval (= continuous value) There are various things such as generating spike signals, earthquakes (= individuals) occurring at certain time intervals (= continuous values).

  If a probabilistic rule that an individual follows when generating a continuous value, that is, a probability density function can be estimated from the data, a continuous value that the individual will generate in the future can be predicted. In the above example, the consumer's future purchase time and the traveler's future place of stay can be predicted. Probability density function estimation methods can be broadly divided into two types: parametric density estimation and nonparametric density estimation. Parametric density estimation is a method in which a probability density function is expressed by a distribution function having parameters, and the values of the parameters are determined from data. On the other hand, nonparametric density estimation is a method of estimating from data without making a strong assumption on the shape of the probability density function. A representative example of nonparametric density estimation is a histogram method, and the present invention relates to this histogram method.

J. Rissanen, "Density estimation by stochastic complexity", IEEE Transactions on Information Theory, Vol. 38, pp.315-323, 1992. C. Bishop. “Pattern Recognition and Machine Learning”, Springer, New York, 2006. K. H. Knuth, "Optimal data-based binning for Histograms", arXiv preprint physics / 0605197, 2006. L. Davies, U. Gather, D. Nordman, H. Weinert, "A comparison of automatic histogram constructions", ESAIM: Probability and Statistics, Vol. 13, pp.181-196, 2009.

  When the data observed from an individual is sufficiently available, the histogram method capable of reproducing more various shapes is suitable instead of the parametric density estimation in which the shape of the probability density function is limited (low degree of freedom). However, in many cases, since there is little data of individuals available, the histogram method cannot be applied and parametric density estimation is used. At this time, when the shape of the distribution function assumed in the parametric density estimation is different from the true shape, a correct probability density function cannot be estimated, and the result prediction accuracy is lowered. Table 1 shows the advantages and disadvantages of the histogram method and parametric density estimation.

  Since we cannot know the true probability density function inherent in an individual, we should use the histogram method whenever possible. The problem to be solved by the present invention is the weakness of the histogram method for data with few individuals (sparse data).

  The present invention has been made in order to solve the above problems, and a probability density function estimation apparatus, method, and method capable of estimating a probability density function for accurately predicting continuous values generated by an individual, and The purpose is to provide a program.

  Another object of the present invention is to provide a continuous value prediction apparatus, method, and program capable of accurately predicting continuous values generated by an individual.

  In order to achieve the above object, a probability density function estimation apparatus according to a first aspect of the present invention receives, as an input, a set of continuous value data to which an individual ID of the individual is assigned, which is observed by a plurality of individuals. For each continuous value data included in the data set, for each individual and each topic, the number of continuous value data of the individual belonging to the topic, and the topic for each interval of continuous values in the histogram corresponding to each topic Among the continuous value data belonging to the topic, the continuous value data belongs according to the posterior probability that the continuous value data belongs to each topic based on the number of continuous value data included in the section of the histogram corresponding to the topic. The topic for each individual and each topic obtained from the calculation result of the topic is calculated. Corresponding to each of the topics based on the number of continuous value data of the individual and the number of continuous value data included in the section of the histogram corresponding to the topic for each section of the histogram corresponding to each topic. A parameter representing the distribution of weights for each of the histograms, a topic estimation unit that repeatedly updates the parameters representing the distribution of the histograms, a parameter representing the distribution of weights updated by the topic estimation unit, Parameters representing the distribution, for each individual and each topic obtained for each iteration, the number of continuous value data of the individual belonging to the topic, and each section of the histogram corresponding to each topic obtained for each iteration The histogram corresponding to the topic The weight of each of the histograms corresponding to each of the topics in each individual, and the shape of each section of the histogram corresponding to each of the topics For each of the plurality of individuals, a weight for each of the histograms estimated for the individual, and a shape of each section of the histogram corresponding to each of the estimated topics And a probability density function estimator for estimating a probability density function represented by a linear sum of each of the histograms corresponding to each of the topics.

  Further, in the probability density function estimation device according to the first invention, the topic estimation unit, for each continuous value data included in the set of continuous value data, according to the posterior probability that the continuous value data belongs to each topic, The topic to which the continuous value data belongs is calculated, and for each of the histograms corresponding to each of the topics, for each individual, the continuous value data of the individual belonging to the topic for each individual is calculated. The histogram division number is calculated according to the posterior probability of the histogram division number based on the number and the number of continuous value data included in each section of the histogram corresponding to the topic, and obtained from the topic calculation result For each individual and each topic, continuous value data of the individual belonging to the topic And the number of continuous value data included in the section of the histogram corresponding to the topic and the number of divisions calculated for each of the histogram corresponding to each of the topics, for each section of the histogram corresponding to each topic And repeatedly updating a parameter representing a distribution of weights for each of the histograms corresponding to each of the topics and a parameter representing the distribution of the histograms, and the probability density function estimator includes the topic estimator A parameter representing the distribution of the weights updated by the parameter, a parameter representing the distribution of the histogram, the number of divisions calculated for each of the histograms corresponding to each of the topics, and each individual and each topic obtained for each iteration. , Of the individual belonging to the topic Based on the number of continuous value data and the number of continuous value data included in the section of the histogram corresponding to the topic, for each section of the histogram corresponding to each topic, obtained for each iteration. The shape of each section of the histogram corresponding to each and the weight for each of the histograms corresponding to each of the topics in each individual may be estimated.

  The continuous value prediction apparatus according to the second invention is a continuous value prediction apparatus for predicting continuous value data of an individual to be predicted, and for each topic estimated in advance for the individual to be predicted. A probability density function represented by a linear sum of each of the histograms corresponding to each of the topics, based on a weight for each of the histograms and a shape of each interval of successive values in each of the histograms corresponding to each of the topics And a continuous value prediction unit for calculating an expected value of a continuous value generated in the future for the individual to be predicted.

  In the probability density function estimation method according to a third aspect of the invention, the topic estimation unit receives as input a set of continuous value data to which the individual ID of the individual is assigned, observed by a plurality of individuals, and sets the continuous value data For each individual and each topic, the number of the continuous value data of the individual belonging to the topic, and each interval of continuous values in the histogram corresponding to each topic Based on the number of continuous value data included in the section of the histogram corresponding to the topic among the continuous value data, the topic to which the continuous value data belongs is determined according to the posterior probability that the continuous value data belongs to each topic. Calculated and obtained from the calculation result of the topic, before belonging to the topic for each individual and each topic A histogram corresponding to each of the topics based on the number of continuous value data of the individual and the number of continuous value data included in the section of the histogram corresponding to the topic for each section of the histogram corresponding to each topic. A step of repeatedly updating a parameter representing a distribution of weights for each of the parameters and a parameter representing the distribution of the histogram, and a parameter representing the distribution of weights updated by the probability density function estimator by the topic estimator, Parameters representing the distribution of the histogram, the number of continuous value data of the individual belonging to the topic for each individual and each topic obtained for each iteration, and the histogram corresponding to each topic obtained for each iteration Corresponding to the topic for each section of Based on the number of continuous value data included in the section of the stogram, the shape of each section of the histogram corresponding to each of the topics, and the weight for each of the histograms corresponding to each of the topics in each individual For each of the plurality of individuals, a weight for each of the histograms estimated for the individual, and a shape of each section of the histogram corresponding to each of the estimated topics And a step of estimating a probability density function represented by a linear sum of each of the histograms corresponding to each of the topics.

  In the probability density function estimation method according to a third aspect of the invention, the step of estimating and updating by the topic estimation unit includes the continuous value data in each topic for each continuous value data included in the set of continuous value data. According to the posterior probability of belonging, calculate the topic to which the continuous value data belongs, and for each of the histograms corresponding to each of the topics, the individual belonging to the topic for each individual obtained from the calculation result of the topic Calculating the number of divisions of the histogram according to the posterior probability of the number of divisions of the histogram based on the number of continuous value data of the individual and the number of continuous value data included in each section of the histogram corresponding to the topic; Belonging to the topic for each individual and each topic obtained from the calculation result of The number of continuous value data of the individual, the number of continuous value data included in the section of the histogram corresponding to the topic, and the histogram corresponding to each of the topics, for each section of the histogram corresponding to each topic And updating the parameter representing the distribution of weights for each of the histograms corresponding to each of the topics and the parameter representing the distribution of the histograms based on the number of divisions calculated for each of the topics, and the probability density function The estimation unit estimates the parameters representing the weight distribution updated by the topic estimation unit, the parameters representing the histogram distribution, the number of divisions calculated for each histogram corresponding to each of the topics, and repetition. Each individual and each topic obtained for each , The number of continuous value data of the individual belonging to the topic, and the continuous value included in the section of the histogram corresponding to the topic for each section of the histogram corresponding to each topic obtained for each iteration Based on the number of data, the shape of each section of the histogram corresponding to each of the topics and the weight for each of the histograms corresponding to each of the topics in each individual may be estimated.

  A continuous value prediction method according to a fourth invention is a continuous value prediction method in a continuous value prediction apparatus for predicting continuous value data of an individual to be predicted, wherein the continuous value prediction unit Each of the histograms corresponding to each of the topics based on a pre-estimated weight for each of the histograms for each of the topics and the shape of each interval of successive values in each of the histograms corresponding to each of the topics. Based on a probability density function represented by a linear sum of the above, it is executed including a step of calculating an expected value of continuous values generated in the future for the individual to be predicted.

  A program according to a fifth invention is a program for causing a computer to function as each part constituting the probability density function estimation device according to the first invention.

  A program according to a sixth invention is a program for causing a computer to function as each unit constituting the continuous value predicting device according to the second invention.

  According to the probability density function estimation apparatus, method, and program of the present invention, according to the posterior probability, the topic to which the continuous value data belongs is calculated, and each individual and each topic obtained from the topic calculation result belong to the topic. Parameters representing the distribution of weights based on the number of continuous value data of the individual and the number of continuous value data included in the histogram section corresponding to the topic for each section of the histogram corresponding to each topic, and the histogram Repeated updating parameters representing distributions, updated weight distribution parameters, histogram distribution parameters, and individual values of each individual and each topic obtained for each iteration. The number of data and hiss corresponding to each topic obtained at each iteration Corresponding to the shape of each section of the histogram corresponding to each of the topics and each topic in each individual based on the number of continuous value data included in the section of the histogram corresponding to the topic for each section of the gram To estimate the continuous value generated by an individual accurately by estimating the weight for each histogram and estimating the probability density function based on the weight for each histogram and the shape of each section of the histogram. The probability density function can be estimated.

  Further, according to the continuous value predicting apparatus, method, and program of the present invention, the weight for each histogram for each topic estimated in advance for the individual to be predicted, and each interval of the continuous value in each histogram The effect that it is possible to accurately predict the continuous value generated by an individual by calculating the expected value of the continuous value generated in the future for the prediction target individual based on the probability density function based on the shape of Is obtained.

It is a block diagram which shows the structure of the probability density function estimation apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the continuous value estimation apparatus which concerns on embodiment of this invention. It is a flowchart which shows the probability density function estimation process routine in the probability density function estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the continuous value prediction process routine in the probability density function estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the probability density function estimation process routine in the probability density function estimation apparatus which concerns on the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>

  First, an outline of the embodiment of the present invention will be described.

  In many cases, there is little data obtained from each individual, but on the other hand (small number) of data can be obtained from a plurality of individuals. For example, in an e-commerce site, most customers purchase only a few times, but the number of registered customers is extremely large. Each individual has its own probability density function, but there should be individuals having similar probability density functions. At this time, (i) the probability density function of all individuals is expressed by a linear sum of a small number of histograms, and (ii) the difference in probability density function for each individual is expressed by a difference in weighting of the linear sum. Assuming that, the high degree of freedom that is an advantage of the histogram method can be secured, and the sparsity of data, which is its weakness, can be avoided. There is already a device that expresses the probability density function of an individual as a linear sum of histograms (see Non-Patent Document 1 above), but it is for a single individual, and the sparsity of data, which is a weak point of the histogram method, is It cannot be avoided.

<Configuration of probability density function estimation device according to first embodiment of the present invention>

  Next, the configuration of the probability density function estimation apparatus according to the first embodiment of the present invention will be described. As shown in FIG. 1, the probability density function estimation apparatus 100 according to the first embodiment of the present invention includes a CPU, a RAM, a program for executing a probability density function estimation processing routine described later, and various data. It can be composed of a computer including a stored ROM. Functionally, the probability density function estimation apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 50 as shown in FIG.

The input unit 10 receives a set of continuous value data to which individual individual IDs are assigned, which are observed by a plurality of individuals to be analyzed. In addition, the number K of histograms for expressing the probability density function of an individual, and the definition area of the probability density function that is the observation period of continuous value data (range of values that the continuous value can take) T≡ [T ₀ , T ₁ ] Is accepted. In the following description, it is assumed that the K histograms are represented by k = 1, 2, 3,..., K, and the k-th histogram corresponds to the topic k. Continuous value data (denoted u) individual ID, the number of consecutive values data observed in individuals u (N _u hereinafter), the total number of successive values data observed in all individuals (N hereinafter), all individuals A set of continuous value data observed in (represented as {t _j } ≡ (t ₁ , t ₂ ,..., T _N )), and a set of individual IDs that generated each continuous value data ({u _j } ≡ (Denoted as u ₁ , u ₂ ,..., U _N )).

  Here, the principle of the generation model used in this embodiment will be described.

Probability density estimation using a histogram requires the number of domain divisions. In the present embodiment, since the division number is not estimated, the division number W≡ (W ₁ , W ₂ ,..., W _K ) of each topic is given as an input.

First, it is assumed that continuous value data is generated from each individual based on the generation model shown in the following equation (1). individuals were generated j-th consecutive value data t _j u _j is the probability density function p expressed by the linear sum of K histogram | generating a continuous value data t _j from (t φ).

Where z _j is a latent variable representing the topic (1 to K) to which the j-th continuous value data belongs,

Is the weight of individual u _j to topic k, and p (t _j | z _j = k, φ _{· k} ) is the histogram

The distribution of the topic k expressed by: φ _lk is a parameter for determining the shape in the section l of the histogram, and w _j ^k is

(1 <w _j ^k <W _k ).

Here, the function int [x] in the above equation (3) is a function for extracting the integer part of the real value x. The probability density function based on the histogram divides the domain [T ₀ , T ₁ ] of the measurement period of continuous value data equally into W _k sections, and calculates the probability density value in each section l ( ₁ to W _k ). _This is expressed by a constant value φ _lk , and w _j ^k indicates in which section the continuous value data t _j is included.

  Next, two kinds of parameters appearing in the generation models of the above equations (1) to (3)

_And Dirichlet distribution, which is a conjugate prior distribution, for φ _lk .

From the above formulas (1) to (4), the topic z≡ (z ₁ , z ₂ ,..., Z _N ) and observation t≡ (t ₁ , t ₂ ,. _N ) joint probability

Get. Here, N _ku is the number of continuous value data whose belonging topic is k among the continuous value data of the individual u, and N _kl is included in the l-th section of the histogram among all continuous value data whose belonging topic is k. N _k represents the total number of continuous value data of which the topic is k, and in order to simplify the model, all elements of the parameter β of the Dirichlet distribution are set to be equal (β ₁ = β ₂ =... = Β).

  The generation model in the present embodiment can be calculated by the above equations (1) to (5). The above is the principle of the generation model in the present embodiment.

  The calculation unit 20 includes a topic estimation unit 30 and a probability density function estimation unit 32.

The topic estimation unit 30 receives a set of continuous value data to which the individual ID of the individual u received by the input unit 10 is input, and for each continuous value data included in the continuous value data set, each individual u and each topic Corresponding to topic k among continuous value data belonging to topic k with respect to the number N _ku of continuous value data of individual u belonging to topic k and each interval l of continuous values in the histogram corresponding to each topic k The topic k to which the continuous value data belongs is calculated according to the posterior probability that the continuous value data belongs to each topic, based on the number N _kl of the continuous value data included in the interval l of the histogram. Further, the topic estimation unit 30 obtains the number N _ku of continuous value data of the individual u belonging to the topic k and the histogram corresponding to each topic k for each individual u body and each topic k obtained from the calculation result of the topic k. Parameter α representing the distribution of weights for each of the histograms corresponding to each of the topics k based on the number N _kl of the continuous value data included in the interval l of the histogram corresponding to the topic k, and the histogram The parameter β representing the distribution of is updated. The topic estimation unit 30 repeatedly performs the above topic calculation and parameter update.

A specific calculation method performed by the topic estimation unit 30 will be described in detail below. The topic estimation unit 30 and the probability density function estimation unit 32 described later calculate the posterior probability p (z | t, α, β, W) of the belonging topic z≡ (z ₁ , z ₂ ,... Z _N ). Thus, all the unknown parameters α, β, θ, and φ can be estimated.

First, since it is difficult to analytically handle the posterior probability, the topic estimation unit 30 performs the Gibbs sampling on the topic z _j to which each continuous value data t _j belongs, in which the joint probability is obtained from the above equation (5). Formula for

, The parameter α, the parameter β, the number N _ku of the continuous value data of the individual u belonging to the topic k, the number N _{kl of} the continuous value data included in the section l of the histogram corresponding to the topic k, Based on the total number N _k of continuous value data _k , P samples (z ⁽¹⁾ , z ⁽²⁾ ,..., z ^{) of} z≡ (z ₁ , z ₂ ,... z _N ) ^(P) ) is generated and held. However, −j appearing in the equation (6) represents a subset obtained by excluding the j-th continuous value data from the total number N of continuous value data sets. Actually, it is not a sample of z itself, but its sufficient statistics (N _ku ⁽¹⁾ , N _ku ⁽²⁾ ,..., N _ku ^(P) ) and (N _kl ⁽¹⁾ , N _kl ^{( 2)} ,..., N _kl ^(P) ).

  At the same time as generating the samples for P times, the topic estimation unit 30 updates the values of the unknown parameters α and β P times based on the stochastic EM method shown in the following equation (7) (see Non-Patent Document 2 above). To do.

Specifically, the topic estimation unit 30 sets the parameter α, the parameter β, the number of continuous value data N _{ku of} the individual u belonging to the topic k, and the number of continuous value data included in the interval l of the histogram corresponding to the topic k. N _kl, based on the total number N _k of consecutive value data is the total number N _u of a continuous value data of individuals u, and affiliation topic k, parameter according to the following equation (8) update equation alpha, and the value of beta P Update once.

Then, the topic estimation unit 30 sets the parameter values α ^(P) and β ^(P) after updating P times as the estimated values of the parameters α and β, respectively. Note that the values of N _ku ^(p) , N _kl ^(p) , and N _k are also updated each time it is updated.

The probability density function estimator 32 includes a parameter α representing the weight distribution updated by the topic estimator 30, a parameter β representing the histogram distribution, and the topic k for each individual u and each topic k obtained for each iteration. The continuous value data included in the interval l of the histogram corresponding to the topic k with respect to the number N _ku of the continuous value data of the individual u belonging to, and each interval l of the histogram corresponding to each topic k obtained for each iteration based on the number N _kl, topic k each corresponding histogram of each shape ~ phi _lk of each section _l, and estimates the weight ~ theta _ku for each of the histogram corresponding to each topic k in each individual u To do. Then, the probability density function estimation unit 32 calculates, for each of the plurality of individuals u, a weight ~ θ _ku for each of the histograms estimated for the individual u and a histogram corresponding to each of the estimated topics k. _{Estimate the} probability density function ~ p _u (t│ ~ φ, ~ θ _{・ u} ) represented by the linear sum of each histogram corresponding to each topic k based on the shape ~ φ _lk of each section l To do.

Specific calculations performed by the probability density function estimation unit 32 will be described in detail below. In the probability density function estimation unit 32, the parameter α, parameter β, (N _ku ⁽¹⁾ , N _ku ⁽²⁾ ,..., N _ku ^(P) ) obtained by the estimation by the topic estimation unit 30 and ( N _kl ⁽¹⁾ , N _kl ⁽²⁾ ,..., N _kl ^(P) ) are used to estimate the probability density function of each individual u.

First, based on the parameter β, the number N _kl of continuous value data included in the interval l of the histogram corresponding to the topic k, and the total number N _k of continuous value data with the affiliated topic k, the topic k (1 ≦ k ≦ K ) _Is the average of the posterior probabilities

Estimated by Next, based on the parameter α, the number N _kl of continuous value data included in the interval l of the histogram corresponding to the topic k, and the total number N _u of continuous value data of the individual u, the weight θ of the topic k in each individual u _ku is the average of posterior probabilities

Estimated by Finally, using the results obtained in the above equations (9) and (10), the estimated value ~ p _u (t | ~ φ, ~ θ _{· u} ) of the probability density function of each individual u is Calculate according to equation (11).

However, the histogram p (t | z _j = k, ~ φ _k , W _k ) is defined by the above equation (2).

In the probability density function estimator 32, the probability density function of the individual u is expressed by ~ θ _ku , ~ φ _lk , W _k , and T. Then, the probability density function estimation unit 32 _obtains ~ θ _ku (1 ≦ k ≦ K) for each individual u, the number of divisions W _k and ~ φ _lk (1 ≦ l ≦ W _k ) for each topic k, The common domain T≡ [T ₀ , T ₁ ] is output to the output unit 50. Table 2 shows an output example when the number of topics K = 3.

<Configuration of continuous value prediction apparatus according to first embodiment of the present invention>

  Next, the configuration of the continuous value prediction apparatus according to the first embodiment of the present invention will be described. As shown in FIG. 2, the continuous value prediction apparatus 200 according to the first embodiment of the present invention stores a CPU, a RAM, a program for executing a continuous value prediction processing routine described later, and various data. It can be composed of a computer including a ROM. Functionally, the continuous value prediction apparatus 200 includes an input unit 210, a calculation unit 220, and an output unit 250 as shown in FIG.

The input unit 210 uses the weight ~ θ _ku for each of the histograms for each topic k estimated in advance by the probability density function estimation apparatus 100 for the individual u to be predicted, and each of the histograms corresponding to each of the topics k. A probability density function ~ p _u (t│ ~ φ, ~ θ _{・ u} ) represented by a linear sum of histograms corresponding to each of the topics k based on the shape ~ φ _lk of each interval l of continuous values in Accept.

  The calculation unit 220 includes a continuous value prediction unit 230.

The continuous value prediction unit 230 is generated in the future for the individual u to be predicted according to the following equation (12) based on the probability density function ~ p _u (t | ~ φ, ~ θ _{· u} ) received by the input unit 210. that the expected value E of the continuous values [t ^u] calculates and outputs the expected value E of the resulting solid u of calculating the [t ^u] to the output unit 250.

However, ΔT≡T ₁ −T ₀ .

<Operation of the probability density function estimation apparatus according to the first embodiment of the present invention>

  Next, the operation of the probability density function estimation apparatus 100 according to the first embodiment of the present invention will be described. When the input unit 10 receives a set of continuous value data to which an individual ID is assigned, the probability density function estimation apparatus 100 executes a probability density function estimation processing routine shown in FIG.

  First, in step S100, a set of continuous value data received by the input unit 10 is acquired.

Next, in step S102, for each continuous value data included in the set of continuous value data acquired in step S100, the number _Nku of the continuous value data of the individual u belonging to the topic k for each individual u and each topic. The initial value and the initial value of the number N _kl of continuous value data included in each interval l of continuous values in the histogram corresponding to each topic k, or for each individual u and each topic obtained in step 104 described later, Based on the number N _ku of continuous value data of the individual u belonging to the topic k and the number N _kl of continuous value data included in each interval l of continuous values in the histogram corresponding to each topic k, the above equation (6) The topic k to which the continuous value data belongs is calculated according to the posterior probability that the continuous value data to belong to each topic.

In step S104, for each individual u body and each topic k obtained from the calculation result of topic k in step S102, the number N _ku of continuous value data of the individual u belonging to topic k, and the histogram corresponding to each topic k. Based on the number N _kl of continuous value data included in the section l of the histogram corresponding to the topic k for each section l, the distribution of weights for each of the histograms corresponding to each of the topics k is calculated according to the above equation (8). The parameter α representing and the parameter β representing the histogram distribution are updated.

  In step S105, it is determined whether the processes in steps S102 to S104 have been repeated a predetermined number of times. If the predetermined number of times has not been repeated, the process returns to step S102 and the processes in steps S102 to S104 are repeated. If it has been repeated, the process proceeds to step S106.

In step S106, the parameter α representing the weight distribution for each of the histograms corresponding to each of the topics k updated in step S104, the parameter β representing the histogram distribution, and the topic k obtained for each iteration belong to the topic k. Based on the number N _ku of continuous value data of the individual u and the number N _kl of continuous value data included in the interval l of the histogram obtained for each iteration, the topic is calculated according to the above formulas (9) and (10). The shape ~ φ _lk of each section l of each histogram corresponding to each of k and the weight ~ θ _ku for each histogram corresponding to each of the topics k in each individual u are estimated.

In step S108, for each of the plurality of individuals u in step S106, the weight ~ θ _ku for each of the histograms estimated for the individual u and each of the histograms corresponding to each of the estimated topics k. Probability density function ~ p _u (t | ~ φ, ~ θ _·) represented by the linear sum of each histogram corresponding to each topic k according to the above equation (11) based on the shape of section l ~ φ _{lk u} ) is estimated and output to the output unit 50 to end the probability density function processing routine.

As described above, according to the probability density function estimation apparatus according to the first embodiment, the topic k to which the continuous value data belongs is calculated according to the posterior probability shown in the above equation (6), and the calculation of the topic k is performed. The number of continuous value data of individuals belonging to topic k for each individual and each topic obtained from the result, and each histogram interval corresponding to each topic k is included in the histogram interval corresponding to the topic. Based on the number of continuous value data, the parameter α representing the distribution of weights and the parameter β representing the distribution of histograms are repeatedly updated according to the above equation (8), and the parameter α representing the updated weight distribution is obtained. Parameter β representing the distribution of the histogram, the individual belonging to topic k for each individual and each topic k obtained for each iteration Continuous value data number N _ku, and was obtained at each iteration, for each section l of the histogram corresponding to each topic k, the number N _kl continuous value data included in the interval l of the histogram corresponding to the topic k Based on the above formulas (9) and (10), the shape of each section l of the histogram corresponding to each of the topics _{k˜φ lk} , and the histogram corresponding to each of the topics k in each individual The weight ~ θ _ku is estimated, and based on the weight ~ θ _ku for each of the histograms and the shape ~ φ _lk of each section l of each histogram, the probability density function ~ p _u (t It is possible to estimate a probability density function for accurately predicting continuous values generated by an individual by estimating │ ~ φ, ~ θ _{· u} ).

<Operation of Continuous Value Prediction Device According to Embodiment of the Present Invention>

  Next, the operation of continuous value prediction apparatus 200 according to the embodiment of the present invention will be described. When the input unit 210 receives the probability density function estimated by the probability density function estimation device 100 for the individual u to be predicted, the continuous value prediction device 200 executes a continuous value prediction processing routine shown in FIG.

In step S200, based on the probability density function received in the input unit 210, an expected value E [t ^u ] of a continuous value generated in the future for the individual u to be predicted is calculated according to the above equation (12).

In step S202, the expected value E [t ^u ] for the individual u obtained as a result of the calculation is output to the output unit 250, and the continuous value prediction processing routine ends.

  As described above, according to the prediction device according to the first embodiment, the weight for each of the histograms for each topic estimated in advance for the individual to be predicted, and the continuous value in each of the histograms Based on the probability density function based on the shape of each section l, the continuous value generated by the individual is calculated by calculating the expected value of the continuous value generated in the future for the prediction target individual according to the above equation (12). Predict with high accuracy.

<Configuration of probability density function estimation apparatus according to second embodiment of the present invention>

  Next, the configuration of the probability density function estimation apparatus according to the second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected about the location similar to 1st Embodiment, and description is abbreviate | omitted.

  As shown in FIG. 1, the probability density function estimation apparatus 100 according to the second embodiment of the present invention includes a CPU, a RAM, and a program and various data for executing a probability density function estimation processing routine described later. And a ROM including a ROM that stores information. Functionally, the probability density function estimation apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 50 as shown in FIG.

The input unit 10 according to the second embodiment receives a set of continuous value data to which individual individual IDs are assigned, which are observed by a plurality of individuals to be analyzed. Also, a domain of probability density function (a range of values that can be taken by continuous values) T≡ [T ₀ , T ₁ ] is accepted.

  The calculation unit 20 according to the second embodiment includes a topic estimation unit 30 and a probability density function estimation unit 32.

The topic estimation unit 30 according to the second embodiment uses the continuous value for each continuous value data included in the set of continuous value data received by the input unit 10 according to the posterior probability that the continuous value data belongs to each topic k. Calculate the topic k to which the data belongs. Also, the topic estimation unit 30 obtains the number N _ku of continuous value data of the individual u belonging to the topic k for each individual u obtained from the calculation result of the topic k for each histogram corresponding to each of the topics k. The histogram division number W is calculated according to the posterior probability of the histogram division number W based on the number N _kl of continuous value data included in each section l of the histogram corresponding to the topic k. Further, the topic estimation unit 30 obtains the number N _ku of continuous value data of the individual u belonging to the topic k and the histogram corresponding to each topic obtained from the calculation result of the topic k, for each individual u and each topic k. Based on the number N _kl of continuous value data included in the interval l of the histogram corresponding to the topic for each interval l and the division number W calculated for each of the histograms corresponding to each of the topics k, The parameter α representing the weight distribution for each of the corresponding histograms and the parameter β representing the histogram distribution are updated. The topic estimation unit 30 repeatedly performs the above topic calculation, histogram division number W calculation, and parameter update.

  The topic estimation unit 30 may calculate the topic k in the same manner as in the first embodiment.

  Next, the estimation method of the histogram division number W and the update method of the parameter α and the parameter β in the topic estimation unit 30 will be described.

The topic estimation unit 30 estimates the histogram division number W≡ (W ₁ , W ₂ ,..., W _K ) of each topic, which is known in the first embodiment, from continuous value data. Uniform distribution as no information prior distribution about the number of divisions

As well as Gibbs sampling of the affiliated topic z in the above equation (6), the parameter β, the number N _kl of continuous value data included in each section l of the histogram corresponding to the topic k, and the affiliated topic is k Based on the total number _Nk of certain continuous value data, Gibbs sampling is also performed for the division number W according to the following equation (14).

Since the convex shape of the distribution of the division number W is empirically sharp, the division number W ^(P) ≡ (W ₁ ^(P) , W ₂ ^(P) ,..., W _K ^{(P )} ) Is an estimated value.

  In the topic estimation unit 30, the unknown parameters α and β are updated P times according to the following equation (15) using the probabilistic EM method as in the first embodiment.

  In the prior art, only a method of determining the number of divisions from data has been proposed only for the histogram method for a single individual (see Non-Patent Documents 1, 3, and 4 above). However, as in the second embodiment of the present invention, (ii) the probability density function for each individual is represented by (i) the probability density function for all individuals expressed as a linear sum of K (≠ 1) histograms. The method of determining the number of divisions W of each histogram from the data on the assumption of the two points that the difference is expressed by the difference in the weighting of the linear sum has not yet been proposed. Such a probability density function estimation device is unique.

The probability density function estimation unit 32 according to the second embodiment includes a parameter α representing a weight distribution updated by the topic estimation unit 30, a parameter β representing a histogram distribution, and a histogram corresponding to each of the topics k. For each individual u and each topic k obtained for each iteration, the number N _ku of continuous value data of the individual u belonging to topic k, and each topic obtained for each iteration For each section l of the histogram corresponding to k, the shape of each section l of each histogram corresponding to each topic k based on the number of consecutive value data N _kl included in the section l of the histogram corresponding to topic k ~ φ _lk and the weights ~ θ _ku for each of the histograms corresponding to each of the topics k in each individual u are estimated. Then, the probability density function estimation unit 32 calculates, for each of the plurality of individuals u, a weight ~ θ _ku for each of the histograms estimated for the individual u and a histogram corresponding to each of the estimated topics k. _{Estimate the} probability density function ~ p _u (t│ ~ φ, ~ θ _{・ u} ) represented by the linear sum of each histogram corresponding to each topic k based on the shape ~ φ _lk of each section l To do. The specific method for calculating the probability density function in the probability density function estimation unit 32 according to the second embodiment is the same as that in the first embodiment.

  In addition, since the other structure of the probability density function estimation apparatus 100 which concerns on 2nd Embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted.

<Configuration of continuous value prediction apparatus according to second embodiment of the present invention>

  Since the configuration of the continuous value predicting apparatus according to the second embodiment of the present invention is the same as the configuration of the continuous value predicting apparatus according to the first embodiment of the present invention, detailed description thereof is omitted.

<Operation of the probability density function estimation apparatus according to the second embodiment of the present invention>

  Next, the operation of the probability density function estimation apparatus 100 according to the second embodiment of the present invention will be described. In addition, about the location which becomes the effect | action similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  When the input unit 10 receives a set of continuous value data to which an individual ID is assigned, the probability density function estimation apparatus 100 executes a probability density function estimation processing routine shown in FIG.

In step S300, for each topic k, the number N _ku of continuous value data of the individual u belonging to the topic k and the histogram corresponding to the topic k obtained from the calculation result of the topic k in step S102. The histogram division number W _k corresponding to the topic _k is calculated according to the a posteriori probability of the histogram division number W _k shown in the above equation (14) based on the number N _kl of continuous value data included in each interval l. .

  In step S302, it is determined whether the processes in steps S102, S300, and S104 have been repeated a predetermined number of times, and if not repeated a predetermined number of times, the process returns to step S102 and the processes in steps S102, S300, and S104 are repeated. If it has been repeated a predetermined number of times, the process proceeds to step S204.

In step S304, a parameter α representing the distribution of weights for each of the histograms corresponding to each of the topics k updated in step S104, a parameter β representing the distribution of the histograms, and each of the histograms corresponding to each of the topics k in step S300. The number of divisions W _k calculated for the number of pieces of continuous value data N _{ku of} the individual u belonging to the topic k obtained for each iteration, and the continuous value data included in the interval l of the histogram obtained for each iteration. based on the number N _kl, according to the above (9) and (10), the histogram of each shape ~ phi _lk of each section l corresponding to each topic _k, and each topic k in each individual u A weight _{˜θ ku} for each corresponding histogram is estimated.

  In addition, since the other effect | action of the probability density function estimation apparatus 100 which concerns on 2nd Embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted.

As described above, according to the probability density function estimation apparatus according to the second embodiment, the topic k to which the continuous value data belongs is calculated according to the posterior probability shown in the above equation (6), and belongs to the topic k. Posterior of the division number W of the histogram shown in the above equation (14), based on the number N _ku of continuous value data of the individual u and the number N _kl of continuous value data included in each section l of the histogram corresponding to the topic k The number of divisions W of the histogram is calculated according to the probability, and the number N _ku of continuous value data of individuals belonging to the topic k for each individual and each topic k, obtained from the calculation result of the topic, and for each topic k for each section of the corresponding histogram, based on the number N _kl continuous value data included in the section of the histogram corresponding to the topic, follow the above (15) Repeatedly updating the parameter α representing the weight distribution and the parameter β representing the histogram distribution, the parameter α representing the updated weight distribution, the parameter β representing the histogram distribution, the division number W, For each individual and each topic k, the number N _ku of continuous value data of individuals belonging to the topic k, and the topic k for each section of the histogram corresponding to each topic k obtained for each iteration. The shape of each section of the histogram corresponding to each of the topics k to φ _lk according to the above formulas (9) and (10) based on the number N _kl of continuous value data included in the section of the histogram corresponding to , And a weight _{˜θ ku} for each of the histograms corresponding to each of the topics k in each individual, The probability density function ~ p _u (t | ~ φ, ~ θ _{· u} ) is estimated according to the above equation (11) based on the weight ~ θ _ku and the shape of each section of the histogram ~ φ _lk By doing so, it is possible to estimate a probability density function for accurately predicting a continuous value generated by an individual.
<Operation of Continuous Value Prediction Device According to Second Embodiment of the Present Invention>

  Since the operation of the continuous value predicting apparatus according to the second embodiment of the present invention is the same as that of the continuous value predicting apparatus according to the first embodiment of the present invention, detailed description thereof is omitted.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

10, 210 Input unit 20, 220 Calculation unit 30 Topic estimation unit 32 Probability density function estimation unit 50, 250 Output unit 100 Probability density function estimation device 200 Continuous value prediction device 230 Continuous value prediction unit

Claims (8)

A set of continuous value data observed by a plurality of individuals and assigned with an individual ID of the individual is used as an input, and for each continuous value data included in the set of continuous value data, the topic for each individual and each topic Included in the interval of the histogram corresponding to the topic among the continuous value data belonging to the topic for the interval of the continuous value in the histogram corresponding to the number of the continuous value data of the individual belonging to Based on the number of continuous value data, the topic to which the continuous value data belongs is calculated according to the posterior probability that the continuous value data belongs to each topic,
The number of continuous value data of the individual belonging to the topic for each individual and each topic obtained from the calculation result of the topic, and the histogram corresponding to the topic for each section of the histogram corresponding to each topic A topic estimation unit that repeats updating a parameter representing a distribution of weights for each of the histograms corresponding to each of the topics and a parameter representing the distribution of the histograms based on the number of continuous value data included in the section; ,
Parameters representing the distribution of the weights updated by the topic estimation unit, parameters representing the distribution of the histogram, continuous values data of the individuals belonging to the topic for each individual and each topic obtained for each iteration Corresponding to each of the topics based on the number and the number of continuous value data included in the section of the histogram corresponding to the topic for each section of the histogram corresponding to each topic obtained for each iteration. Estimating the shape of each section of each of the histograms and the weight for each of the histograms corresponding to each of the topics in each individual, and for each of the plurality of individuals, the histogram estimated for the individual Corresponding to each of the estimated topics Serial based on the shape of each section of each of the histograms, probability density function estimating unit for estimating a probability density function expressed by a linear sum of each of the histograms corresponding to each of the topics,
Probability density function estimation device. The topic estimation unit calculates, for each continuous value data included in the set of continuous value data, a topic to which the continuous value data belongs according to a posterior probability that the continuous value data belongs to each topic,
For each of the histograms corresponding to each of the topics, the number of continuous value data of the individual belonging to the topic for each individual, obtained from the calculation result of the topic, and each section of the histogram corresponding to the topic Calculating the number of divisions of the histogram according to the posterior probability of the number of divisions of the histogram based on the number of continuous value data included in the
The number of continuous value data of the individual belonging to the topic for each individual and each topic obtained from the calculation result of the topic, and the histogram corresponding to the topic for each section of the histogram corresponding to each topic A parameter representing a weight distribution for each of the histograms corresponding to each of the topics based on the number of continuous value data included in the section and the number of divisions calculated for each of the histograms corresponding to each of the topics. And updating the parameter representing the distribution of the histogram,
The probability density function estimator includes a parameter representing the weight distribution updated by the topic estimator, a parameter representing the histogram distribution, the number of divisions calculated for each histogram corresponding to each of the topics, and repetition. For each individual and each topic obtained for each topic, the number of continuous value data of the individual belonging to the topic, and corresponding to the topic for each section of the histogram corresponding to each topic obtained for each iteration The shape of each section of the histogram corresponding to each of the topics, and each of the histograms corresponding to each of the topics in each individual, based on the number of continuous value data included in the section of the histogram The probability density function estimation apparatus according to claim 1, wherein the weight for is estimated. A continuous value prediction device for predicting continuous value data of an individual to be predicted,
The weight corresponding to each of the histograms for each of the topics, which is estimated in advance by the probability density function estimating device according to claim 1 or 2, and the corresponding to each of the topics for the individual to be predicted Continuously generated in the future for the individual to be predicted based on a probability density function represented by a linear sum of each of the histograms corresponding to each of the topics, based on the shape of each interval of continuous values in each of the histograms A continuous value prediction apparatus including a continuous value prediction unit for calculating an expected value of a value. The topic estimation unit receives a set of continuous value data that is observed by a plurality of individuals and is assigned with the individual ID of the individual, and for each continuous value data included in the set of continuous value data, each individual and each The histogram corresponding to the topic among the continuous value data belonging to the topic with respect to each section of the continuous value in the histogram corresponding to the number of continuous value data of the individual belonging to the topic and the topic for the topic Based on the number of continuous value data included in the interval, the topic to which the continuous value data belongs is calculated according to the posterior probability that the continuous value data belongs to each topic,
The number of continuous value data of the individual belonging to the topic for each individual and each topic obtained from the calculation result of the topic, and the histogram corresponding to the topic for each section of the histogram corresponding to each topic Repetitively updating a parameter representing a distribution of weights for each of the histograms corresponding to each of the topics and a parameter representing the distribution of the histograms based on the number of continuous value data included in the section;
A probability density function estimation unit belongs to the topic for each individual and each topic obtained for each iteration, a parameter representing the distribution of the weight updated by the topic estimation unit, a parameter representing the distribution of the histogram Based on the number of continuous value data of the individual and each section of the histogram corresponding to each topic obtained for each iteration, based on the number of continuous value data included in the section of the histogram corresponding to the topic, Estimating the shape of each section of each of the histograms corresponding to each of the topics and the weight for each of the histograms corresponding to each of the topics in each individual, and for each of the plurality of individuals to the individual A weight for each of the histograms estimated for the Based on the shape of each section of each of the histograms corresponding to each of the click, and estimating a probability density function expressed by a linear sum of each of the histograms corresponding to each of the topics,
A probability density function estimation method including: The step of estimating and updating by the topic estimation unit includes, for each continuous value data included in the set of continuous value data, the topic to which the continuous value data belongs according to the posterior probability that the continuous value data belongs to each topic. Calculate
For each of the histograms corresponding to each of the topics, the number of continuous value data of the individual belonging to the topic for each individual, obtained from the calculation result of the topic, and each section of the histogram corresponding to the topic The number of divisions of the histogram is calculated according to the a posteriori probability of the number of divisions of the histogram based on the number of continuous value data included in the data, and each individual and each topic obtained from the calculation result of the topic belong to the topic And the number of continuous value data included in the section of the histogram corresponding to the topic for each section of the histogram corresponding to each topic, and the histogram corresponding to each of the topics And the number of divisions calculated for each of the above Parameter represents the weight distribution for each of the histograms corresponding to each of the pick, and repeatedly updating the parameters representing the distribution of the histogram,
The step of estimating by the probability density function estimator is calculated for each of the parameter representing the distribution of the weight updated by the topic estimator, the parameter representing the distribution of the histogram, and the histogram corresponding to each of the topics. For each individual and each topic obtained for each number of iterations, the number of continuous value data of the individual belonging to the topic, and for each section of the histogram corresponding to each topic obtained for each iteration, Based on the number of continuous value data included in the section of the histogram corresponding to the topic, the shape of each section of the histogram corresponding to each of the topics, and each of the topics in each individual The probability of claim 4, wherein a weight for each of said histograms is estimated. Degree function estimation method. A continuous value prediction method in a continuous value prediction device for predicting continuous value data of an individual to be predicted,
A continuous value predicting unit, for the individual to be predicted, the weight for each of the histograms for each of the topics estimated in advance by the probability density function estimation method according to claim 4 or 5, and the topic Based on a probability density function represented by a linear sum of each of the histograms corresponding to each of the topics, based on the shape of each section of continuous values in each of the histograms corresponding to each of the histograms A continuous value prediction method including the step of calculating an expected value of continuous values generated in the future for.   The program for functioning a computer as each part which comprises the probability density function estimation apparatus of Claim 1 or 2.   The program for functioning a computer as each part which comprises the continuous value prediction apparatus of Claim 3.