JP6908286B2 - Information processing equipment, information processing methods and programs - Google Patents

Description

The present disclosure relates to information processing devices, information processing methods and programs, and in particular, information processing devices, information processing methods and programs capable of improving accuracy in the process of segmenting series data and classifying them into segments. Regarding the program.

In general, humans perceive continuous information that they perceive by segmenting it into meaningful units. For example, humans can recognize words that are meaningful units by segmenting continuous speech waveforms, or recognize meaningful unit movements by segmenting continuous movements. can do.

In addition, when segmenting continuous information in this way, human beings learn without being given explicit segment points and segment into meaningful units in the myriad of possibilities. Can be done. Therefore, for example, it is very important for robots to have the ability to segment continuous time series information into unit series without a teacher in order to flexibly learn languages, movements, gestures, etc. It is believed that there is.

Therefore, the present inventors have proposed, for example, a method of segmenting a body motion sequence based on a hidden semi-Markov model of the Gaussian process and classifying each unit motion (for example, Non-Patent Document 1). reference).

Tomoaki Nakamura, A. Muhammad, Takayuki Nagai, Daichi Mochihashi, Ichiro Kobayashi, Hideki Aso, Masahide Kaneko, "Segmentation of Body Movement Based on Hidden Semi-Markov Model of Gaussian Process", Japanese Society for Artificial Intelligence, 1O3-5, 2016.

By the way, in the method proposed in Non-Patent Document 1 described above, the movement series of the body is segmented by using only the fixed coordinate system, and each unit movement is classified. However, in general, among various movements, the object to perform the movement may be important.

For example, in the action of grasping an object, the motion trajectory changes depending on the position of the object to be grasped. For such movements, it is assumed that it is difficult to accurately classify each unit movement by segmenting the movement series of the body without considering the object to be performed. ..

The present disclosure has been made in view of such a situation, and is intended to improve the accuracy in the process of segmenting the series data and classifying it into segments.

The information processing apparatus of one aspect of the present disclosure includes a coordinate system conversion unit that converts a coordinate system of series data into a first coordinate system or a second coordinate system, and the first coordinate system by the coordinate system conversion unit. Alternatively, a plurality of unit series data obtained by segmenting the series data converted into the second coordinate system, a specific unit that specifies a class for classifying the unit series data, and the specific unit that specifies the unit series data and It is provided with a parameter adjusting unit that adjusts the parameters used when specifying the class.

The information processing method or program of one aspect of the present disclosure converts the coordinate system of the series data into the first coordinate system or the second coordinate system, and converts it into the first coordinate system or the second coordinate system. It includes a plurality of unit series data obtained by segmenting the series data, a class for classifying the unit series data, and adjusting the unit series data and parameters used when specifying the class. ..

In one aspect of the present disclosure, the coordinate system of the series data is converted to the first coordinate system or the second coordinate system, and the series data converted into the first coordinate system or the second coordinate system is segmented. The plurality of unit series data and the class for classifying each unit series data are specified, and the unit series data and the parameters used when specifying the class are adjusted.

According to one aspect of the present disclosure, it is possible to improve the accuracy in the process of segmenting the series data and classifying it into segments.

It is a figure explaining a Gaussian process. It is a figure which shows the graphical model of the segmentation method. It is a figure explaining the length of a unit series. It is a figure explaining the classification of a unit series. It is a figure explaining forward filtering. It is a figure explaining the backward sampling. It is a block diagram which shows the structural example of one Embodiment of the information processing apparatus to which this technique is applied. It is a figure explaining the coordinate system transformation with the trajector and the landmark as the origin. It is a flowchart explaining BGS processing. It is a flowchart explaining FFBS processing. It is a flowchart explaining the coordinate system conversion process. It is a figure which shows an example of the algorithm of BGS processing. It is a figure which shows an example of the algorithm of FFBS processing and coordinate system conversion processing. It is a figure which shows the result of segmentation by the simulation. It is a figure which shows the outline of the operation of a robot. It is a figure which shows the result of having segmented the movement of a robot. It is a figure which shows the operation of the robot generated based on learning. It is a figure which shows the graphical model of the method of estimating the number of classes and segmenting. It is a figure explaining the occurrence probability of a class. It is a figure which shows an example of the BGS processing algorithm which can estimate the number of classes. It is a figure which shows one processing example of the FFBS processing algorithm which can estimate the number of classes. It is a figure which shows the example which segmented the observed value. It is a figure explaining an example of setting of a reference point and recognition of an operation. It is a figure explaining the coordinate transformation which considered the posture of an object. It is a figure explaining correspondence to a plurality of landmarks. It is a figure which shows an example of the algorithm of the FFBS processing and the coordinate system conversion processing which realized the introduction of the object posture and correspondence to a plurality of landmarks. It is a block diagram which shows the structural example of one Embodiment of the computer to which this technique is applied.

Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

<About the Gaussian process>

First, the Gaussian process will be described with reference to FIGS. 1 to 4.

In FIG. 1, the vertical axis represents the observed value, the horizontal axis represents the time, and the change of the observed value with the passage of time is represented as the observation series S.

As shown in FIG. 1, the observation sequence S can be segmented into predetermined classes (waveforms having similar shapes) and classified into _{unit sequences x j representing the waveforms of each predetermined shape.} In the example of FIG. 1, the unit series x ₁ and x ₂ are classified into a class in which the observed value becomes a convex shape and then a concave shape, and the unit series x ₃ and x ₅ have an observed value in a flat shape. It is classified into classes, and the unit series x ₄ can be classified into classes in which the observed values have a convex shape.

As a method for performing such segmentation, for example, by setting the output in the Hidden Semi-Markov Model (HSMM) as a Gaussian process, one state can be a continuous unit series x _j . You can use the model to represent.

That is, each class shown in FIG. 1 can be represented by a Gaussian process, and the observation sequence S is a unit sequence x _j generated from each class (in the example of FIG. 1, j = 1 to 5). It is generated by connecting. Then, by learning the parameters of the model based only on the observation series S, it is possible to estimate the segment points that segment _{the observation series S into the unit series x j and the class of the unit series x j without a teacher.} can.

FIG. 2 shows a graphical model of a method of segmenting the observation sequence S for _{each unit sequence x j.}

As shown in FIG. 2, observation sequence S is the unit sequence x _j, class c _j of a unit sequence x _j, and can be represented by the model using the parameters X _c of Gaussian process of class c. Then, by combining these unit series x _j , the observation series S (= x ₀ , x ₁ , ..., X _J ) is generated. The parameter X _{c of the} Gaussian process is a set of unit series x classified into class c, and the number of segments J is an integer representing the number of unit series x in which the observation series S is segmented.

Here, assuming that the time series data is generated by a hidden semi-Markov model whose output distribution is a Gaussian process, the class c _j is determined by the following equation (1), and the unit series x _j is as follows. It is generated by the formula (2).

Then, a hidden semi Markov models, to estimate the parameters X _c of Gaussian process shown in Equation (2), and segmented the observation sequence S into unit sequence x _j, each unit sequence x _j for each class c It becomes possible to classify.

Further, for example, the output value x _i at time step i of the unit series by learning a Gaussian process regression, represented as a continuous track. Therefore, in the Gaussian process, when the set (i, x) of the output value x in the time step i of the unit series belonging to the same class is obtained, the predicted distribution of the output value x'in the time step i'is as follows. It has a Gaussian distribution represented by (3). In the equation (3), k is a vector of _{k (i} p, i _q) of an element, c is, k (i ', i' ) and a scalar composed, C is the following formula It is a matrix having elements as shown in (4).

However, in equation (4), β is a hyperparameter representing the accuracy of noise included in the observed value.

Further, in the Gaussian process, even series data that changes in a complicated manner can be learned by using the kernel. For example, the Gaussian kernel represented by the following equation (5) widely used for Gaussian process regression. Can be used. However, in equation (5), θ _* is a kernel parameter.

Then, when the output value x _i is a multidimensional vector (x _i = x _{i, 0} , x _{i, 1} , ...), It is assumed that each dimension is generated independently, and the time i probability GP of observations x _i is generated from Gaussian process corresponding to the class c is determined by calculating the following equation (6).

By using the probability GP obtained in this way, similar unit series can be classified into the same class.

By the way, in the hidden semi-Markov model, _{the length of the unit series x j} classified into one class c differs depending on the class c. Therefore, _{when estimating the parameter X c of the} Gaussian process, the length of the unit series x _j It also needs to be estimated.

For example, as shown in FIG. 3, _{the length k of the unit series x j} is determined by sampling from the probability _{that the unit series x j} of the length k having the data point at time t as the end point is classified into the class c. can do. Therefore, in order to determine the length k of the unit series x _j , the probability of the combination of various lengths k and all the classes c is determined by using FFBS (Forward filtering-Backward sampling) as described later. Need to calculate.

Then, by estimating the parameter X _{c of the} Gaussian process, as shown in FIG. 4, the unit series x _j can be classified into a class in which the observed value becomes a convex shape and then becomes a concave shape, or the unit series x. _{j + 1} can be classified into a class in which the observed value has a flat shape.

<About FFBS>

The FFBS will be described with reference to FIGS. 5 and 6.

_{For example, in FFBS, the probabilities α [t] [k] [c] in which the unit series x j} of length k is classified into class c with the data point at time t as the end point are positively calculated, and the probabilities α [t] are calculated positively. ] [k] in order from the back in accordance with [c], the length k and class c of a unit sequence x _j can be determined by sampling. For example, the probabilities α [t] [k] [c] are recursively calculated by marginalizing the possibility of transition from time tk to time t, as shown in the equation (11) described later. Can be done.

In FIG. 5, as an example of forward filtering, the probabilities α [t] [k = 2] [c = 2] of transitioning to _{the unit series xj} of length k = 2 and class c = 2 with time t as the end point. How to find is schematically shown.

As represented by formula shown at the bottom of FIG. 5, for example, the possibility of transition to the length k = 2 and class c = 2 units sequence x _j at time t, the length k at time t-2 = 1 and the possibility of transition from the class c = 1 unit sequence _{x j} is p | a (2 1) α [t- 2] [1] [1]. Moreover, the possibility of the transition from the length k = 2 and class c = 1 unit sequence _{x j} at time t-2 p (2 | 1 ) α [t-2] a [2] [1], the possibility of the transition from the length k = 3 and class c = 1 unit sequence _{x j} at time t-2 p (2 | 1 ) α [t-2] [3] is [1]. Similarly, the time length in t-2 k = 1 and the possibility of transition from the class c = 2 units series _{x j p (2 | 2)} α [t-2] [1] be [2] , the possibility of the transition from the length k = 2 and class c = 2 units sequence _{x j} at time t-2 p | a (2 2) α [t- 2] [2] [2], the time the possibility of the transition from the length k = 3 and class c = 2 units sequence _{x j} in t-2 p (2 | 2 ) α [t-2] [3] is [2].

By performing such a calculation positively from the probabilities α [0] [*] [*] by dynamic programming, any probabilities α [t] [k] [c] can be obtained.

FIG. 6 schematically shows sampling when the length k = 3 and the length k = 2 as an example of backward sampling.

_{For example, the transition to the unit series x j} of the length k = 3 and the class c = 1 at the time t can be any _{of the unit series x j at} the time t-3 because the length k = 3. Yes, it can be determined from their probabilities α [t-3] [*] [*].

Here, for example, it is assumed that the unit series x _j having a length k = 2 and a class c = 2 is determined at time t-3. In this case, since _{the transition to the unit series x j} has a length k = 2, _{any one of the unit series x j} at time t-5 is possible, and their probabilities α [t-5] [ *] Can be determined from [*].

In this way, by performing sampling based on the probabilities α [t] [k] [c] in order from the back, the length k and the class c of _{all the unit series x j can be determined.}

<Configuration example of information processing device>

FIG. 7 is a block diagram showing a configuration example of an embodiment of an information processing device to which the present technology is applied.

As shown in FIG. 7, the information processing apparatus 11 includes a coordinate system conversion unit 12, a BGS execution unit 13, and an FFBS execution unit 14. Further, the FFBS execution unit 14 includes a forward filtering unit 15 and a backward sampling unit 16.

Here, in the present embodiment, the robot hand is directed toward an object (hereinafter referred to as a landmark) that is the target of the operation of grasping, such as the operation of grasping an object with the robot hand (see FIG. 15 described later). An example of performing an operation of bringing an object (hereinafter, referred to as a projector), which is a main body of the operation, close to the object will be described. Further, in the present embodiment, whether to set the reference point as the origin of the coordinate system to the projector or the landmark is set in advance for each class c.

The coordinate system converter 12, as an observation value, vector p _n representing the coordinates trajector moves with respect to a predetermined reference point is supplied. Further, the coordinate system conversion unit 12 is set _{with a vector p landmark} that represents the coordinates of the position where the landmark is arranged with respect to a predetermined reference point. Then, the coordinate system converting unit 12, according to the reference point is set for each class c, and the vector p _n, by converting trajector or landmark either in the vector p _'n obtained by converting the coordinate system with the origin, It is supplied to the forward filtering unit 15 of the FFBS execution unit 14.

For example, the coordinate system converting unit 12, when the reference point of the class c is set to trajector, segment from the vector _{p t-k} to the vector _{p k s' t-k:} k (= p t-k, p _t-k + 1, ···, the _{p k),} according to the following equation (7), converted into the coordinate system with the origin of the initial position of trajector.

Thus, as shown in A of FIG. 8, the vector p _n, as the origin of trajector, x-axis is set toward a horizontal direction, a direction orthogonal to the x axis, and the direction of gravity y-axis is converted into a vector p _'n coordinate system that is set to be backward against.

On the other hand, when the reference point of the class c is set to the landmark, the coordinate system conversion unit 12 has the x-axis in the direction from the landmark to the projector with the landmark as the origin, as shown in B of FIG. It is converted into a coordinate system that is set and has a direction orthogonal to the x-axis and a y-axis set so as to go to the opposite side of the gravity direction. For example, when the data point is a two-dimensional vector _pn = ( _{pn, 0} , _{pn, 1} ), the coordinate system conversion unit 12 follows the following equations (8), (9), and (10). , Find the coordinate system with the landmark as the origin. However, R (θ) shown in the equation (8) is a rotation matrix of θ rotation, and is expressed as shown in the equation (10).

The BGS execution unit 13 executes BGS (Blocked Gibbs Sampler) estimated by sampling _{the length k of the unit series x j} when segmenting the observation series S and the class c of each unit series x _j. do. Further, the BGS execution unit 13 puts together the length k of _{the unit series x j} when segmenting one observation series S and the class c of _{each unit series x j in order to perform the calculation efficiently.} Can be sampled. The BGS algorithm executed by the BGS execution unit 13 will be described later with reference to the flowchart of FIG.

_{Then, the BGS execution unit 13 uses the parameters N (cn, j} ) and the parameters N (cn _{, j} , cn _{, j + 1} ) used by the FFBS execution unit 14 to obtain the transition probability by the equation (13) described later. Is supplied to the FFBS execution unit 14. For example, the parameter N _{(c n, j)} is the class _{c n,} represents the number of segments in which a _j, a parameter _{N (c n, j, c} n, j + 1) , the class _{c n,} classes from _{j c n , J + 1} represents the number of transitions. Further, the BGS execution unit 13 sets the parameter N (cn _{, j} ) and the parameter N (cn _{, j} , cn _{, j + 1} ) as the current parameter N (c') and the parameter N (c', c). The observation sequence S to be segmented is supplied to the FFBS execution unit 14 while being supplied to the FFBS execution unit 14.

Similar to Non-Patent Document 1 described above, the FFBS execution unit 14 hides both the length k of _{the unit series x j} when segmenting the observation series S and the class c of _{each unit series x j.} FFBS that samples at the same time is executed.

As described above with reference to FIG. 5, the forward filtering unit 15 has a probability α [t] [k] [c] that _{the unit series x j of length k is classified into the class c with a certain time t as the end point.} Is supplied to the backward sampling unit 16.

For example, forward filtering unit 15, 'segmental s based on' _t-k vector p supplied from the coordinate system conversion unit _{12: k (= p 't} -k, p' t-k + 1, ···, p The _{probability α [t] [k] [c] in which'k} ) is in class c can be obtained by calculating the following equation (11).

However, in equation (11), C is the number of classes and K is the maximum length of the unit series. Further, P ( _{s'tk: k} | X _{c ) is the probability that the segment s'tk: k} is generated from the class c, and is obtained by the following equation (12).

_{However, Plen} (k | λ) in Eq. (12) is a Poisson distribution having an average of λ, and is a probability distribution of segment length. Further, p (c | c') in the equation (11) represents the transition probability of the class, and is obtained by the following equation (13).

However, in the equation (13), N (c') represents the number of segments that have become class c', and N (c', c) represents the number of transitions from class c'to class c. _{, Parameters N (cn, j} ) and N (cn _{, j} , cn _{, j + 1} ) supplied from the BGS execution unit 13, respectively, are used. Also, k 'is segmented s' _t-k: represents the length of the previous segment of _k, c 'is segmented s' _t-k: represents the class of the previous segment of _k, the formula (11 ), It is marginalized in all lengths k and class c.

When t−k <0, the probability α [t] [k] [*] = 0 and the probability α [0] [0] [*] = 1.0. The equation (11) is a recurrence formula, and all patterns can be calculated by dynamic programming by calculating from the probabilities α [1] [1] [*].

As described above with reference to FIG. 6, the backward sampling unit 16 backwardly sets the length and class of the unit series according to the forward probabilities α [t] [k] [c] supplied from the forward filtering unit 15. Sampling. As a result, the backward sampling unit 16 can determine the length k of _{the unit series x j} , which is a segmentation of the observation series S, and the class c of _{each unit series x j.}

As described above, the information processing device 11 is configured, and the coordinate system conversion unit 12 can convert the information processing device 11 into a coordinate system having either a projector or a landmark as an origin. As a result, the information processing apparatus 11 accurately segments the unit series representing the movement for the movement of the tracker moving toward the landmark, and classifies the unit series into the class indicating the movement. Can be done.

Further, when the information processing apparatus 11 performs such segmentation and classification, the observation sequence S is segmented by repeating the processing by the BGS execution unit 13 and the FFBS execution unit 14 for all the observation series S. The length k (segmentation point) of the converted unit series x _{j and the parameter X c of the} Gaussian process can be optimized alternately.

<BGS processing>

The BGS process executed by the BGS execution unit 13 will be described with reference to FIG.

In step S11, the BGS execution unit 13 initializes the observation sequence S when the observation sequence S to be segmented and classified is supplied to the information processing apparatus 11. For example, BGS execution unit 13 performs sampling collectively for a plurality of observation sequence _S n (n = _1~N), in step S11, for all the observation sequence _{S 1} to the observation sequence _{S N,} Initialization is performed by randomly segmenting and classifying.

In step S12, BGS execution unit 13, the index n of the observation sequence _{S n,} and sets 1 as an initial value, in step S13, BGS execution unit 13, the unit sequence _{x n,} the index j of the _j, initial Set 1 as the value.

In step S14, the BGS execution unit 13 decrements the parameter N (cn _{, j} ), and in step S15, the BGS execution unit 13 decrements the parameter N (cn _{, j} , cn _{, j + 1} ). ..

In step S16, the BGS execution unit 13 deletes the _{unit series x n, j} from the parameters X (cn _{, j) of the Gaussian process.}

In step S17, BGS execution unit 13 determines the unit sequence _{x n, j} index j of the unit sequence _{x n} that is segmented at the present _time, whether treatment was repeated until the number _{J n} of _j ..

In step S17, if the BGS execution unit 13, the unit sequence x _{n, j} index j of has determined moment in unit sequence x _n being _segmented, and not repeat the process until the number J _n of _j , The process proceeds to step S18.

In step S18, the BGS execution unit 13 _{increments the index j of the unit series x n, j} , and then the process returns to step S14, and the same process is repeated thereafter.

On the other hand, in step S17, if the BGS execution unit 13, the unit sequence _{x n, j} index j of has determined unit sequence _{x n} that is segmented at the present _time, the treatment was repeated until the number _{J n} of _j , The process proceeds to step S19.

In step S19, the FFBS processing is executed by the FFBS execution unit 14, as will be described with reference to FIG. 10 described later.

In step S20, the BGS execution unit 13 sets 1 as an initial value in the index j of the _{unit series x n, j.}

In step S21, the BGS execution unit 13 _{increments the parameter N (cn, j} ), and in step S22, the BGS execution unit 13 increments the parameter N (cn _{, j} , cn _{, j + 1} ).

In step S23, the BGS execution unit 13 adds the _{unit series x n, j} to the parameters X (cn _{, j) of the Gaussian process.}

In step S24, BGS execution unit 13 determines the unit sequence _{x n, j} index j of the unit sequence _{x n} that is segmented at the present _time, whether treatment was repeated until the number _{J n} of _j ..

In step S24, if the BGS execution unit 13, the unit sequence x _{n, j} index j of has determined moment in unit sequence x _n being _segmented, and not repeat the process until the number J _n of _j , The process proceeds to step S25.

In step S25, the BGS execution unit 13 _{increments the index j of the unit series x n, j} , and then the process returns to step S21, and the same process is repeated thereafter.

On the other hand, in step S24, if the BGS execution unit 13, the unit sequence _{x n, j} index j of has determined unit sequence _{x n} that is segmented at the present _time, the treatment was repeated until the number _{J n} of _j , The process proceeds to step S26.

In step S26, BGS execution unit 13, the index n of the observation sequence _{S n} determines whether treatment was repeated until the sequence number N of the observation sequence _{S n.}

In step S26, BGS execution unit 13, the index n of the observation sequence _{S n} is, if it is determined that no repeat process until the sequence number N of the observation sequence _{S n,} the process proceeds to step S27.

In step S27, BGS execution unit 13, after incrementing the index n of the observation sequence _{S n,} the process returns to step S13, and the same processing is repeatedly performed.

On the other hand, in step S26, BGS execution unit 13, the index n of the observation sequence _{S n} is, when it is determined that the treatment was repeated until the sequence number N of the observation sequence _{S n,} the process proceeds to step S28.

In step S28, BGS execution unit 13 a result of conducted by repeating the processing from step S12 to step S27, whether the unit sequence _{x n} sought FFBS processing in step _{S19, j} and class _{c n, j} respectively converged Judge whether or not.

In step S28, when the BGS execution unit 13 determines that the unit series x _{n, j} and the classes c _{n, j} currently required have not converged, the process returns to step S12, and so on. Processing is repeated.

On the other hand, in step S28, BGS execution unit 13, the unit sequence _{x n} sought at the present _{time, j} and class _{c n, j,} when judging that converged respectively, BGS process ends.

<FFBS processing>

The FFBS process executed by the FFBS execution unit 14 in step S19 of FIG. 9 will be described with reference to FIG. The forward filtering unit 15 performs processing from step S31 to step S41, and the backward sampling unit 16 performs processing from step S42 to step S47.

In step S31, the forward filtering unit 15 sets 1 as the initial value in the time step t, and in step S32, the FFBS execution unit 14 sets the length k of the _{unit series x n, j as the initial value.} Set 1.

In step S33, the forward filtering unit 15 selects and processes the classes c in a predetermined order in order to obtain the probabilities α [t] [k] [c] for the plurality of classes c. Select class c.

In step S34, the coordinate system conversion process is performed by the coordinate system conversion unit 12 as described with reference to FIG. 11 described later.

In step S35, the forward filtering unit 15 calculates the probabilities α [t] [k] [c] by calculating the above-mentioned equation (11) using the vector p'of which the coordinate system is transformed in step S34. calculate.

In step S36, the forward filtering unit 15 determines whether or not the process of obtaining the probabilities α [t] [k] [c] has been performed for all the classes c.

If it is determined in step S36 that the forward filtering unit 15 has not performed the process of obtaining the probabilities α [t] [k] [c] for all the classes c, the process proceeds to step S37.

In step S37, the forward filtering unit 15 selects the next class in a predetermined order, and then the process returns to step S34, and the same process is repeated.

On the other hand, if it is determined in step S36 that the forward filtering unit 15 has performed the process of obtaining the probabilities α [t] [k] [c] for all the classes c, the process proceeds to step S38.

In step S38, the forward filtering unit 15 determines the unit sequence x _n, the length k of _j is a unit sequence x _n, whether treatment was repeated until the maximum length K of _j.

In step S38, the case of forward filtering unit 15, the unit sequence x _n, the length k of _j has determined unit sequence x _n, and not repeat the process until the maximum length K of _j, the process steps Proceed to S39.

In step S39, the forward filtering unit 15 _{increments the length k of the unit series x n, j} , and then the process returns to step S33, and the same process is repeated thereafter.

On the other hand, in step S38, the case of forward filtering unit 15, the unit sequence x _n, the length k of _j has determined unit sequence x _n, and treatment was repeated until the maximum length K of _j, the process steps Proceed to S40.

In step S40, the forward filtering unit 15 determines whether or not the process is repeated until the time step t reaches the maximum value T of the time step t.

If the forward filtering unit 15 determines in step S40 that the process is not repeated until the time step t reaches the maximum value T of the time step t, the process proceeds to step S41.

In step S41, the forward filtering unit 15 increments the time step t, then returns to step S32, and the same processing is repeated thereafter.

On the other hand, in step S40, when the forward filtering unit 15 determines that the process is repeated until the time step t reaches the maximum value T of the time step t, the process proceeds to step S42.

In step S42, the backward sampling unit 16 sets the maximum value T as the initial value in the time step t, and sets 1 as the initial value in the index j of the _{unit series x n, j.}

In step S43, the backward sampling unit 16 samples the length k and the class c of _{the unit series x n, j} based on the probabilities α [t] [k] [c] obtained in step S35.

In step S44, the backward sampling unit 16 _{returns the time step t backward by the length k of the current unit series x n, j} (t = tk).

In step S45, the backward sampling unit 16 increments the index j of _{the unit series x n, j.}

In step S46, the backward sampling unit 16 determines whether or not the process is repeated until the time step t becomes 0.

If the backward sampling unit 16 determines in step S46 that the process is not repeated until the time step t becomes 0, the process returns to step S43, and the same process is repeated thereafter.

On the other hand, in step S46, when the backward sampling unit 16 determines that the process is repeated until the time step t becomes 0, the process proceeds to step S47.

In step S47, the backward sampling unit 16 _{outputs the combination of the unit series x n, j} and the class _{cn, j} obtained by repeating the sampling in step S43 to the BGS execution unit 13, and the FFBS process is completed.

<Coordinate system conversion process>

With reference to FIG. 11, the coordinate system conversion process executed by the coordinate system conversion unit 12 in step S34 of FIG. 10 will be described.

In step S51, the coordinate system conversion unit 12 represents all the vectors p (= _pt-k , _{pt-k + 1} , ..., P) _{representing the unit series x n, j} that are the targets of the coordinate system conversion processing. _{Of k} ), the first vector p is selected.

In step S52, the coordinate system conversion unit 12 determines whether the reference point set in the class c currently selected (selected in step S33 or S37 of FIG. 10) is a projector or a landmark. do.

If the coordinate system conversion unit 12 determines in step S52 that the reference point set in the class c is a projector, the process proceeds to step S53.

In step S53, the coordinate system conversion unit 12 converts the vector p into the vector p'of the coordinate system with the initial position of the projector as the origin using the above equation (7), and the process proceeds to step S57.

On the other hand, if the coordinate system conversion unit 12 determines in step S52 that the reference point set in the class c is a landmark, the process proceeds to step S54.

In step S54, the coordinate system conversion unit 12 converts the vector p into the vector p'of the coordinate system with the landmark as the origin by using the above equation (8).

In step S55, the coordinate system conversion unit 12 determines whether or not the y-axis of the vector p'converted in step S54 is oriented in the direction of gravity.

If the coordinate system conversion unit 12 determines in step S55 that the y-axis of the vector p'converted in step S54 is oriented in the direction of gravity, the process proceeds to step S56.

In step S56, the coordinate system conversion unit 12 inverts the y-axis so as to face the opposite side with respect to the direction of gravity (see the above equation (9)).

After the process of step S56, or when it is determined in step S55 that the y-axis of the vector p'is not oriented in the direction of gravity, the process proceeds to step S57.

In step S57, the coordinate system conversion unit 12 determines whether or not all the vectors p representing the _{unit series x n, j} that are the targets of the coordinate system conversion process have been converted into the vector p'.

If the coordinate system conversion unit 12 determines in step S57 that all the vectors p have not been converted into the vector p', the process proceeds to step S58.

In step S58, the coordinate system conversion unit 12 selects the next vector p, and then the process returns to step S52, and the same process is repeated thereafter.

On the other hand, in step S57, when the coordinate system conversion unit 12 determines that all the vectors p have been converted into the vector p', the coordinate system conversion process is terminated.

As described above, in the information processing apparatus 11, by performing BGS processing, FFBS processing, and coordinate system conversion processing, the class in which the tracker moves in relation to the landmark and the trajector become the landmark. It is possible to accurately classify moving classes that are irrelevant to each other.

An example of the above-mentioned BGS processing algorithm can be described as shown in FIG. 12, and an example of the above-mentioned FFBS processing and coordinate system conversion processing algorithm can be described as shown in FIG. ..

<Simulation segmentation>

FIG. 14 shows the result of evaluating the segmentation of the object operation task by simulation in order to confirm the effectiveness of the segmentation and classification by the information processing apparatus 11.

As shown in FIG. 14, for example, on a computer, the position of the landmark is fixed, the simulation condition is set so that the position of the projector can be operated by the user, and the landmark and the projector are displayed on the display. , A simulation was performed in which the projector was operated by the operation input means. In addition, the initial position of the projector is randomly changed each time the simulation is performed.

In such a simulation, four operations were randomly performed five times: the operation of rotating the position of the projector, the operation of moving the projector closer to the landmark, the operation of moving the projector away from the landmark, and the operation of placing the projector on the landmark. .. In addition, when performing each operation, the image coordinates of the projector and the landmark were acquired, and one continuous operation was performed. Then, these four operations were randomly performed five times, and the process was repeated seven times to obtain a total of seven continuous operation data.

Then, the acquired continuous operation data was segmented and classified into four classes by the information processing device 11. In addition, the number of classes and the coordinate system of each class are set in advance and simulated, the transistor coordinate system (TCS) is set in one of the four classes, and the landmark coordinate system (landmark coordinate system) is set in the other three classes. LCS) was set.

As shown in FIG. 14, the operation of placing the projector on the landmark is classified into class 1, the operation of moving the projector closer to the landmark is classified into class 2, and the operation of moving the projector away from the landmark is classified into class 3. Was done. These operations are performed on landmarks, and the results are correctly classified into the landmark coordinate system (LCS) class.

Further, the operation of rotating the position of the projector was classified into class 4. This operation is performed independently of the landmark, and the result is that it is correctly classified into the class of the Traveler Coordinate System (TCS).

<Segmentation of robot movement>

The segmentation and classification of the object grasping motion by the robot will be described with reference to FIGS. 15 to 17.

As shown in FIG. 15, the operation was performed so as to perform three operations: an operation in which the robot moves forward, an operation in which the robot moves backward, and an operation in which the gripper (trajector) of the robot grips an object (landmark). For example, these three operations were randomly performed five times to form one continuous operation, and these three operations were randomly performed five times five times to obtain five continuous operation data in total.

As a result, as shown in FIG. 16, the three-dimensional coordinates of the robot's hand and the information on the opening and closing of the gripper that grips the object are acquired as observed values, and the robot's operation is performed using these total four-dimensional features. Can be segmented.

For example, such observed values are used as time-series data, and the information processing apparatus 11 is used to segment and classify them into four classes. In addition, the number of classes and the coordinate system of each class are set in advance, and the transistor coordinate system (TCS) is set in two of the four classes, and the landmark coordinate system (LCS) is set in the other two classes. did.

As a result, the segmentation results obtained by segmenting the robot movements were obtained as classes 1 to 4 as shown in FIG.

In FIG. 16, the x-coordinate is the movement of the robot back and forth, the movement of the robot moving forward is classified into class 4, and the movement of the robot moving backward is classified into class 3. These actions are actions that are performed independently of the object, and the results are correctly classified into the class of the Traveler Coordinate System (TCS).

Further, in FIG. 16, the z coordinate and the gripper are the height and opening / closing (1: open, 0: closed) of the gripper, and the action of lowering the arm to approach the object and closing the gripper is classified into class 1, and the gripper is an object. The movement of grabbing and lifting was classified into class 2. These movements are related to the object, and the results are correctly classified into the Landmark Coordinate System (LCS) class.

The Gaussian process has the advantage that a learned continuous orbit can be easily generated. For example, by using the information processing device 11 as a learning device for learning the movement of the robot, the movement of the robot can be acquired as a learning result. For example, when the segmentation result of segmenting the robot movement as shown in FIG. 16 is learned and the class 4, class 1, and class 2 are combined to generate a trajectory, the learning is performed as shown in FIG. It is possible to acquire the motion of the robot generated based on.

That is, at times T1 to T2 shown in FIG. 17, an action of moving the robot forward is generated based on the class 4, and at times T2 to T4, an action of lowering the arm to approach the object and closing the gripper based on the class 1. Was generated, and at times T4 to T5, an action of grabbing and lifting an object with a gripper was generated based on class 2. Comparing the generated motion as shown in FIG. 17 with the observed value as shown in FIG. 16, the trajectory such as advancing, grasping the object, and lifting the object can be correctly generated. Then, when this trajectory is regenerated by the robot, the operation as shown in FIG. 15 is correctly performed.

As described above, the information processing apparatus 11 uses a hidden semi-Markov model with a Gaussian process as an output distribution to convert the entire operation into a tracker coordinate system or a landmark coordinate system according to a reference point, and segment the entire operation into unit operations. This makes it possible to accurately segment the actions performed on the landmark. That is, the operation performed on the landmark is classified as the operation with the landmark as the reference point, and the operation unrelated to the landmark is classified as the operation with the tracker as the reference point.

In the above-described embodiment, the number of classes to be divided into classes is specified in advance, and the process in which reference points are set for each class has been described. On the other hand, for example, by introducing nonparametric Bayes, it is possible to estimate the number of classes and automatically estimate the reference points of each class.

<Estimation of the number of classes>
Therefore, a method of automatically estimating the reference points of each class by estimating the number of classes will be described.

For example, in nonparametric Bayes, clustering can be performed assuming that there are an infinite number of classes. In reality, the learning data is finite, and the number of classes can be estimated by using only a finite number of classes to express the learning data generation process. Specifically, by introducing HPD (Hierarchical Dirichlet Process) to GP-HSMM (Gaussian Process --Hidden Semi Markov Model), a method using HDP-GP-HSMM extended to a nonparametric Bayes model is used. Can be adopted.

Various methods have already been proposed as methods for segmenting movements without a teacher, but here, GP (Gaussian Process), which can express movements without discretization, is used to make it more complicated. Adopt a method suitable for expressing the movement.

First, a nonparametric Bayesian model will be described.

For example, in a parametric mixture model, the number of classes is finite, the parameters of the multinomial distribution are generated from the finite-dimensional Dirichlet distribution, and the hidden state is determined from this multinomial distribution. On the other hand, in nonparametric Bayes, it is necessary to generate an infinite dimensional multinomial distribution because it assumes an infinite number of classes. For example, in SPB (Stick Breaking Process), which is one of the construction methods, an infinite dimensional multinomial distribution is constructed by continuously folding a bar of length 1 at a value sampled from a beta distribution.

That is, in SPB, as shown in the following equations (14) and (15), the parameter β of the infinite dimensional multinomial distribution can be constructed. This process is commonly referred to as β-GEM (γ).

By using such a Dirichlet process, it is possible to construct a transition probability of an HMM having an infinite number of states. However, simply configuring the transition probability of each state by SBP will result in different possible transition destinations for each state. That is, in the case of a model such as HMM, possible transition destinations are shared in all states, and the transition probability must form a distribution different for each state.

Therefore, the distribution generated by SBP is shared as a base measure, and the transition probability π _c, which differs for each state c, is generated by DP as shown in the following equation (16). In this way, the method of constructing an infinite dimensional probability distribution by two-step DP is called HPD.

Therefore, as shown in FIG. 18, a method of changing the graphical model as described with reference to FIG. 2 so that the number of classes can be estimated is proposed.

FIG. 18 shows a graphical model of a method of segmenting the _{observation sequence S for each unit sequence x j} , assuming an infinite number of classes. In this graphical model, the data is finite, and the number of classes is estimated as a result of using only a finite number of infinite classes to represent the data. At this time, as shown in FIG. 19, if the index of the class is large, the class is adjusted so that the probability of occurrence of the class is small, and a penalty is set so that the number of classes does not become too large (P (c'| c). ) = Π _c ). As a result, the number of classes converges to a finite number, and as a result, the final number can be estimated.

That is, in the graphical model shown in FIG. 18, the unit series class c _j (j = 1, 2, ..., J) assumes an infinite number of classes, and the transition probability π _c is a class. It represents the probability of transition from c to each class. Specifically, the transition probability π _c is shown in β generated by the GEM distribution (Stick Breaking Process) with γ as a parameter and in the following equation (18), as shown in the following equation (17). In addition, it is generated from DP (Dirichlet Process) with α as a parameter.

Further, the j-th class c _j is determined by the _j-1st class class c j-1 and the transition probability π _c. Here, it is assumed that the _{unit series x j is generated from the Gaussian process using the parameter X c} corresponding to the class, as shown in the following equations (19) and (20).

However, the parameter X _c is a set of unit series classified into the class c. Then, by combining these unit series, the observation series S (= x ₀ , x ₀ , ..., X _J ) can be obtained.

However, by learning with Gaussian process regression output values x _i in the time step i of the unit sequence is represented as a continuous track. Therefore, the Gaussian process, when the set of output x in the time step i of the unit series belonging to the same class (i, x) is obtained, predictive distribution of output x ^new at time step i ^{new new,} the following formula (21 ), The Gaussian distribution is obtained.

However, in the formula (21), the matrix C, using a kernel function k (·, ·) is a matrix with elements _{k (i} p, _{i q)} as shown in the following equation (22), beta Is a hyperparameter that represents the accuracy of the noise contained in the observed value.

Further, in the equation (21), the vector k will _have k a (i ^{p, i new new)} element, the class c ^is a ^{k (i new, i new)} .

Then, in the Gaussian process, even series data that changes in a complicated manner can be learned by using the kernel. Here, a Gaussian kernel as shown in the following equation (23), which is widely used for Gaussian process regression, is used. However, in equation (23), θ _* is a kernel parameter.

Furthermore, when the output value x _i is a multidimensional vector, it is assumed that each dimension is generated independently, and the observed value x (= x ₀ , x ₁ , ...) At time i corresponds to the class c. The probability GP (x | X _c ) generated from the Gaussian process is obtained by calculating the following equation (24).

By using the probability GP (x | _Xc ) obtained in this way, similar unit series can be classified into the same class.

Next, a method of applying such a probability GP to the information processing apparatus 11 of FIG. 7 and sampling and estimating a unit series in the observation series and its class by a dynamic programming method will be described.

For example, in the information processing apparatus 11, the BGS execution unit 13 can collectively sample the unit series in one observation series and their classes in order to efficiently calculate the dynamic programming method.

First, the BGS execution unit 13 initializes all observation sequences by randomly segmenting and classifying them. Next, an observation series _{S n} all unit sequence _x nj obtained by segmenting _{(j = 1,2, ···, J} n) and removes everything from the class in which they are classified, each _{Update the parameters X c} and the transition probability P (c | c') of the Gaussian process of the class.

Next, the FFBS execution unit 14 generates a new unit series x _ng (j = 1, 2, ..., J _n ) and their classes c _ng (j = 1, 2, ..., J _n ). _{And update the parameters Xc} and transition probability P (c | c') of the Gaussian process corresponding to each class. Then, by repeating the above operation for all the sequences, the BGS execution unit 13 and the FFBS execution unit 14 can alternately optimize the segment points of the observation sequence and the parameters of the Gaussian process.

Next, a process in which the FFBS execution unit 14 regards both the unit series and its class as hidden variables and simultaneously samples them by the FFBS will be described.

First, in the forward filtering unit 15 of the FFBS execution unit 14 _{, the probability α [t] that k sample s tk: k} ending at the time step t of a certain observation series s is a unit series and its class is c. [K] and [c] can be obtained by calculating the following equation (25).

However, in equation (25), C is the number of classes and K is the maximum length of the unit series. Further, P (c | c', β, α) represents the transition probability of the class c, and is obtained by the following equation (26).

However, in the equation (26), N (c') represents the number of unit series in the class c', and N (c', c) represents the number of transitions from the class c'to the class c. Therefore, the parameters N (cn _{, j} ) and N (cn _{, j} , cn _{, j + 1} ) supplied from the BGS execution unit 13 are used, respectively. Further, k'represents _{the length of the unit series before the unit series x tk: k} , c'represents the class of the unit series, and in equation (25), all lengths k. And is marginalized in class c.

When t−k <0, the probability α [t] [k] [*] = 0 and the probability α [0] [0] [*] = 1.0. The equation (25) is a recurrence formula, and all patterns can be calculated by dynamic programming by calculating from the probabilities α [1] [1] [*].

Then, the backward sampling unit 16 of the FFBS execution unit 14 samples the length and class of the unit series backward according to the forward probabilities α [t] [k] [c] supplied from the forward filtering unit 15. As a result, the backward sampling unit 16 can determine the length k of _{the unit series x j} , which is a segmentation of the observation series S, and the class c of _{each unit series x j.}

For example, FIG. 20 shows an example of a BGS processing algorithm capable of estimating the number of classes executed by the BGS execution unit 13. However, in the algorithm shown in FIG. 20, N _cnj and N _{cnj, cn, j + 1} are parameters for calculating the transition probability p shown in the above equation (26). Further, FIG. 21 shows an example of an FFBS processing algorithm capable of estimating the number of classes executed by the FFBS execution unit 14.

In order to confirm the effectiveness of segmentation and classification by the information processing apparatus 11, an experiment was conducted in which the observed values obtained as a result of actually measuring the operation were segmented.

FIG. 22 shows an example in which, for example, the movement when a human moves only the right hand is measured, and the observed values (x-coordinate and y-coordinate of the right hand) are segmented. In the experiment, three actions of raising the right hand small, raising the right hand high, and extending the right hand sideways were randomly performed, and time-series data as shown in the figure was acquired. As a result of segmenting and classifying this time series data by HDP-GP-HSMM as described above, the number of classes was estimated to be 4. In FIG. 22, the classified classes are represented by different hatching.

In the experiment, the number of correct classes is 3 because 3 actions were performed, but the number of classes is estimated to be 4 as a result of segmenting the state in which nothing is working before the start of the action as one class. ing. Then, as shown in the figure, the result is that the action of raising the hand small is classified into the class 2, the action of raising the hand high is classified into the class 3, and the action of extending the hand sideways is classified into the class 4. Was asked correctly.

As described above, by introducing the Dirichlet process into the hidden semi-Markov model with the Gaussian process as the output distribution, it is possible to segment and classify the motion while estimating the number of motion classes. By such a method, the motion break and the motion class are estimated at the same time by FFBS processing, and the number of classes is estimated by the nonparametric Bayesian model, so that the motion can be segmented correctly.

<Introduction of object posture>
As described above, in the information processing apparatus 11, the same operation can be expressed by the same orbit by appropriately converting the orbit to the coordinates with reference to the reference point and performing the FFBS process.

That is, as shown in A of FIG. 23 and B of FIG. 23, when a reference point is set in the gripper (trajector) of the robot, if the positions of the objects (landmarks) to be gripped are different, the robots use each object. The action of gripping by the gripper is expressed in a different trajectory. As a result, the same motion of the robot gripper gripping the object is recognized as a different motion.

On the other hand, as shown in C of FIG. 23 and D of FIG. 23, when a reference point is set for the object to be gripped, each object is gripped by the gripper of the robot even if the positions of the objects are different. Can be expressed in the same orbit. That is, regarding the operation of gripping an object by the gripper of the robot, the trajectory of the operation can be recognized as the same operation by appropriately setting a reference point on the object to be gripped.

Then, when setting the reference point to the object in this way, it is possible to segment the movement in consideration of the posture of the object by appropriately setting the coordinate system according to the posture of the object and performing the FFBS processing. Become.

For example, as shown in A of FIG. 24, when the longitudinal direction of the object is the vertical direction and the robot grips the object by the gripper of the robot, the gripper of the robot moves in a trajectory from the horizontal direction toward the object. It becomes. On the other hand, as shown in B of FIG. 24, when the longitudinal direction of the object is the horizontal direction, when the robot gripper grips the object, the robot gripper has a trajectory that goes from the vertical direction to the object. It becomes.

At this time, by appropriately setting the coordinate system according to the posture of the object (the longitudinal direction is the vertical direction or the horizontal direction), the trajectory of the gripper of the robot that grips the object in each posture is estimated in consideration of the posture of the object. can do. That is, by setting a coordinate system such that the longitudinal direction is the z-axis and the direction from the object to the gripper is the x-axis, in the example of A in FIG. 24, the gripper of the robot is directed from the horizontal direction to the object. It becomes an operation. Then, by setting the same coordinate system, the operation can be expressed by the same trajectory in the example B of FIG. 24. That is, by using the posture of the object as an element for estimating the motion, when a predetermined motion (the motion of gripping the object as described above) is performed on the objects having different postures, the motion is relative to each object. Actions can be classified as the same action.

Therefore, in the information processing device 11 of FIG. 7, the FFBS execution unit 14 executes the FFBS process by setting the coordinate system according to the posture of the object regardless of the posture of the object to be gripped. By doing so, it is possible to estimate the movement in consideration of the posture of the object.

On the other hand, when the posture is fixed for each object, such as the handle of a drawer, a coordinate system unique to each object exists. That is, as shown in C of FIG. 24, the posture of the object is fixed with the longitudinal direction facing the horizontal direction, and as shown in D of FIG. 24, the posture of the object is in the horizontal direction in the longitudinal direction. There is a coordinate system unique to the object, such as when it is fixed towards.

Even in such a case, the information processing device 11 of FIG. 7 considers the posture of each object by setting the unique coordinate system of the object and causing the FFBS execution unit 14 to execute the FFBS process. It is possible to estimate the behavior that has been performed.

<Correspondence to multiple landmarks>
When there are a plurality of objects in the environment, it is possible to estimate which object the operation is for by setting a reference point for each object and performing the FFBS process.

For example, as shown in FIG. 25, an example in which an object to be gripped (landmark 1) and a table (landmark 2) to which the object is moved exists as an object will be described. When there are two landmarks of an object and a table in this way, both the estimation of the operation in which the reference point is set in the object and the estimation of the operation in which the reference point is set in the table are performed. Then, based on the probability (consistency) of the trajectory and posture that have been coordinate-transformed at each reference point, the target object for movement is estimated by selecting the landmark that sets the reference point with the higher probability. can do.

For example, as shown in A of FIG. 25, when an operation of approaching an object to be gripped is performed, the probability obtained by setting a reference point on the object is the probability obtained by setting a reference point on the table. It can be estimated that the movement is for an object. Further, as shown in B of FIG. 25, when the operation of gripping the object is performed, the probability obtained by setting the reference point on the object is higher than the probability obtained by setting the reference point on the table. It becomes high and can be estimated to be an action targeting an object. Further, as shown in C of FIG. 25, when the operation of placing the object on the table is performed, the probability obtained by setting the reference point on the table is higher than the probability obtained by setting the reference point on the object. Is also high, and it can be estimated that the operation is for a table.

Therefore, when a plurality of objects exist in the environment, the information processing device 11 of FIG. 7 operates by the FFBS execution unit 14 setting a reference point for each object and executing the FFBS process. At the same time as estimating, it is possible to estimate which object is the object of motion.

Here, in FIG. 26, both the introduction of the object posture as described with reference to FIG. 24 and the correspondence with a plurality of landmarks as described with reference to FIG. 25 can be realized. Algorithms for FFBS processing and coordinate system conversion processing are shown.

For example, in the processing from the 9th line to the 12th line in FIG. 26, reference points are set for L landmarks, R postures are set for each landmark, and each landmark l and posture are set. With r as a candidate, the respective probabilities α are calculated in the process of the 15th line. Then, in the process of the 23rd line, the landmark to be operated and the posture of the landmark are estimated at the same time as the motion is estimated by sampling.

That is, in the information processing device 11 of FIG. 7, the coordinate system conversion unit 12 sets reference points for a plurality of landmarks and performs coordinate conversion, and the FFBS execution unit 14 sequentially candidates each landmark (l =). As 1 to L), a plurality of posture candidates (r = 1 to R) are applied to each landmark to obtain the probabilities α [t] [k] [c] [l] [r]. _{Then, the FFBS execution unit 14 estimates (identifies) the unit series x j} and the class c according to the probabilities α [t] [k] [c] [l] [r], and at the same time, is the target of the operation. It is possible to estimate (specify) a predetermined landmark l and estimate (specify) the posture r of the landmark.

Of course, as the number of classes used in the FFBS processing shown in FIG. 26, those estimated by the BGS processing shown in FIG. 20 described above can be used. That is, the information processing device 11 estimates the number of classes and estimates the operation, and at the same time, estimates the object to be operated from among a plurality of objects and also considers the posture of the object. Can be estimated.

Moreover, the above-mentioned method can be applied to various unsupervised learning.

It should be noted that each process described with reference to the above-mentioned flowchart does not necessarily have to be processed in chronological order in the order described as the flowchart, and is executed in parallel or individually (for example, parallel process or object). Processing by) is also included. Further, the program may be processed by one CPU or may be distributed by a plurality of CPUs.

Further, the series of processes (information processing method) described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software execute various functions by installing a computer embedded in dedicated hardware or various programs. It is installed from a program recording medium on which a program is recorded, for example, on a general-purpose personal computer.

FIG. 27 is a block diagram showing a configuration example of the hardware of a computer that executes the above-mentioned series of processes programmatically.

In a computer, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103 are connected to each other by a bus 104.

An input / output interface 105 is further connected to the bus 104. The input / output interface 105 includes an input unit 106 composed of a keyboard, a mouse, a microphone, etc., an output unit 107 composed of a display, a speaker, etc., a storage unit 108 composed of a hard disk, a non-volatile memory, etc., and a communication unit 109 composed of a network interface, etc. , A drive 110 that drives a removable medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is connected.

In the computer configured as described above, the CPU 101 loads the program stored in the storage unit 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executes the above-described series. Is processed.

The program executed by the computer (CPU101) is, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), an optical magnetic disk, or a semiconductor. It is recorded on removable media 111, which is a packaged medium including memory, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

Then, the program can be installed in the storage unit 108 via the input / output interface 105 by mounting the removable media 111 in the drive 110. Further, the program can be received by the communication unit 109 and installed in the storage unit 108 via a wired or wireless transmission medium. In addition, the program can be installed in the ROM 102 or the storage unit 108 in advance.

The present embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present disclosure.

11 Information processing device 12 Coordinate system conversion unit 13 BGS execution unit 14 FFBS execution unit 15 Forward filtering unit 16 Backward sampling unit

Claims (9)

A coordinate system conversion unit that converts the coordinate system of the series data to the first coordinate system or the second coordinate system,
Specifying a plurality of unit series data obtained by segmenting the series data converted into the first coordinate system or the second coordinate system by the coordinate system conversion unit, and a class for classifying the unit series data. Department and
An information processing device including a parameter adjusting unit for adjusting parameters used when the specific unit specifies the unit series data and the class. The series data is
The unit series data classified into a class indicating that the first object is moving in relation to the second object, and the unit series data.
It can be segmented into the unit series data classified into a class indicating that the first object is moving independently of the second object.
The coordinate system conversion unit converts the coordinate system of the series data into the first coordinate system with the initial position of the first object as the origin, and the coordinate system of the series data of the second object. The information processing apparatus according to claim 1, wherein at least one of the conversion to the second coordinate system with the position as the origin is performed. The specific unit uses a method of simultaneously sampling both the length of the unit series data when segmenting the series data and each class of the unit series data as hidden variables. The information processing apparatus according to claim 1 or 2, which specifies the class. The parameter adjusting unit alternately optimizes the unit series data and the parameter by repeating the adjustment of the parameter each time the unit series data is specified by the specific unit. The information processing apparatus according to any one of 3. The parameter adjusting unit assumes that the number of the classes is infinite, and each time the specific unit specifies the unit series data and the class, the larger the index of the class, the smaller the occurrence probability of the class. The information processing apparatus according to any one of claims 1 to 4, wherein the number of classes is estimated by repeating the process until the number of classes converges to a finite number. The coordinate system conversion unit targets the coordinate system of the series data with the first coordinate system in which a reference point is set for the first object that is the main body that performs the operation, and the operation by the first object. Converted to the second coordinate system in which the reference point is set for the second object,
The specific unit applies a plurality of posture candidates to the posture of the second coordinate system, and performs a process of specifying the unit series data and the class in each posture of the second object. The information processing apparatus according to any one of claims 2 to 5, which specifies the unit series data and the class with the posture as an element. The coordinate system conversion unit targets the coordinate system of the series data with the first coordinate system in which a reference point is set for the first object that is the main body that performs the operation, and the operation by the first object. Converted to the second coordinate system in which reference points are set for a plurality of second objects,
The specifying unit specifies the unit series data and the class by performing a process of specifying the unit series data and the class by using the plurality of second objects as candidates and using each as the second object. At the same time, any one of claims 2 to 5 for specifying a predetermined second object to which the first object has performed an operation from the plurality of second objects. The information processing device described. Convert the coordinate system of the series data to the first coordinate system or the second coordinate system,
A plurality of unit series data obtained by segmenting the series data converted into the first coordinate system or the second coordinate system, and a class for classifying the unit series data are specified.
An information processing method that includes a step of adjusting the unit series data and parameters used when identifying the class. Convert the coordinate system of the series data to the first coordinate system or the second coordinate system,
A plurality of unit series data obtained by segmenting the series data converted into the first coordinate system or the second coordinate system, and a class for classifying the unit series data are specified.
A program that causes a computer to perform information processing that includes a step of adjusting the unit series data and parameters used when identifying the class.

Images