JP4764362B2 - Event discrimination device and event discrimination program - Google Patents

Description

  The present invention relates to an event determination device and an event determination program for determining an event in video data.

  Conventionally, as an event detection method for detecting an event that is a specific scene from sports video that is video data of sports, the composition and camera work such as camera pan and zoom operations are detected from the sports video, and the angle of view and There is a method of indirectly detecting an event based on a shooting direction and a transition of camera work (for example, Non-Patent Document 1).

  In addition, conventional event detection methods determine the importance of sports video based on color, cut points, etc. in addition to composition, and determine the field of sports video (different from news, sports broadcasts, etc.) (For example, Patent Document 1 and Patent Document 2).

Furthermore, the conventional event detection method includes a method for detecting the type and timing of an event from a feature amount related to a player in a sports video, and a method for analyzing an instantaneous situation (formation) of a player position and a player speed (for example, Patent Document 2).
In addition, as a conventional event detection method, there is a method corresponding to event detection in various fields by modularizing an event estimation unit and using the module properly (for example, Patent Document 3).
JP-A-9-65287 JP 2006-251885 A JP 2006-285878 A JP 2000-123184 A "A feature analysis method for sports broadcast video database based on camera work information" Database Society of Japan Letters Vol.1, No.2, pp.32-35.

  However, the conventional event detection methods (Non-Patent Document 1, Patent Document 1 and Patent Document 2) discriminate the scene indirectly based on the camera work, so the skill difference between the cameraman who captured the sports video Depending on the intention of the production, an unexpected event detection result may be output.

  Incidentally, the conventional event detection method disclosed in Patent Document 1 can also refer to video switching or color in an image, but does not originally analyze the movement of a subject unit in an image, It is not suitable for detailed event determination.

  On the other hand, the conventional event detection method disclosed in Patent Document 3 discriminates the scene from the arrangement and movement of the subject, so that the difference in the skill of the photographer is not affected, and the event in the sports video of the arrangement and movement of the subject is not affected. Information related directly can be analyzed, and an event of a sports video can be determined. However, in order to apply the conventional event detection method disclosed in Patent Document 3 to various sports, it is necessary to subjectively prepare a rule base for each sport. As a result, there is a problem that the accuracy with which an event can be detected depends solely on the amount of knowledge about the sport of the person who built the rule base.

  Furthermore, in the conventional event detection method disclosed in Patent Document 4, the rule base event estimation means is modularized to be generalized, but the rule base construction method is subjective. After all, there is a problem that the accuracy with which an event can be detected depends exclusively on the amount of knowledge about the sport of the person who built the rule base.

  Therefore, the present invention provides an event determination device and an event determination program that solve the above-described problems, can be used universally for each sport, and can determine an event based on an objective determination criterion. Objective.

  In order to solve the above-mentioned problem, the event discriminating device according to claim 1 is an event discriminating device for discriminating an event which is a specific scene in sports video data, comprising a silhouette image extracting means and a silhouette feature amount extracting unit. And a positional relationship feature vector generation unit, a conversion unit, and an event determination unit.

According to such a configuration, the event determination device distinguishes the non-background region from the background region and the non-background region by using the silhouette image extraction unit as a reference based on the predetermined pixel value of the pixel included in the image constituting the video data. A silhouette image in which the region is a silhouette region of a person is extracted. Subsequently, the event discriminating apparatus is a silhouette feature vector which is a vector indicating a feature amount of the silhouette region including at least coordinate information regarding the silhouette region included in the silhouette image extracted by the silhouette image extracting unit by the silhouette feature amount extracting unit. To extract. Then, the event discriminating apparatus obtains a positional relationship feature vector that is a vector obtained by quantifying the positional relationship of the person using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction unit by the positional relationship feature vector generation unit. Generate.
At this time, the event discriminating apparatus uses the silhouette feature quantity extraction means to calculate the coordinate value on the coordinate space in which the ground space where the sport is being performed and the coordinate value are associated in advance as the coordinate information. Include in According to this, the event discriminating device makes it possible to determine where the subject exists in the ground space by outputting the coordinate value in the coordinate space by including it in the silhouette feature vector by the silhouette feature amount extraction means.
Also, at this time, the event discriminating device counts the persons present in each of the local areas obtained by dividing the ground space based on the coordinate values included in the silhouette feature vector by the positional relationship feature vector generation means, and the result of the counting Are included in the positional relationship feature vector and output. According to this, the event discriminating apparatus determines the number of subjects existing in the local region by outputting the result of counting the subjects included in the positional relationship feature vector by the positional relationship feature vector generating means.

  In addition, the event discriminating apparatus outputs a discriminating feature vector, which is a vector obtained by multiplying the positional relationship feature vector generated by the positional relationship feature vector generating unit by a preset matrix for weighting the feature amount by the converting unit. Then, the event discriminating apparatus uses the discriminating feature vector output from the converting unit by the event discriminating unit, and the event occurrence probability when the discriminating feature vector is obtained is previously modeled for each event type. The event is discriminated by referring to the probability model.

  Note that determining an event by referring to the probability model may mean that the event occurrence probability exceeds a threshold set in the probability model, or a probability density function obtained from the event occurrence probability. May be approximated to the density distribution set in advance by the probability model, or the occurrence probabilities for each event are arranged in order of magnitude, and a predetermined number of events are extracted because the occurrence probability is large It may be.

The event discriminating apparatus according to claim 2 is an event discriminating apparatus that discriminates an event that is a specific scene in sports video data, and includes a silhouette image extracting unit, a silhouette feature amount extracting unit, and a positional relationship feature vector. The configuration includes a generation unit, a conversion unit, and an event determination unit.

According to such a configuration, the event determination device distinguishes the non-background region from the background region and the non-background region by using the silhouette image extraction unit as a reference based on the predetermined pixel value of the pixel included in the image constituting the video data. A silhouette image in which the region is a silhouette region of a person is extracted. Subsequently, the event discriminating apparatus is a silhouette feature vector which is a vector indicating a feature amount of the silhouette region including at least coordinate information regarding the silhouette region included in the silhouette image extracted by the silhouette image extracting unit by the silhouette feature amount extracting unit. To extract. Then, the event discriminating apparatus obtains a positional relationship feature vector that is a vector obtained by quantifying the positional relationship of the person using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction unit by the positional relationship feature vector generation unit. Generate.
At this time, the event discriminating apparatus uses the silhouette feature amount extraction means to coordinate values in the coordinate space in which the space of the ground where the sport is performed and the coordinate values are associated in advance, and the silhouette region included in the silhouette image Each time, the color vector statistic is evaluated, and the team information that identifies the team to which the subject included in the silhouette area belongs is included in the silhouette feature vector and output. According to this, by outputting the coordinate value in the coordinate space and the belonging team information included in the silhouette feature vector, it is possible to determine where the subject exists in the ground space for each belonging team.
Further, at this time, the event discriminating device determines the number of subjects existing in each of the local regions divided from the ground space based on the coordinate values included in the silhouette feature vector and the belonging team information by the positional relationship feature vector generating means. Counting is performed for each team, and the counted result is included in the positional relationship feature vector and output. According to this, the event discriminating device outputs the result of counting the subjects for each team by the positional relationship feature vector generation means, including the positional relationship feature vector, and outputs the result for each team belonging to the local region. Determine the number of subjects.

The event discriminating apparatus according to claim 3 is the event discriminating apparatus according to claim 1 or 2 , wherein the silhouette feature amount extraction unit detects movement of a silhouette area included in the silhouette image according to elapsed time. The amount of movement of the silhouette region is output as a velocity vector included in the silhouette feature vector, and the positional relationship feature vector generation means obtains a set average value of the velocity vectors included in the silhouette feature vector. A result is included in the positional relationship feature vector and output.

According to such a configuration, the event discriminating apparatus makes it possible to determine the moving speed of the subject by outputting the speed vector including the silhouette vector by the silhouette feature amount extraction unit . Further, according to this configuration, the event discriminating device outputs the average value of the velocity vectors included in the positional relationship feature vector by the positional relationship feature vector generation means, and outputs the velocity ( Determine if you are moving at an average speed.

The event discriminating apparatus according to claim 4 , wherein the conversion unit is configured to convert the position of a weight matrix including a principal component group by principal component analysis into the position, according to any one of claims 1 to 3. The feature vector for discrimination is output by being applied to the related feature vector.

According to such a configuration, the event discriminating device converts the positional relationship feature vector into the discriminating feature vector by using the weight matrix composed of the principal component group obtained by the principal component analysis by the converting means. In other words, this conversion means replaces the vector indicating the characteristics of the positional relationship between the persons with a vector indicating what kind of event it corresponds to.

The event discriminating apparatus according to claim 5 is the event discriminating apparatus according to any one of claims 1 to 3, wherein the conversion unit uses a weight matrix based on a Fisher discrimination criterion as the positional relationship feature vector. By applying, the feature vector for discrimination is output.

According to such a configuration, the event discriminating apparatus converts the positional relationship feature vector into the discriminating feature vector by using the weight matrix based on the Fisher discriminating criterion by the converting means.

The event discriminating apparatus according to claim 6 is the event discriminating apparatus according to any one of claims 1 to 5, wherein the event discriminating means is the occurrence probability of the feature vector for discrimination for each event type. The event type with the highest probability value is output by referring to the probability model that models the above.

According to such a configuration, the event discriminating apparatus objectively discriminates the event type by referring to the probability model by the event discriminating means.

The event discriminating device according to claim 7 is the event discriminating device according to claim 6, wherein a Gaussian mixture model is used as the probability model.
According to this configuration, the event discriminating apparatus objectively discriminates the event type by referring to the Gaussian mixture model by the event discriminating means.

An event determination program according to claim 8, wherein a computer is used to determine an event that is a specific scene in sports video data, a silhouette image extraction unit, a silhouette feature amount extraction unit, a positional relationship feature vector generation unit, It is configured to function as a conversion unit that outputs a discrimination feature vector and an event discrimination unit.

According to such a configuration, the event determination program distinguishes the non-background region from the background region and the non-background region by using the silhouette image extraction unit with reference to the predetermined pixel value of the pixel included in the image constituting the video data. A silhouette image having a region as a person's silhouette region is extracted, and the feature amount of the silhouette region including at least coordinate information about the silhouette region included in the silhouette image extracted by the silhouette image extraction unit is extracted by the silhouette feature amount extraction unit. A silhouette feature vector, which is a vector to be shown, is extracted. Then, the event discrimination program uses a positional relationship feature vector generation unit to calculate a positional relationship feature vector that is a vector obtained by quantifying the positional relationship of a person using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction unit. Then, the converting means outputs a discrimination feature vector which is a vector obtained by multiplying the positional relation feature vector generated by the positional relation feature vector generation means by a preset matrix for weighting the feature amount. The event discriminating program uses the discriminating feature vector output from the converting unit by the event discriminating unit, and the occurrence probability of the event when the discriminating feature vector is obtained is previously modeled for each event type. The event is discriminated by referring to the probability model.
At this time, the event discriminating apparatus uses the silhouette feature amount extraction means to coordinate values in the coordinate space in which the space of the ground where the sport is performed and the coordinate values are associated in advance, and the silhouette region included in the silhouette image Each time, the color vector statistic is evaluated, and the team information that identifies the team to which the subject included in the silhouette area belongs is included in the silhouette feature vector and output. According to this, by outputting the coordinate value in the coordinate space and the belonging team information included in the silhouette feature vector, it is possible to determine where the subject exists in the ground space for each belonging team.
Further, at this time, the event discriminating device determines the number of subjects existing in each of the local regions divided from the ground space based on the coordinate values included in the silhouette feature vector and the belonging team information by the positional relationship feature vector generating means. Counting is performed for each team, and the counted result is included in the positional relationship feature vector and output. According to this, the event discriminating device outputs the result of counting the subjects for each team by the positional relationship feature vector generation means, including the positional relationship feature vector, and outputs the result for each team belonging to the local region. Determine the number of subjects.

The event determination program according to claim 9, in order to determine an event that is a specific scene in the video data for sports, the computer includes a silhouette image extraction unit, a silhouette feature amount extraction unit, a positional relationship feature vector generation unit, It is configured to function as a conversion unit that outputs a discrimination feature vector and an event discrimination unit.

According to such a configuration, the event determination program distinguishes the non-background region from the background region and the non-background region by using the silhouette image extraction unit with reference to the predetermined pixel value of the pixel included in the image constituting the video data. A silhouette image having a region as a person's silhouette region is extracted, and the feature amount of the silhouette region including at least coordinate information about the silhouette region included in the silhouette image extracted by the silhouette image extraction unit is extracted by the silhouette feature amount extraction unit. A silhouette feature vector, which is a vector to be shown, is extracted. Then, the event discrimination program uses a positional relationship feature vector generation unit to calculate a positional relationship feature vector that is a vector obtained by quantifying the positional relationship of a person using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction unit. Then, the converting means outputs a discrimination feature vector which is a vector obtained by multiplying the positional relation feature vector generated by the positional relation feature vector generation means by a preset matrix for weighting the feature amount. The event discriminating program uses the discriminating feature vector output from the converting unit by the event discriminating unit, and the occurrence probability of the event when the discriminating feature vector is obtained is previously modeled for each event type. The event is discriminated by referring to the probability model.
At this time, the event discriminating apparatus uses the silhouette feature amount extraction means to coordinate values in the coordinate space in which the space of the ground where the sport is performed and the coordinate values are associated in advance, and the silhouette region included in the silhouette image Each time, the color vector statistic is evaluated, and the team information that identifies the team to which the subject included in the silhouette area belongs is included in the silhouette feature vector and output. According to this, by outputting the coordinate value in the coordinate space and the belonging team information included in the silhouette feature vector, it is possible to determine where the subject exists in the ground space for each belonging team.
Further, at this time, the event discriminating device determines the number of subjects existing in each of the local regions divided from the ground space based on the coordinate values included in the silhouette feature vector and the belonging team information by the positional relationship feature vector generating means. Counting is performed for each team, and the counted result is included in the positional relationship feature vector and output. According to this, the event discriminating device outputs the result of counting the subjects for each team by the positional relationship feature vector generation means, including the positional relationship feature vector, and outputs the result for each team belonging to the local region. Determine the number of subjects.

The event determination program according to claim 10 is the event determination program according to claim 8 or 9, wherein the silhouette feature amount extraction unit detects movement of a silhouette region included in the silhouette image according to elapsed time. The amount of movement of the silhouette region is output as a velocity vector included in the silhouette feature vector, and the positional relationship feature vector generation means obtains a set average value of the velocity vectors included in the silhouette feature vector. A result is included in the positional relationship feature vector and output .

According to such a configuration, the event discriminating apparatus makes it possible to determine the moving speed of the subject by outputting the speed vector including the silhouette vector by the silhouette feature amount extraction unit . Further, according to this configuration, the event discriminating device outputs the average value of the velocity vectors included in the positional relationship feature vector by the positional relationship feature vector generation means, and outputs the velocity ( Determine if you are moving at an average speed.

According to the first and eighth aspects of the present invention, a silhouette image is extracted from a sport image, and then a silhouette feature vector, a positional relationship feature vector, and a discrimination feature vector are obtained. The event can be discriminated based on objective discrimination criteria.

In addition, by including the coordinate value in the coordinate space in the silhouette feature vector, the position of the silhouette region can be accurately determined, and an objective determination criterion can be adopted for determining the event.
Further, by including the count of subjects in the local region in the positional relationship feature vector, the number of subjects can be accurately determined, and an objective discrimination criterion can be employed for event discrimination.

According to the second and ninth aspects of the present invention, a silhouette image is extracted from a sport image, and then a silhouette feature vector, a positional relationship feature vector, and a discrimination feature vector are obtained. The event can be discriminated based on objective discrimination criteria.

In addition, by including the coordinate value in the coordinate space and the team information belonging to the silhouette feature vector, the position of the silhouette area and the team to which the team belongs can be accurately determined. Standards can be adopted.
Furthermore, by including the number of subjects for each team in the local area in the positional relationship feature vector, the number of subjects for each team can be accurately determined. Can be adopted.

According to the third and tenth aspects of the present invention, by including the velocity vector in the silhouette feature vector, the amount of movement of the silhouette region can be accurately determined, and an objective determination criterion is used for determining the event. Can be adopted. In addition, by including the velocity vector set average value in the positional relationship feature vector, it is possible to accurately determine how fast (average velocity) the set of subjects is moving, and for event determination Objective discrimination criteria can be employed.

According to the fourth aspect of the present invention, an objective discrimination can be performed for event discrimination by converting the positional relationship feature vector into a discrimination feature vector using a weight matrix composed of principal component groups by principal component analysis. Standards can be adopted.

According to the fifth aspect of the present invention, an objective discrimination criterion is adopted for event discrimination by converting the positional relationship feature vector into a discrimination feature vector using a weight matrix based on the Fisher discrimination criterion. be able to.

According to the sixth aspect of the invention, the event type can be objectively determined by using the probability model for determining the event.

According to the seventh aspect of the present invention, the event type can be objectively determined by using a Gaussian mixture model for determining the event.

Next, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
(Configuration of event discriminator)
FIG. 1 is a block diagram of an event discrimination device. As shown in FIG. 1, the event discriminating device 1 discriminates an event type for input video data (images per shot (input image I)). An extraction unit 5, a formation quantification unit (positional relationship feature vector generation unit) 7, a conversion unit 9, and an event determination unit 11 are provided.

  The silhouette extracting means 3 distinguishes the background area and the non-background area with respect to the pixel values of the pixels included in the image (input image I) for each shot constituting the video data, based on a predetermined pixel value. A silhouette image indicating the silhouette of a subject (here, a person or a player) that is a non-background region is extracted. Here, with reference to FIG. 6, the case where the silhouette image B is extracted from the input image I is demonstrated.

  Here, the silhouette extraction means 3 uses the background difference method to extract the silhouette image B from the input image I, and includes the background difference calculation means 3a when using this background difference method. As illustrated in FIG. 6, the non-background region and the background region of the input image I are classified, and a silhouette image B that is a binary image is output.

  In this case, the silhouette extraction means 3 is provided with a storage means (not shown), and a background image J where no subject is captured is stored in the storage means. Then, the silhouette extraction unit 3 generates the silhouette image B by evaluating the difference value for each pixel between the input image I and the background image J by the background difference calculation unit 3a.

  The calculation process in the background difference calculation means 3a will be specifically described. First, the pixel values of the input image I, the background image J, and the silhouette image B at the pixel position (X, Y) are set to I (X, Y), J (X, Y), and B (X, Y), respectively. The pixel values I (X, Y) and J (X, Y) are one-dimensional luminance values in the case of a monochrome image, and two-dimensional or more vector values in the case of a color image or a multiband image. (For example, a three-dimensional color vector value composed of red, green, and blue). Further, on the other hand, the pixel value B (X, Y) is assumed to be either 0 or 1, and the value 0 is set for the background pixel and the value 1 is set for the non-background pixel. To do.

  Then, for example, the background difference calculation means 3a can obtain the silhouette image B by performing a calculation using the following formula (1) with a threshold value b of 0 or more.

  Further, for example, the silhouette extracting means 3 may extract the silhouette image B by a chroma key method other than the background difference method. In this case, a hard chroma key can be applied using a commercially available general-purpose chroma key device. Returning to FIG.

Silhouette feature extraction means 5 extracts each silhouette regions (simply connected region) S _p included in the silhouette image B extracted by the silhouette extraction unit 3, with respect to the silhouette region (simply connected region) S _p, feature vector including at least coordinate information and extracts the (silhouette feature vector) g _p.

The subscript _p attached to the silhouette area Sp represents a number for distinguishing the single connected areas included in the silhouette image B from each other. The coordinate information is represented by coordinate values (actual coordinates) in a coordinate space in which a ground space where sports are performed and coordinate values are associated in advance.

Furthermore, the silhouette feature vector g _p, in addition to the actual coordinates in the coordinate information, the rate of the time derivative or time difference of the actual coordinates, estimates the number of persons included in the silhouette area S _p, the silhouette area S _p It is possible to include any 0 or more of the team identifiers (affiliation team information) that classify the teams to which the persons included in the group belong.

Here, with reference to FIGS. 2 and 3, a detailed configuration of the silhouette feature quantity extraction unit 5 will be described.
As shown in FIG. 2, the silhouette feature quantity extraction unit 5 receives the silhouette image B and the input image I and outputs a silhouette feature vector {g _p }, and includes a labeling unit 13, a center-of-gravity calculation unit 15, and the like. , Back projection means 17, speed estimation means 19, number estimation means 21, and color discrimination means 23.

The labeling means 13 distinguishes each single connected area from each other by labeling each single connected area included in the silhouette image B. Here, the labeling by the labeling means 13 is illustrated in FIG. As shown in FIG. 7, by labeling each single connected area (area represented by white in silhouette image B in FIG. 7) included in silhouette image B, each single connected area is distinguished from each other. cage, wrote simply connected region obtained by the labeling unit 13 and _{S p, p = 1,2, ......} , and P (the total number of simply connected region P is contained in the silhouette image B). Returning to FIG.

Gravity calculation means 15, the labeling unit 13, and requests the center of gravity (ξ _p, η _p) of the single connection region S _p that labeling by calculation. The center-of-gravity calculation means 15 performs a calculation using equation (2) shown below, seeking the center of gravity of the simply connected region _{S p (ξ p, η p} ).

The center of gravity here is a point representing the single connection region S _p existing on the image plane refers to the center of gravity of the plane figure. In addition to this the center of gravity, for example, circle circumscribing the simply connected region S _p, ellipse, such as the center of the polygonal it is possible used instead of the center of gravity.

The back projection means 17 determines which position (real coordinate) in the real space the centroid coordinates (ξ _p , η _p ) obtained by the centroid calculation means 15 correspond to. The backprojection means 17, and starting from the first optical principal point of the camera took the picture data, the first optical principal point and the center of gravity of the simply connected region _{S p (ξ p, η p} ) a half line passing through the r _p assuming, in advance virtually seeking curved Π and the half-line r _p and the intersection of intersection is being placed on the real space, the coordinates of the intersection actual coordinate _{(x p, y p, z} p) and.

  Note that the curved surface ridge can be, for example, a curved surface having a ground height of H / 2 (a flat surface when the ground surface is flat). Here, H is the average height of a person (sports player who can appear in the target image), for example, H = 1.8 [meter].

Speed estimating unit 19, based on the history of the actual coordinates obtained by the back projection unit _{17 (x p, y p,} z p), the actual speed _{(u p, v p, w} p) and calculates a. The speed estimation means 19, for example, the current actual coordinates (time _{t) (x p (t)} , y p (t), z p (t)) and the stored time ΔT past the actual coordinate _{(x p (t-ΔT), y p} (t-ΔT), based on the _{z p (t-ΔT))} , by a calculation using the equation (3) shown below, the actual speed _{(u p, v} p, w _p ) can be calculated.

Instead of using a back projection means 17 and the speed estimating means 19, by using the extended Kalman filter 25 as shown in FIG. 3, the center of gravity (ξ _p, η _p) real coordinates from _{(x p, y p, z} p) and actual speed _{(u p, v p, w} p) may estimate the configuration (silhouette feature extraction means 5A).
In this case, extended Kalman filter 25 silhouettes feature extraction means 5A, first, using Equation (4) shown below, that defines the state vector s _{p (t)} at time t.

Furthermore, extended Kalman filter 25 silhouettes feature extraction means 5A is a state transition to the state vector _s p (t + [Delta] T) from the state vector _s p (t) at time t at time t + [Delta] T, using equation (5) shown below Model.

In this formula _{(5), φ (t,} s p (t)) is the nominal value of the state transition from the state vector _s p at time t (t) to the state vector _s p (t + [Delta] T) at time t + [Delta] T (nominal On the other hand, ν (t) is a process noise added at time t.
For example, when modeling the nominal state transition as a constant velocity linear motion, the following formula (6) is used.

Further, the process noise, for example, average 0, assuming a Gaussian noise covariance matrix sigma _[nu, modeled using Equation (7) shown below.

この数式（７）において、Ｅ［・］は集合平均を、また、上付きのＴはベクトル又は行列の転置をそれぞれ表している。

In Equation (7), E [•] represents a set average, and superscript T represents a transposition of a vector or a matrix.

Then, extended Kalman filter 25 silhouettes feature extraction means 5A is a center of gravity of the simply connected region _{S p} at time _{t o p (t) = [} ξ p (t), η p (t)] at is ^T, for example, The observation model is defined using the following formula (8).

In this formula (8), h (t, s) represents the ideal (no noise) when converted from the state quantity _s p (t) to obtain a centroid _o p (t). For example, 'I assumed _p, said half-linear r' half lines r extending from the real coordinates corresponding to the state quantity s _{p (t)} as shown in FIG. 9 to the first optical principal point with a _{point p} intersects the image plane a series of conversion operation that the center of gravity _o p (t) can be a h (t, s).

On the other hand, in Equation (8), μ (t) is observation noise. For example, assuming that the observation noise μ (t) is Gaussian noise having an average of 0 and a covariance matrix Σ _μ , the observation noise μ (t) is modeled using the following formula (9).

Based on the models of the above formulas (4) to (9), the silhouette feature amount extraction unit 5A configures the extended Kalman filter 25 so that the real coordinates (x _p , y _p ) are obtained from the center of gravity (ξ _p , η _p ). , _{z p)} and the actual speed _{(u p,} v _p, it is possible to estimate the _{w p).} Returning to FIG.

Number estimating means 21 for each of the simply connected region S _p, it is to estimate the number (number) n _p of a person included in the region S _p. Here, the number estimation means 21 can estimate the number _{n p} based on the single connection region _{S p} area _{A p} and the actual coordinates _{(x p, y p, z} p) and. The area _{A p} of the connecting area _{S p,} when defined using Equation (10) shown below, using Equation (11) shown below, it is possible to estimate the number _{n p.}

この数式（１１）において、αは正の定数であり、単連結領域Ｓ_ｐに含まれる人数ｎ_ｐが得られるよう定めるものとする。

In this formula (11), alpha is a positive constant, and the provisions to persons n _p included in the simply connected region S _p is obtained.

Color determining means 23 evaluates the color of the input image I in the single consolidated area S _p, is to determine their teams of person corresponding to the region S _p. Here, the color determination unit 23 determines his team using color histograms gamma _p of the input image I in the single consolidated area S _{p (c).} Assuming that c represents a color vector, the color histogram γ _p (c) can be expressed as the following formula (12).

In Equation (12), δ _{c, I} is a Kronecker delta, and takes a value of 1 when c = I and a value of 0 when c ≠ I.
It is to be noted that, as football or basketball, in sports to carry out the competition is divided into coat left and right, the identifier k _p value of the team that the Court left the player's side -1, the value of the identifier k _p of the team to the coat right side and their own half +1.

The color discriminating means 23 holds a color template registered in advance for each type of uniform ψ in a storage means not shown in advance. For example, the color histogram Γ _ψ (c) can be used as a template of the uniform type ψ.

Subsequently, the color discriminating means 23 discriminates which uniform type ψ the observed color histogram γ _p (c) is similar to by the calculation using the following formula (13), and uses the result as ψ And

In Equation (13), {} represents a distance between histograms, and for example, a Bhattacharrya distance can be used.
The color discrimination means 23 outputs k _p = −1 if the obtained uniform type Ψ is that of the left team (in soccer, the left team field player or the left team goalkeeper uniform), and the uniform type Ψ Is the right team (in soccer, the right team field player or the right team goalkeeper uniform), k _p = + 1 is output.

Further, the color determination unit 23 may consider the number of people n _p obtained by the number of people estimation unit 21. For example, when the number n _p is about 1 (for example, 0.5 to less than 1.5), k _p = −1 or k _p = + 1 is output as described above, and the number n _p is about 1 Otherwise, k _p = 0 can be output. In this color discriminating means 23, not dare team classify simply connected region S _p which overlap a plurality of persons, for example, be treated as two teams of people in the simply connected region S _p is present at a fifty-fifty ratio of Can do. Returning to FIG.

Silhouette feature extraction means 5, the actual coordinates obtained as described above _{(x p, y p, z} p), the actual speed _{(u p, v p, w} p), one of the estimated number _{n p} and the team identifier _{k p} A vector obtained by further arranging vectors composed of one or more components for the number P of the single connected regions is output to the formation quantifying means 7 as a silhouette feature vector σ.

  The formation quantification unit 7 obtains a formation feature vector f which is a vector obtained by quantifying the formation features indicating the positional relationship of the person based on the silhouette feature vector σ obtained by the silhouette feature amount extraction unit 5.

For example, the formation feature vector f lists the number of persons existing in each of the partial area group {D _q } (q is a number for distinguishing the partial areas from each other) obtained by dividing the ground (stadium). can do.

Furthermore, the formation feature vector f may be a list of the number of players belonging to a specific team existing in each of the partial region groups {D _q } obtained by dividing the ground (stadium). In the following description, the number of players belonging to the team κ (where the left team is κ = −1 and the right team is κ = + 1) existing in the partial region D _q is described as N (q, k). .

For the partial region D _q , for example, as shown in FIG. 10, a stadium (for example, a soccer court in soccer, which may include some off-field regions) is divided into Q grids, and the q th ( for example, the main scanning from left to right, a partial area of the order of raster scanning by-scan from the front to the back) can be placed and D _q. At this time, the number present in the partial region D _q N (q, kappa) can be counted using the following equation (14).

However, in this equation (14), d (k _p , κ) can be defined as a function like the following equation (15).

Incidentally, formation quantification means 7, of the simply connected region group {S _p}, has the function of extracting only the S _p team identifier k _p in silhouette feature vectors σ matches the team κ counting target ing. Further, the formation quantification means 7 uses the definition of d (k, κ), and when the team identifier k _p included in the silhouette feature vector σ takes a value of 0 (in the case of a plurality of people), Counting was performed with a weight of 1/2.

Furthermore, formation quantification means 7, it is possible to handle a partial region D _q as fuzzy sets. In this case, the formation quantification means 7 sets the membership function in the coordinates (x, y) of the partial region D _q to m _q (x, y), and the number of people N (q, κ) existing in the partial region D _q . ) Can be counted using the following equation (16).

Here, an example of a method for setting the partial region _Dq will be described. For example, as in the partial region D _q shown in FIG. 11, when the ground (stadium) is a plane, the membership function m centered on coordinates (G _x (q), G _y (q)) on the plane. _{With f} (x, y), a fuzzy set can be defined using the following equation (17).

For example, partial lateral E _x, the depth E _y size of the area (the stadium) transverse delta _x pieces, by total Q = Δ _x × Δ _y-number of fuzzy sets are arranged in the depth direction delta _y-number of grid-like When covering with the area _| region Dq, each parameter represented using Numerical formula (18) shown next can be used.

However, in this formula (18), qmodΔ _x represents the remainder when divided by q in delta _x, x sandwiched between opposite brackets and "represents the maximum integer not greater than x.

For example, the area (E _x = 131.25, E _y = 85.00) obtained by enlarging a 105 m × 68 m soccer court to 125% on each side (Δ _x = 5 in the horizontal direction and Δ _y = 5 in the depth direction) Is covered by the partial region D _q by the fuzzy set of Q = 25, it can be expressed using the following formula (19).

  When this numerical formula (19) is put together, it can be expressed as the following numerical formula (20).

以下、説明を簡単にするために、部分領域Ｄ_ｑのインデクスｑを、次に示す数式（２１）を用いて、（ｑ_ｘ，ｑ_ｙ）の組み合わせで表すこととする。

Hereinafter, in order to simplify the explanation, the index q of the partial region D _q is represented by a combination of (q _x , q _y ) using the following formula (21).

However, in this equation (21), when _x is an odd number, q _x is an integer, and when _x is an even number, q _x is a half integer, and in any case q _x is It shall take the value of-((DELTA) _x- 1) / 2 or more and + ((DELTA) _x- 1) / 2 or less.

In addition, q _y is an integer when Δ _y is an odd number, and q _y is a half integer when Δ _y is an even number. In either case, q _y is − (Δ _y −1). The value is not less than / 2 and not more than + (Δ _y −1) / 2.

Regarding the relationship between the combination of (q _x , q _y ) and q, FIG. 12 shows the case of Δ _x = 5 and Δ _y = 5, and FIG. 13 shows the case of Δ _x = 4 and Δ _y = 4. Each is illustrated. Although not shown, delta _x or if odd cutlet delta _y is an even number, delta _x is may be a case even number cutlet delta _y is an odd number.

Further, the formation quantification means 7 uses the average velocity vector (U, V, W) of all players and the average of players belonging to a specific team κ (κ∈ {−1, + 1}) as components of the formation feature vector f. Some or all of the velocity vectors (U _κ , V _κ , W _κ ) can be included. In this case, the average velocity vector (U _κ , V _κ , W _κ ) is defined using the following formula (22) or formula (23).

The formation quantification means 7 determines N (q, k), the average velocity vector (U, V, W) of all the players, and the identification of the number of players belonging to the team κ existing in the partial region D _q . A formation feature vector f is a vector obtained by collecting a part or all of the average speed vectors (U _κ , V _κ , W _κ ) of the players belonging to the team κ (κε {−1, + 1}).
For example, the formation feature vector f may be expressed as the following formula (24) using all the above, or may be expressed as the following formula (25) using the above part. Can do.

  The conversion means 9 performs an operation of multiplying the formation feature vector f generated by the formation quantification means 7 by a preset matrix (weight matrix described later) for weighting the feature quantity, and the operation result is used as a discrimination feature vector χ. Is output as

  Here, the conversion means 9 calculates the product of the weight matrix Φ and the formation feature vector f, and outputs the calculation result as a discrimination feature vector χ. The distinguishing feature vector χ can be expressed using the following equation (26).

For example, the weight matrix Φ can be determined by performing principal component analysis on the formation feature vector group {f _i } obtained from the image input group {I _i } for learning in advance.
In this case, the learning image input group {I _i } (i∈ {1, 2,..., T _L }) is processed by the silhouette extraction means 3, the silhouette feature quantity extraction means 5, and the formation quantification means 7. , A formation feature vector group {f _i } is obtained.

Subsequently, the covariance matrix Σ _f of the formation feature vector group {f _i } is obtained using the following formula (27).

Further, the eigenvectors of the covariance matrix sigma _chi, obtaining the M from the direction its eigenvalues is greater. Let the m-th (m∈ {1, 2,..., M}) eigenvector be β _m . Finally, a weight matrix Φ is defined using a matrix in which transpositions of eigenvectors β _m are arranged in the row direction, for example, using the following formula (28).

  The weight matrix Φ is a Fisher weight map (references: Shinohara, Otsu: “Recognition of facial expressions from face images using the Fisher weight map”, IEICE Technical Report PRMU 2003-269, vol. 103, no. 737, pp. 79-84, 2003).

  Here, the procedure for determining the weight matrix Φ used in the conversion means 9 will be described with reference to FIG. In addition, what is shown in FIG. 4 is a device in which means for determining a weight matrix Φ (a classification means 27, a variance calculation means 29, and a generalized eigenvalue problem calculation means 31) are added to the event determination apparatus 1 shown in FIG. It is assumed that the event discriminating apparatus 1A. And the thing similar to the structure of the event discrimination | determination apparatus 1 attaches | subjects the same code | symbol, and abbreviate | omits the description.

First, as shown in FIG. 4, the event discriminating apparatus 1A inputs learning video data (learning image input group {I _i }) to the silhouette extracting means 3, and this silhouette extracting means 3 and silhouette feature extraction. Processing is performed by means 2 to obtain a formation feature vector group {f _i }.

Based on the event type correct answer data input (input event type correct answer data group {e _i }) and the formation feature vector group {f _i }, the classification unit 27 determines the formation feature vector group {f _i } for each event type e. The formation feature vector set F _e for each event type e is obtained using the following formula (29).

Variance calculating means 29, using the following equation (30), and requests the variance sigma _B in the class (between the same event) dispersion sigma _W and between classes (between different events).

In Equation (30), | F | represents the number of elements in the set F.
The generalized eigenvalue problem calculation means 31 determines the weight matrix Φ by solving the generalized eigenvalue problem based on the Fisher's criterion for solving the following equation (31).

In Equation (31), β is an eigenvector and λ is its eigenvalue. In this case, when the m-th eigenvector β counted from the larger eigenvalue λ is β _m , the weight matrix Φ can be defined by the following equation (32).

Also, the weight matrix Φ can be set without depending on the principal component analysis and the Fisher's criterion.
For example, a weight matrix that facilitates event identification by weighting and combining components of the formation feature vector f can be set by the designer's subjectivity. For example, players in the partial area D _q corresponding to the four corners of the ground (stadium) (indexes q of the partial areas at the four corners are the left front q _A , the right front q _B , the left back q _C, and the right back q _D ) A weight vector β _m1 that takes a linear combination of corresponding components of N (q, k) so as to obtain a simple average of the number of people can be defined using the following equations (33), (34), and (35). .

In Equation (35), weight vectors β _m2 and β _m3 with different signs are defined for each place of the ground (stadium).
Then, the weight matrix Φ can be defined by combining these as in the following formula (36).

Furthermore, the weight vector β _● obtained (or subjectively designed) by various methods such as Expression (27), Expression (32), Expression (33) to Expression (35) may be freely combined.

  Further, the conversion means 9 may obtain the discrimination feature vector χ using the following mathematical formula (37) by the nonlinear transformation φ of the formation feature vector f.

In this case, for example, when the conversion unit 9 sets the n-th component of the N-dimensional formation feature vector f as f _n and the m-th component of the M-dimensional discrimination feature vector as χ _m , Using it, it is possible to perform non-linear transformation by combining the secondary form and the primary form of f.

In Equation (38), a _● and b _● are the third and second floor tensors, respectively, and each component is a constant. Returning to FIG.

  The event discriminating unit 11 discriminates the event type by referring to the discrimination feature vector χ obtained by the conversion unit 9 and the probability model obtained by modeling the occurrence probability of the event type in which the event is classified in advance. The result (discrimination result E) is output.

  Here, the event discriminating means 11 refers to a probability model obtained by modeling the occurrence probability P (e | χ) of the event type e when the discriminating feature vector χ is given, so that the most probable event type E Is obtained and output using the following mathematical formula (39). Here, referring to a probability model and determining an event means performing an operation of selecting a specific event from the magnitude of the probability value of the occurrence probability of each event and the distribution shape.

  In this mathematical formula (39), the occurrence probability P (e | χ) may be represented by, for example, a Gaussian mixture model for each event type e using the following mathematical formula (40).

In this equation (40), K _e is the number of Gaussian mixture models for the event type e, we _{, k} is the weight for the kth Gaussian distribution constituting the Gaussian mixture model, and μ _{e, k} is the above-mentioned number. An average vector of k Gaussian distributions, and Σ _{e, k} represent a covariance matrix of the kth Gaussian distribution, respectively.

The mixing speed K _e may be a different constant depending event type e, it may be as a common constant for all event types (e.g., 5).
The weights w _{e, k} , the mean vector μ _{e, k} and the covariance matrix Σ _{e, k} can be determined based on the video data for learning by the following procedure. Hereinafter, the weights w _{e, k} are represented by the product of the parameter ω _{e, k} and the parameter p _e using the following formula (41).

Here, a detailed configuration of the event determination unit 11 will be described with reference to FIG. As shown in FIG. 5, the event determination unit 11 includes an EM algorithm execution unit 33, a prior probability calculation unit 35, and a multiplication unit 37. Note that it is assumed that a discrimination feature vector set X _e for each event type _e is input to the event discrimination means 11.
The discrimination feature vector set X _e for each event type e is obtained by converting all the formation feature vectors f constituting the formation feature vector set F _e for each event type e into a discrimination feature vector χ by the conversion means 9. It was created by.

For example, the conversion means 9, by a linear transformation by weighting matrix [Phi, when creating a decision feature vector set X _e for each event type e uses the formula (42) shown below.

The EM algorithm executing means 33 uses the following equation (43) so that the probability density function p (χ | e) defined by the following equation approximates the density distribution of the set X _e , and the parameters ω _{e, k} , μ _{e, k} and Σ _{e, k} are determined.

In this number (43), the parameters ω _{e, k} , μ _{e, k} and Σ _{e, k} can be obtained by an EM (Expectation-Maximization) algorithm.
Prior probability calculation unit 10 uses the formula (44) shown below, is intended to define the parameters p _e.

The multiplication means 11 obtains the weights we _{, k} for the kth Gaussian distribution of the Gaussian mixture model of the event type e by executing the calculation of the equation (41).

The event discriminating means 11 performs the operations of the equations (41) to (44) for all event types e, whereby the weights we _{, k} of the Gaussian mixture model group shown in the equation (40), the average vector μ _{e, k} and covariance matrix Σ _{e, k} can all be determined.

  According to the event discriminating apparatus 1, after extracting the silhouette image B from the sports input image I, the silhouette feature vector g, the formation feature vector f, and the discriminating feature vector χ are obtained. The event can be discriminated based on objective discrimination criteria.

(Operation of event discriminator)
Next, with reference to FIG. 14, the general operation of the event determination apparatus 1 will be described (see FIG. 1 as appropriate).
First, the event discriminating apparatus 1 extracts a silhouette image by the silhouette image extraction means 3 (step S1), extracts a silhouette area (single connected area) from the silhouette image by the silhouette feature amount extraction means 5, and obtains a silhouette feature vector. Is extracted (step S2).

  Subsequently, the event discriminating apparatus 1 generates a formation feature vector by the formation quantifying unit 7 (step S3), and outputs a discrimination feature vector by applying a weighting matrix to the formation feature vector by the conversion unit 9. (Step S4). Then, the event discriminating apparatus 1 discriminates the event type by the event discriminating means 11 based on the discriminating feature vector (step S5).

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, in the present embodiment, the event determination device 1 has been described. However, it is also possible to configure as an event determination program described in a computer language so that the processing of each component of the device 1 can be realized. In this case, the same effect can be obtained.

It is a block diagram of the event discriminating device concerning the embodiment of the present invention. It is a block diagram which shows the detailed structure of a silhouette feature-value extraction means. It is a block diagram which shows the other detailed structure (structure in the case of using an extended Kalman filter) of a silhouette feature-value extraction means. It is a block diagram of an event discriminating device to which means for obtaining a weight matrix by learning is added. It is a block diagram which shows the detailed structure (The means to obtain the parameter of a Gaussian mixture model by learning) of an event discrimination means. It is a figure explaining operation | movement of the silhouette extraction means by a background difference method. It is a figure explaining operation | movement of a labeling means. It is a figure explaining operation | movement of a back projection means. It is a figure explaining the operation | movement of observation in the observation model of an extended Kalman filter. It is a figure which shows an example of arrangement | positioning of a partial area | region. It is a figure which shows an example of the membership function in the case of making a partial area | region into a fuzzy set. _(Q x, _{q y)} is a diagram showing an example of the relationship between the combination and q of. (Q _{x, q y)} is a diagram showing another example of the relationship between the combination and q of. It is the flowchart which showed the general | schematic operation | movement of the whole event determination apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Event discrimination device 3 Silhouette extraction means 3a Background difference calculation means 5 Silhouette feature amount extraction means 7 Formation quantification means (positional relation feature vector generation means)
DESCRIPTION OF SYMBOLS 9 Conversion means 11 Event discrimination means 13 Labeling means 15 Center of gravity calculation means 17 Back projection means 19 Speed estimation means 21 Number of persons estimation means 23 Color discrimination means 25 Extended Kalman filter 27 Sorting means 29 Variance calculation means 31 Generalized eigenvalue problem calculation means 33 EM algorithm Execution means 35 Prior probability calculation means 37 Multiplication means

Claims (10)

An event discriminating device for discriminating an event that is a specific scene in video data for sports,
A silhouette image that distinguishes a background region from a non-background region on the basis of a predetermined pixel value of pixels included in the image constituting the video data and extracts a silhouette image using the non-background region as a silhouette region of a person Extraction means;
Silhouette feature amount extraction means for extracting a silhouette feature vector, which is a vector indicating the feature amount of the silhouette area including at least coordinate information about the silhouette area included in the silhouette image extracted by the silhouette image extraction means;
Using the coordinate information of the silhouette feature vector extracted by the silhouette feature quantity extracting means, a positional relationship feature vector generating means for generating a positional relationship feature vector which is a vector quantifying the positional relationship of the person;
Conversion means for outputting a discrimination feature vector which is a vector obtained by multiplying the positional relation feature vector generated by the positional relation feature vector generation means by a preset matrix for weighting the feature amount;
By using the discrimination feature vector output by the conversion means, and referring to a probability model that is modeled in advance for each event type, for the occurrence probability of the event when the discrimination feature vector is obtained, Event discriminating means for discriminating an event;
Equipped with a,
The silhouette feature amount extraction unit includes, as the coordinate information, a coordinate value on a coordinate space in which a space of the ground where the sport is performed and a coordinate value are associated in advance in the silhouette feature vector,
The positional relationship feature vector generation means counts persons existing in each of the local areas obtained by dividing the ground space based on the coordinate values included in the silhouette feature vector, and the counted result is the positional relationship feature vector. An event discriminating apparatus characterized in that it is included and output . An event discriminating device for discriminating an event that is a specific scene in video data for sports,
A silhouette image that distinguishes a background region from a non-background region on the basis of a predetermined pixel value of pixels included in the image constituting the video data and extracts a silhouette image using the non-background region as a silhouette region of a person Extraction means;
Silhouette feature amount extraction means for extracting a silhouette feature vector, which is a vector indicating the feature amount of the silhouette area including at least coordinate information about the silhouette area included in the silhouette image extracted by the silhouette image extraction means;
Using the coordinate information of the silhouette feature vector extracted by the silhouette feature quantity extracting means, a positional relationship feature vector generating means for generating a positional relationship feature vector which is a vector quantifying the positional relationship of the person;
Conversion means for outputting a discrimination feature vector which is a vector obtained by multiplying the positional relation feature vector generated by the positional relation feature vector generation means by a preset matrix for weighting the feature amount;
By using the discrimination feature vector output by the conversion means, and referring to a probability model that is modeled in advance for each event type, for the occurrence probability of the event when the discrimination feature vector is obtained, Event discriminating means for discriminating an event;
With
The silhouette feature amount extraction unit includes coordinate values on a coordinate space in which a space of the ground where the sport is performed and coordinate values are associated in advance;
For each silhouette region included in the silhouette image, the color vector statistics are evaluated, and the team information identifying the team to which the person included in the silhouette region belongs is included in the silhouette feature vector and output. ,
The positional relationship feature vector generation means counts the number of persons existing in each local area divided into the ground space for each team based on the coordinate values and team information included in the silhouette feature vector. , features and be Louis vent discriminating device to output, including the count result to the positional relationship feature vector. The silhouette feature amount extraction means detects the movement of the silhouette region included in the silhouette image according to the elapsed time, and outputs the amount of movement of the silhouette region as a velocity vector included in the silhouette feature vector,
The positional relationship between the feature vector generating unit, the calculated a set average value of the velocity vectors included in silhouette feature vector, according to claim 1 or claim, characterized in that the output results included the determined on the positional relationship feature vector Item 3. The event discriminating apparatus according to Item 2 . And the converting means, by multiplying the weight matrix consisting of the main component group by the principal component analysis on the positional relationship feature vector, any of claims 1 to 3 and outputs the decision feature vector The event discrimination device according to one item. And the converting means, by multiplying the weight matrix based on Fisher discriminant reference to the positional relationship feature vector, according to any one of claims 1 to 3 and outputs the decision feature vector Event discriminator. The event determination unit outputs an event type having the highest probability value with reference to a probability model obtained by modeling the occurrence probability of the determination feature vector for each event type. event discrimination apparatus according to any one of 5. The event discriminating apparatus according to claim 6 , wherein a Gaussian mixture model is used as the probability model. In order to identify events that are specific scenes in sports video data,
A silhouette image that distinguishes a background region from a non-background region on the basis of a predetermined pixel value of pixels included in the image constituting the video data and extracts a silhouette image using the non-background region as a silhouette region of a person Extraction means,
Silhouette feature quantity extraction means for extracting a silhouette feature vector, which is a vector indicating the feature quantity of the silhouette area including at least coordinate information about the silhouette area included in the silhouette image extracted by the silhouette image extraction means;
Positional relationship feature vector generation means for generating a positional relationship feature vector, which is a vector obtained by quantifying the positional relationship of the person, using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction means;
Conversion means for outputting a feature vector for discrimination, which is a vector obtained by multiplying the formation feature vector generated by the formation quantification means by a preset matrix for weighting the feature amount;
By using the discrimination feature vector output by the conversion means, and referring to a probability model that is modeled in advance for each event type, for the occurrence probability of the event when the discrimination feature vector is obtained, Event discriminating means for discriminating an event,
To function as,
The silhouette feature amount extraction unit includes, as the coordinate information, a coordinate value on a coordinate space in which a space of the ground where the sport is performed and a coordinate value are associated in advance in the silhouette feature vector,
The positional relationship feature vector generation means counts persons existing in each of the local areas obtained by dividing the ground space based on the coordinate values included in the silhouette feature vector, and the counted result is the positional relationship feature vector. Event discriminating program characterized in that it is included and output . In order to identify events that are specific scenes in sports video data,
A silhouette image that distinguishes a background region from a non-background region on the basis of a predetermined pixel value of pixels included in the image constituting the video data and extracts a silhouette image using the non-background region as a silhouette region of a person Extraction means,
Silhouette feature quantity extraction means for extracting a silhouette feature vector, which is a vector indicating the feature quantity of the silhouette area including at least coordinate information about the silhouette area included in the silhouette image extracted by the silhouette image extraction means;
Positional relationship feature vector generation means for generating a positional relationship feature vector, which is a vector obtained by quantifying the positional relationship of the person, using the coordinate information of the silhouette feature vector extracted by the silhouette feature amount extraction means;
Conversion means for outputting a feature vector for discrimination, which is a vector obtained by multiplying the formation feature vector generated by the formation quantification means by a preset matrix for weighting the feature amount;
By using the discrimination feature vector output by the conversion means, and referring to a probability model that is modeled in advance for each event type, for the occurrence probability of the event when the discrimination feature vector is obtained, Event discriminating means for discriminating an event,
To function as,
The silhouette feature amount extraction unit includes coordinate values on a coordinate space in which a space of the ground where the sport is performed and coordinate values are associated in advance;
For each silhouette region included in the silhouette image, color vector statistics are evaluated, and the team information that identifies the team to which the person included in the silhouette region belongs is included in the silhouette feature vector and output. ,
The positional relationship feature vector generation means counts the number of persons existing in each local area divided into the ground space for each team based on the coordinate values and team information included in the silhouette feature vector. An event discriminating program for outputting the counted result by including it in the positional relationship feature vector . The silhouette feature amount extraction means detects the movement of the silhouette region included in the silhouette image according to the elapsed time, and outputs the amount of movement of the silhouette region as a velocity vector included in the silhouette feature vector,
The positional relationship between the feature vector generating unit, the calculated a set average value of the velocity vectors included in silhouette feature vector, according to claim 8 or claim, characterized in that the output results included the determined on the positional relationship feature vector Item 10. The event discrimination program according to Item 9 .

Images