JP7437652B2 - Ball game video analysis device, ball game video analysis system, ball game video analysis method, and computer program - Google Patents

Description

The present disclosure relates to a ball game video analysis device, a ball game video analysis system, a ball game video analysis method, and a computer program.

BACKGROUND ART Conventionally, "Data Volley" has been commercially available as volleyball analysis software, and a technique is known that converts the status of players on a team into data based on the subjective judgment of an analyst familiar with this software.

In recent years, cameras have been installed in each of the four corners overlooking the court, and these four cameras are synchronized to capture footage of the play, track the movement of the ball, and detect the ball, without relying on analysts' operations. There is a known technology that obtains three-dimensional positional information of a player and simultaneously outputs analytical data regarding the player's uniform number and the type of action included in the play (serve, reception, dig, toss, attack, block) (Patent Document 1).

International Publication No. 2019/225415

Incidentally, a video of a person practicing a ball game action (for example, a volleyball serve) is taken using a nearby camera, and the video is later played back to be used for practice. However, conventionally, videos are searched based on the shooting date and time and thumbnail of the video, and the target video and comparison video are identified from among multiple videos of the same action, and the proficiency of the action and the form are searched. It is difficult to understand areas for improvement.

The purpose of this disclosure is to identify target videos and comparison videos from among multiple videos of the actions of people playing ball games, and to easily understand the proficiency level of actions and areas for improvement in form. The goal is to provide technology.

A ball game video analysis device according to the present disclosure provides trajectory information indicating a movement trajectory of the moving object based on a first video and a second video that are taken from different positions of a person's action toward a moving object used in a ball game. a trajectory analysis unit that generates a trajectory analysis unit; a record management unit that manages a plurality of recorded information, which is information indicating a correspondence relationship between the first moving image, the second moving image, and the trajectory information; Displaying a UI (User Interface) for selecting first recorded information and second recorded information from among them based on the trajectory information; and a display control unit that displays the first moving image or the second moving image and the first moving image or the second moving image that have a corresponding relationship in the second recorded information .

A ball game video analysis system according to the present disclosure is a ball game video analysis system including a first mobile terminal with a camera, a second mobile terminal with a camera, a ball game video analysis device, and a display terminal, The first mobile terminal captures the action of the person against a moving object used in a ball game from a first position to generate a first video, and the second mobile terminal captures the action of the person from a first position. The ball game video analysis device generates trajectory information indicating a trajectory of the moving body based on the first video and the second video, and A plurality of pieces of recorded information are managed, which are information indicating a correspondence relationship between a first moving image and a second moving image, and the trajectory information, and first recorded information is selected from among the plurality of recorded information based on the trajectory information. and displaying on the display terminal a UI for selecting second recorded information , and displaying the first moving image or the second moving image in a corresponding relationship in the first recorded information selected through the UI, and the second moving image. The first moving image or the second moving image having a corresponding relationship in the recorded information of No. 2 is displayed on the display terminal.

In the ball game video analysis method according to the present disclosure, a device calculates the movement trajectory of a moving object used in a ball game based on a first video and a second video that are taken from different positions of a person's action on a moving object. generates trajectory information indicating the trajectory information, manages a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information, and selects the trajectory information from among the plurality of recorded information. displaying a UI that allows selection of first recorded information and second recorded information based on the first recorded information and the second recorded information, the first moving image or the second moving image having a corresponding relationship in the first recorded information selected through the UI; and the first moving image or the second moving image that have a corresponding relationship in the second recorded information .

A computer program according to the present disclosure generates trajectory information indicating a movement trajectory of a moving object used in a ball game, based on a first video and a second video that are taken from different positions of a person's action on a moving object. and manages a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information, and selects a first moving image from among the plurality of recorded information based on the trajectory information . displays a UI that allows selection of recorded information and second recorded information , and displays the first moving image or the second moving image that has a corresponding relationship in the first recorded information selected through the UI, and the second moving image. The first moving image or the second moving image that has a corresponding relationship in the recorded information is displayed.

Note that these comprehensive or specific aspects may be realized by a system, a device, a method, an integrated circuit, a computer program, or a recording medium, and any of the systems, devices, methods, integrated circuits, computer programs, and recording media may be implemented. It may be realized by any combination.

According to the present disclosure, it is possible to identify a target video and a comparison video from among multiple videos of the actions of a person playing a ball game, and easily understand the proficiency level of the action, points for improvement in form, etc. .

A diagram showing a configuration example of a ball game video analysis system according to Embodiment 1. Diagram to explain how to locate the ball Block diagram showing a configuration example of a ball game video analysis device according to Embodiment 1 Diagram for explaining how to identify a reference frame image Flowchart showing an example of preliminary work Sequence chart showing an example of processing related to preliminary work Sequence chart showing an example of processing related to preliminary work when an operation to change the reference frame image is performed in FIG. 6 Diagram showing a detailed example of the reference frame confirmation UI (User Interface) Diagram showing a detailed example of the reference frame change UI Diagram showing an example of displaying analysis results A diagram showing a configuration example of a ball game video analysis system according to Embodiment 2 Block diagram showing a configuration example of a ball game video analysis device according to Embodiment 2 Block diagram showing a configuration example of a display terminal according to Embodiment 2 Diagram for explaining an example of a posture model Diagram to explain the sub-actions that make up a jumping serve Diagram showing a display example of the video comparison UI Diagram showing a display example of the sub-action video comparison UI A diagram showing an example of a posture comparison UI when posture models are displayed side by side. A diagram showing an example of a posture comparison UI when posture models are displayed overlapping each other. Diagram showing an example of UI for comparing sub-action postures Diagram for explaining an example of selecting a posture model in the posture comparison UI A diagram showing the hardware configuration of a computer that implements the functions of the ball game video analysis device according to the present disclosure through a program. A diagram showing a hardware configuration of a computer that implements the functions of a first mobile terminal, a second mobile terminal, and a display terminal according to the present disclosure by a program.

Embodiments of the present disclosure will be described in detail below with appropriate reference to the drawings. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters and redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

(Embodiment 1)
<Configuration of ball game video analysis system>
FIG. 1 is a diagram showing an example of the configuration of a ball game video analysis system according to the present embodiment.

The ball game video analysis system 10 is a system that analyzes a video shot of a ball game and analyzes the trajectory of a moving object used in the ball game. In this embodiment, volleyball, which is one of the ball games, will be explained as an example. However, the ball game video analysis system 10 is applicable not only to volleyball but also to various ball games such as soccer, baseball, table tennis, basketball, tennis, rugby, American football, lacrosse, badminton, or ice hockey. Further, the moving object targeted by the ball game video analysis system 10 is typically a ball. However, the moving object targeted by the ball game video analysis system 10 is not limited to a ball, and may be of a shape that does not fit the concept of a "ball", such as a badminton shuttle or an ice hockey puck, for example.

The ball game video analysis system 10 includes a first mobile terminal 20A, a second mobile terminal 20B, a display terminal 30, and a ball game video analysis device 100. The first mobile terminal 20A, the second mobile terminal 20B, the display terminal 30, and the ball game video analysis device 100 can send and receive information to and from each other through the communication network N. The communication network N is configured by, for example, a wireless communication network (wireless LAN (Local Area Network), 4G, 5G, etc.), an Internet network, or a combination thereof.

The first mobile terminal 20A and the second mobile terminal 20B are wireless communication terminals equipped with a camera. Examples of the first mobile terminal 20A and the second mobile terminal 20B are a mobile phone, a smartphone, a tablet, a mobile PC, or the like.

The first mobile terminal 20A and the second mobile terminal 20B photograph the actions of a person (hereinafter referred to as a "player") playing a ball game. In this embodiment, a volleyball serve, which is one of the player's actions, will be explained as an example. However, the player's action is not limited to a volleyball serve, but may also be a dig, a reception, a toss, an attack, a block, etc. Alternatively, the action of a person playing a ball game may be a specific action in the above sports other than volleyball.

The first mobile terminal 20A and the second mobile terminal 20B, for example, take videos of a player's serve practice from different directions, and generate a first video 201A and a second video 201B, respectively. For example, as shown in FIG. 1, the first mobile terminal 20A photographs the player from the rear right, and the second mobile terminal 20B photographs the player from the rear left.

The first mobile terminal 20A and the second mobile terminal 20B transmit the generated first video 201A and second video 201B to the ball game video analysis device 100 via the communication network N, respectively.

The display terminal 30 displays the analysis results by the ball game video analysis device 100. Note that a display example of the analysis results will be described later (see FIG. 10). Examples of the display terminal 30 are a mobile phone, a smartphone, a tablet, or a mobile PC. The display terminal 30 may be read as a third mobile terminal. Although a configuration in which the display terminal 30 for displaying the analysis results is separately provided has been illustrated, either the first mobile terminal 20A or the second mobile terminal 20B may be used as a terminal for displaying the analysis results.

The ball game video analysis device 100 analyzes the movement trajectory of the ball in three-dimensional space based on the first video 201A and second video 201B received from the first mobile terminal 20A and the second mobile terminal 20B, respectively. . The movement trajectory of the ball in the three-dimensional space can be generated by arranging the three-dimensional coordinates of the ball in time series.

As shown in FIG. 2, the three-dimensional coordinates of the ball at time t can be calculated from the frame image F1 at time t in the first video 201A and the frame image F2 at time t in the second video 201B. . That is, as shown in FIG. 2, a vector V1 extending from the position U1 of (the camera of) the first mobile terminal 20A to the position B1 of the ball in the frame image F1 and the position of (the camera of the second mobile terminal 20B) The intersection with the vector V2 extending from U2 to the ball position B2 in the frame image F2 becomes the three-dimensional coordinate Pa of the ball.

Here, the time code (time information) that the first mobile terminal 20A gives to each frame image of the first video 201A, and the time code that the second mobile terminal 20B gives to each frame image of the second video 201B. If the time code is completely accurate, the three-dimensional coordinates of the ball can be accurately specified using the above-described method using the frame images F1 and F2 to which the time code of time t has been added.

However, when using a general-purpose camera such as a camera mounted on a mobile terminal rather than a dedicated camera specialized for ball game video analysis as shown in Patent Document 1, the first mobile terminal 20A and the second mobile The terminal 20B is not necessarily able to provide an accurate time code. The reason for this will be explained below.

Since the ball in sports related to ball games moves at high speed, in order to identify the three-dimensional position of the ball, it is necessary to shoot at a high frame rate of 30 fps or higher, for example, and combine the frame image shot by the first mobile terminal 20A with the second frame image. Synchronization precision in milliseconds is required between the frame image captured by the mobile terminal 20B.

Here, it is possible to synchronize the internal clocks of the first mobile terminal 20A and the second mobile terminal 20B using NTP (Network Time Protocol). However, since the synchronization accuracy of NTP is on the order of seconds, there is a possibility that a synchronization difference on the order of milliseconds will occur between the internal clock of the first mobile terminal 20A and the internal clock of the second mobile terminal 20B. Therefore, with this method, the three-dimensional coordinates of a ball moving at high speed may not be specified with sufficient accuracy.

Alternatively, it is possible to synchronize the internal clocks of the first mobile terminal 20A and the second mobile terminal 20B using a satellite positioning system such as GPS (Global Positioning System). However, with this method, the camera of a mobile terminal that does not have a GPS function cannot be used. Furthermore, if a GPS signal cannot be received in an indoor facility or the like, there is a possibility that a synchronization difference will occur between the internal clock of the first mobile terminal 20A and the internal clock of the second mobile terminal 20B. Therefore, with this method, the three-dimensional coordinates of a ball moving at high speed may not be specified with sufficient accuracy.

Alternatively, when the first mobile terminal 20A and/or the second mobile terminal 20B assigns a time code, a synchronization error may occur. For example, if the processing performance of the second mobile terminal 20B is lower than that of the first mobile terminal 20A, the processing load of time code assignment on the mobile terminal 20B is large, and overall it is a little slower than the first mobile terminal 20A. There is a possibility that a time code will be added. In this case, even if the time codes of frame image F1 and frame image F2 are the same, frame image F2 may actually be photographed a little earlier than frame image F1. In the frame image F2 taken a little earlier, the ball is positioned at B3, and as shown in FIG. 2, the three-dimensional coordinates of the ball also shift to Pb. Therefore, the calculated movement trajectory of the ball also deviates.

Therefore, the ball game video analysis device 100 according to the present embodiment has a function of identifying the synchronization shift of frame images between the first mobile terminal 20A and the second mobile terminal 20B. Then, the ball game video analysis device 100 calculates the movement trajectory of the ball in consideration of the identified synchronization deviation. As a result, the movement trajectory of the ball can be calculated using a general-purpose camera such as a camera mounted on a mobile terminal, rather than a dedicated camera specialized for ball game video analysis as shown in Patent Document 1. This will be explained in detail below.

<Configuration of ball game video analysis device>
FIG. 3 is a block diagram showing a configuration example of ball game video analysis device 100 according to the present embodiment.

The ball game video analysis device 100 includes a moving image receiving section 101 , an image converting section 102 , a reference frame specifying section 103 , a reference frame changing section 104 , a trajectory analyzing section 105 , and a result display section 106 .

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A, and receives the second video 201B from the second mobile terminal 20B. Examples of video formats are MPEG4, TS file, H. 264, H. 265 mag. Each frame image constituting the first moving image 201A is given a time code by the first mobile terminal 20A. A time code is given to each frame image constituting the second moving image 201B by the second mobile terminal 20B. As described with reference to FIG. 1, the first video 201A and the second video 201B include, for example, a scene in which a player hits a serve.

The image conversion unit 102 converts each frame image of the first moving image 201A into a still image, and generates a first frame image group 202A. The image conversion unit 102 converts each frame image of the second moving image 201B into a still image, and generates a second frame image group 202B. Examples of still image formats include JPEG, GIF, and TIFF.

The reference frame identifying unit 103 identifies frame images (hereinafter referred to as "reference frame images") that serve as a reference for frame synchronization from the first frame image group 202A and the second frame image group 202B, respectively. . Hereinafter, the reference frame image specified from the first frame image group 202A will be referred to as a first reference frame image, and the reference frame image specified from the second frame image group 202B will be referred to as a second reference frame. It's called an image.

The reference frame identifying unit 103 identifies a frame image in which a specific sub-action constituting an action is photographed as a reference frame image. For example, the reference frame specifying unit 103 selects a frame image in which a moment when the hand that hits the serve (sub-action) hits the ball among a group of frame images in which a series of movements of a serve (action) is captured. Specify as the reference frame image. Next, an example of a method for specifying a reference frame image will be described with reference to FIG. 4.

As shown in FIG. 4, the reference frame specifying unit 103 sets a ball detection area 211A for the first frame image group 202A. The reference frame specifying unit 103 sets a ball detection area 211B for the second frame image group 202B. This reduces the ball detection area, so the time required for ball detection processing can be shortened. Note that the detection area for the ball does not need to be set.

The reference frame specifying unit 103 detects a ball from each frame image of the first frame image group 202A using a known method. For example, the reference frame identifying unit 103 detects a ball from a frame image by pattern matching. Alternatively, the reference frame identifying unit 103 detects a ball from the frame image using deep learning learned in advance for ball detection. The reference frame identifying unit 103 similarly detects a ball from each frame image of the second frame image group 202B.

The reference frame specifying unit 103 calculates a two-dimensional movement trajectory of the ball based on the position of the ball detected from each frame image of the first frame image group 202A. Then, the reference frame identifying unit 103 identifies the first reference frame image based on the change in the two-dimensional movement trajectory of the ball. For example, the reference frame identifying unit 103 identifies the first reference frame image through the following processes S11 to S13.

(S11) The reference frame identification unit 103 selects a first frame image in which the amount of movement of the ball in the X direction is greater than or equal to a predetermined threshold (that is, the ball has moved significantly in the horizontal direction) from the first frame image group 202A. Extract. As a result, it is possible to extract the first frame image in which the ball is flying toward the net immediately after the hand that hits the serve hits the ball.

(S12) The reference frame specifying unit 103 determines that among a plurality of first frame images included in a certain period before the first frame image extracted in S11, the amount of movement of the ball in the Y-axis direction is greater than or equal to a predetermined threshold. The first frame image that has changed and is closest to the first frame image extracted in S11 is identified. This makes it possible to identify the first frame image at the moment when the hand that served the ball hit the ball, among the multiple first frame images in which the ball is rising or falling after being thrown up for a serve. .

(S13) The reference frame identifying unit 103 sets the first frame image identified in S12 as the first reference frame image.

The reference frame identifying unit 103 identifies the second reference frame image by the same process as described above. Then, the reference frame specifying unit 103 extracts a time code corresponding to the first reference frame image (hereinafter referred to as "first reference time code") and a time code corresponding to the second reference frame image (hereinafter referred to as "second reference time code"). "Reference time code") is written in the synchronization difference information 203.

Note that the reference frame identifying unit 103 may identify the reference frame image using an action and sub-action different from serve. For example, the reference frame specifying unit 103 may use a frame image captured at the moment when the player's hand attacking (or digging, receiving, tossing, blocking, etc.) hits the ball as the reference frame image.

Further, the reference frame specifying unit 103 may write a value obtained by subtracting the second reference time code from the first reference time code (hereinafter referred to as “synchronization time difference”) to the synchronization shift information 203. The synchronization time difference written in this synchronization shift information 203 corresponds to the magnitude of the synchronization shift of frame images between the first mobile terminal 20A and the second mobile terminal 20B. For example, when the synchronization time difference is approximately 0, this indicates that the time code provided by the first mobile terminal 20A and the time code provided by the second mobile terminal 20B tend to be almost the same. If the synchronization time difference is a positive value, this indicates that the time code provided by the second mobile terminal 20B tends to be later than the time code provided by the first mobile terminal 20A. If the synchronization time difference is a negative value, this indicates that the time code provided by the first mobile terminal 20A tends to be later than the time code provided by the second mobile terminal 20B.

Returning to the explanation of FIG. 3.

The reference frame image automatically identified by the reference frame specifying unit 103 using the method described above is not necessarily the correct reference frame image. Therefore, the reference frame specifying unit 103 provides a UI (User Interface) 310 (hereinafter referred to as "reference frame confirmation UI") for the user to confirm the reference frame image. Note that details of the reference frame confirmation UI 310 will be described later (see FIG. 8).

The reference frame changing unit 104 provides a UI (hereinafter referred to as "reference frame changing UI") 320 for the user to change the reference frame image. Note that details of the reference frame change UI 320 will be described later (see FIG. 9). When the reference frame image is changed through the reference frame change UI 320, the reference frame changing unit 104 changes the synchronization shift information 203 based on the changed reference frame image.

The trajectory analysis unit 105 uses the synchronization difference information 203 from a first video 201A and a second video 201B that capture a serve action by a player, for example, to identify the movement trajectory of the ball in a three-dimensional space related to the action. , generates analysis result information 204 including information indicating the movement trajectory of the ball. Note that the first video 201A and second video 201B that are analyzed by the trajectory analysis unit 105 are the same as the first video 201A and second video 201B that were shot to generate the synchronization difference information 203 described above. It can be different.

For example, the trajectory analysis unit 105 calculates the camera parameters of each of the first mobile terminal 20A and the second mobile terminal 20B (based on a perspective projection transformation matrix for converting world coordinates into a camera image) obtained in advance through a calibration process. ), the position of the ball in the first frame image, and the position of the ball in the second frame image that is synchronized with the first frame image, corrected by the synchronization difference information 203. The three-dimensional coordinates of the ball at the timing when the frame image and the second frame image were photographed are identified. The trajectory analysis unit 105 specifies the movement trajectory of the ball in the three-dimensional space by arranging the three-dimensional coordinates of the ball identified in this way in time series. The trajectory analysis unit 105 generates analysis result information 204 including information indicating the movement trajectory of the identified ball. As a result, the movement trajectory of a ball in three-dimensional space can be analyzed with sufficient accuracy even when a camera mounted on a mobile terminal is used instead of a dedicated camera specialized for ball game video analysis.

The result display unit 106 displays the analysis result information 204 on, for example, the display terminal 30. Note that the result display unit 106 may display the analysis result information 204 on the first mobile terminal 20A and/or the second mobile terminal 20B.

<Preliminary work>
FIG. 5 is a flowchart showing an example of preliminary work. The preliminary work is work performed before the trajectory analysis unit 105 analyzes the actual action, and is, for example, work performed to generate the synchronization difference information 203.

The user installs the first mobile terminal 20A and the second mobile terminal 20B at positions where they can photograph the player and the ball from different directions (S101).

The user pairs the first mobile terminal 20A and the second mobile terminal 20B (S102). Pairing information may be transmitted to the ball game video analysis device 100.

The user instructs the first mobile terminal 20A and the second mobile terminal 20B to start photographing related to the preliminary work (S103). Thereby, the first mobile terminal 20A and the second mobile terminal 20B start photographing.

The player hits a serve (S104). Thereby, the first mobile terminal 20A and the second mobile terminal 20B can respectively photograph a first video 201A and a second video 201B in which the player hits a serve and the ball flies. Note that the user and the player may be the same person.

The user instructs the first mobile terminal 20A and the second mobile terminal 20B to stop photographing related to the preliminary work (S105). The first mobile terminal 20A and the second mobile terminal 20B each receive the instruction to stop shooting, and transmit the first video 201A and the second video 201B to the ball game video analysis device 100. The ball game video analysis device 100 specifies the first reference frame image from the first video 201A and the second reference frame image from the second video 201B, and displays the reference frame confirmation UI 310 on the display terminal 30.

The user checks the first reference frame image and the second reference frame image from the reference frame confirmation UI 310 displayed on the display terminal 30 (S106). For example, when changing the reference frame image, the user instructs the ball game video analysis device 100 to change the reference frame image through the reference frame confirmation UI 310. In this case, the ball game video analysis device 100 displays the reference frame change UI 320 on the display terminal 30.

Note that instructions to start and stop photographing in S103 and S105 above may be given from either one of the mobile terminals. For example, when an instruction to start and stop shooting is given from the first mobile terminal 20A, the instruction to start and stop shooting is automatically transmitted to the second mobile terminal 20B via the ball game video analysis device 100. It's fine.

The user operates the reference frame change UI 320 displayed on the display terminal 30 to change the reference frame image (S107). Note that details of S106 and S107 will be described later (see FIGS. 7 and 8).

Through the above processing, the ball game video analysis device 100 can generate the synchronization shift information 203 based on the first reference frame image and the second reference frame image that have been confirmed and corrected by the user.

<Processing related to preliminary work>
FIG. 6 is a sequence chart showing an example of processing related to preliminary work. FIG. 6 corresponds to detailed explanation of S106 to S107 in FIG. 5.

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A (S201), and receives the second video 201B from the second mobile terminal 20B (S202).

The image conversion unit 102 generates a first frame image group 202A from the first moving image 201A, and generates a second frame image group 202B from the second moving image 201B (S203).

The reference frame identifying unit 103 searches the first frame image group 202A for a first reference frame image in which a specific sub-action constituting an action has been photographed (S204). For example, as described above, the reference frame specifying unit 103 photographs the moment when the player's serve hand hits the ball based on changes in the two-dimensional (X direction, Y direction) movement trajectory of the ball. A first reference frame image is searched for.

Similarly to the above, the reference frame identifying unit 103 searches for the second reference frame image from the second frame image group 202B (S205). At this time, the reference frame specifying unit 103 may set the second frame image group 202B after a time code that is a predetermined time back from the time code of the first reference frame image as the search range for the second reference frame image. Thereby, the time required for the search can be shortened compared to the case where the entire second frame image group 202B is set as the search range.

The reference frame specifying unit 103 uses the time code of the first reference frame image (first reference time code) and the time code of the second reference frame image (second reference time code) as synchronization shift information 203. (S206).

The reference frame changing unit 104 transmits the first reference frame image, the second reference frame image, and information for displaying the reference frame confirmation UI 310 to the display terminal 30 (S207).

The display terminal 30 displays the first reference frame image, the second reference frame image, and the reference frame confirmation UI 310 (S208). Note that details of the reference frame confirmation UI 310 will be described later (see FIG. 8).

When the user performs an operation to the effect that the reference frame image is not changed on the reference frame confirmation UI 310 displayed in S208, the display terminal 30 sends instruction information indicating no change to the ball game video analysis device 100. Send (S209).

When the reference frame changing unit 104 receives instruction information indicating no change, it determines the contents of the synchronization shift information 203 written in S206 (S210). Then, the ball game video analysis device 100 ends this process.

FIG. 7 is a sequence chart showing an example of processing related to preliminary work when an operation to change the reference frame image in FIG. 6 is performed.

When the user performs an operation to change the reference frame image on the reference frame confirmation UI 310 displayed in S208, the display terminal 30 sends instruction information indicating that the reference frame image has been changed to the ball game video analysis device 100. Send (S221).

When the reference frame changing unit 104 receives instruction information indicating that there is a change, the reference frame changing unit 104 performs the following process. That is, the reference frame changing unit 104 extracts a plurality of first frame images taken before and after the first reference frame image searched in S204 from the first frame image group 202A, and The first frame image is set as a candidate for the first reference frame image.

Similarly, the reference frame changing unit 104 extracts a plurality of second frame images taken before and after the second reference frame image searched in S205 from the second frame image group 202B, and The second frame image is set as a candidate for the second reference frame image. Then, the reference frame change unit 104 displays the plurality of first candidate frame candidates and their time codes, the plurality of second reference frame image candidates and their time codes, and the reference frame change UI 320. information is transmitted to the display terminal 30 (S222).

The display terminal 30 displays the reference frame change UI 320 (S223). Note that details of the reference frame change UI 320 will be described later (see FIG. 9).

The user operates the reference frame change UI 320 displayed in S223 to select the changed first reference frame image from among the plurality of first reference frame image candidates, and selects the changed first reference frame image from among the plurality of first reference frame image candidates. The changed second reference frame image is selected from among the reference frame image candidates (S224).

The display terminal 30 displays the time code of the first reference frame image after the change selected in S224 (hereinafter referred to as the "first reference time code after the change") and/or the time code of the second reference frame image after the change. The time code of the reference frame image (hereinafter referred to as the "changed second reference time code") is transmitted to the ball game video analysis device 100 (S225).

The reference frame changing unit 104 overwrites the synchronization difference information 203 with the changed first reference time code and/or the changed second reference time code received from the display terminal 30, and removes the synchronization difference. The contents of the information 203 are determined (S226). As a result, the contents of the synchronization difference information 203 automatically generated by the reference frame specifying unit 103 are overwritten with the time code of the reference frame image changed by the user through the reference frame changing unit 104. Then, the ball game video analysis device 100 ends this process.

<Details of UI for checking reference frame>
FIG. 8 is a diagram showing a detailed example of the reference frame confirmation UI.

As shown in FIG. 8, the reference frame confirmation UI 310 includes a first image display area 210A, a second image display area 210B, an OK button 312, a change button 313, and a redo button 314. Note that the reference frame confirmation UI 310 does not need to include either the change button 313 or the redo button 314.

The display terminal 30 displays the first reference frame image transmitted from the ball game video analysis device 100 in S207 of FIG. 6 in the first image display area 210A. The display terminal 30 displays the second reference frame image transmitted from the ball game video analysis device 100 in S207 of FIG. 6 in the second image display area 210B. As shown in FIG. 8, the first image display area 210A and the second image display area 210B may be arranged side by side. However, the manner in which the first image display area 210A and the second image display area 210B are arranged is not limited to being arranged side by side. For example, the first image display area 210A and the second image display area 210B may be displayed as separate windows.

When the OK button 312 is pressed, the display terminal 30 transmits instruction information indicating no change in S209 of FIG. 6 to the ball game video analysis device 100. The user can confirm that both the first reference frame image displayed in the first image display area 210A and the second reference frame image displayed in the second image display area 210B are captured by the subaction correctly. If the image is an image that has been edited, the OK button 312 may be pressed. Thereby, as shown in S210 of FIG. 6, the contents of the synchronization difference information 203 are determined.

When the change button 313 is pressed, the display terminal 30 transmits instruction information indicating that there is a change in S221 of FIG. 7 to the ball game video analysis device 100. The user can correctly determine whether at least one of the first reference frame image displayed in the first image display area 210A and the second reference frame image displayed in the second image display area 210B is a subaction. If the image is not a photographed image, the change button 313 may be pressed. Thereby, as shown in S223 in FIG. 7, the reference frame change UI 320 is displayed on the display terminal 30.

When the redo button 314 is pressed, the display terminal 30 transmits instruction information indicating redo of the preliminary work to the ball game video analysis device 100. When receiving this instruction, the ball game video analysis device 100, the first mobile terminal 20A, and the second mobile terminal 20B redo the preliminary work from S103 in FIG. 5. For example, if the serve in S104 fails, the first reference frame image and/or the second reference frame image clearly shows a ball other than the ball served by the player (for example, a ball lying on the floor or another player). If the user recognizes a ball being practiced (such as a ball being practiced), the user may press the redo button 314.

Note that the first reference frame images include three first frame images immediately before the first reference frame image searched in S204, and three first frame images immediately after the first reference frame. The frame image may be included. In this case, there are a total of seven first reference frame images, including the first reference frame image searched in S204. Similarly, the second reference frame image includes three second frame images immediately before the second reference frame image searched in S205 and three second frame images immediately after the second reference frame image. A second frame image may be included. In this case, there are a total of seven second reference frame images, including the second reference frame image searched in S205.

In this case, the display terminal 30 displays seven first reference frame images and seven second reference frame images in the first image display area 210A and the second image display area 210B, respectively. It may be synchronized and displayed continuously.

Note that the above seven images are just an example, and the number of the plurality of first and second reference frame images may be any number. Further, the plurality of first and second reference frame images are not limited to being consecutive one frame at a time, and may be skipped at predetermined frame intervals.

This allows the user to not only check the synchronization between one first reference frame image and the second reference frame image, but also to check the synchronization between the first reference frame image and the second reference frame image. You can easily check the synchronization between

Although an example has been shown in which the reference frame confirmation UI 310 shown in FIG. 8 is displayed on the display terminal 30, this reference frame confirmation UI 310 may be displayed on either the first mobile terminal 20A or the second mobile terminal 20B. may be displayed. Alternatively, the reference frame confirmation UI 310 for the first reference frame image may be displayed on the first mobile terminal 20A, and the reference frame confirmation UI 310 for the second reference frame image may be displayed on the second mobile terminal 20B. .

<Details of UI for changing reference frame>
FIG. 9 is a diagram showing a detailed example of the reference frame change UI.

As shown in FIG. 9, the reference frame change UI 320 includes a first image display area 210A, a second image display area 210B, an OK button 322, and a return button 323. Furthermore, the reference frame change UI 320 includes a 1 frame return button 324A, an N frame return button 325A, a 1 frame advance button 326A, and an N frame advance button 327A, which correspond to the first image display area 210A. Here, N is a predetermined integer of 2 or more. Further, the reference frame change UI 320 includes a 1 frame return button 324B, an N frame return button 325B, a 1 frame advance button 326B, and an N frame advance button 327B, which correspond to the second image display area 210B.

When the return button 323 is pressed, the display terminal 30 returns to displaying the reference frame confirmation UI 310 shown in FIG. 8 .

When the one-frame return button 324A is pressed, the display terminal 30 displays a first reference frame image candidate that is one frame earlier than the first reference frame image candidate currently displayed in the first image display area 210A. , is displayed in the first image display area 210A. Note that the same applies to the one-frame return button 324B corresponding to the second image display area 210B.

When the N frame return button 325A is pressed, the display terminal 30 displays a first reference frame image candidate that is N frames earlier than the first reference frame image candidate currently displayed in the first image display area 210A. , is displayed in the first image display area 210A. Note that the same applies to the N frame return button 325B corresponding to the second image display area 210B.

When the advance one frame button 326A is pressed, the display terminal 30 selects a first reference frame image candidate one frame later than the first reference frame image candidate currently displayed in the first image display area 210A. , is displayed in the first image display area 210A. Note that the same applies to the advance one frame button 326B corresponding to the second image display area 210B.

When the N frame advance button 327A is pressed, the display terminal 30 selects a first reference frame image candidate N frames later than the first reference frame image candidate currently displayed in the first image display area 210A. , is displayed in the first image display area 210A. Note that the same applies to the advance N frame button 327B corresponding to the second image display area 210B.

The user presses each of the above-mentioned buttons 324A to 327A and 324B to 327B so that the image in which the subaction has been correctly captured is displayed in both the first image display area 210A and the second image display area 210B. You can operate it.

When the OK button 322 is pressed, the display terminal 30 displays the time code of the candidate for the first reference frame image being displayed in the first image display area 210A (the first reference time code after the change), and The time code of the candidate for the second reference frame image being displayed in the second image display area 210B (the changed second reference time code) is transmitted to the ball game video analysis device 100. This process corresponds to S225 in FIG. The user may press the OK button 322 after performing an operation such that an image in which the sub-action has been correctly captured is displayed in both the first image display area 210A and the second image display area 210B.

<Analysis result screen>
FIG. 10 is a diagram showing a display example of the analysis results.

The first mobile terminal 20A and the second mobile terminal 20B photograph the player's serve practice, and generate and transmit a first video 201A and a second video 201B, respectively. Note that the first video 201A and the second video 201B may be different from the first video 201A and the second video 201B shot for preliminary work.

The trajectory analysis unit 105 corrects the synchronization deviation of the frame images between the first video 201A and the second video 201B in which serve practice is shot based on the synchronization difference information 203, and then calculates the movement trajectory of the ball. is analyzed and analysis result information 204 is generated. For example, in the synchronization difference information 203, if the second reference time code is later than the first reference time code, the trajectory analysis unit 105 determines the difference between the second reference time code and the first reference time code. After making corrections to delay each frame image of the first moving image 201A by the difference time, each frame image of the first moving image 201 and each frame image of the second moving image 201B are synchronized, Analyze the movement trajectory of. As a result, the delay in assigning the time code by the second mobile terminal 20B is corrected, and the movement trajectory of the ball can be calculated with sufficient accuracy.

The result display unit 106 causes the display terminal 30 to display an analysis result screen 330 based on the analysis result information 204. For example, as shown in FIG. 10, on the analysis result screen 330, the movement trajectory 331 of the ball indicated by the analysis result information 204 may be displayed superimposed on the playback of the first video 201A or the second video 201B. . In addition, the analysis result screen 330 may display the height and speed of the ball hit by the hand at the moment it leaves the hand. In addition, the analysis result screen 330 may display the height and speed of the ball at the moment the ball passes over the net.

This allows the player to check in detail the movement of the ball during the serve as well as the movement trajectory of the ball due to the serve from the analysis result screen 330.

In addition, in the above, as a preliminary work, a video for synchronization adjustment is taken in advance, synchronization adjustment and confirmation is performed, and then a video is taken for analysis processing, and the video with the trajectory of the ball etc. superimposed is used as the analysis result. An example of displaying this is explained. However, in the present embodiment, the synchronization adjustment and analysis processing are performed continuously on the first video shot, for example, without distinguishing between the video for synchronization adjustment and the video for analysis processing. Good too. Furthermore, detection of synchronization deviation may be automated and synchronization adjustment may be performed at any timing while performing analysis processing on sequentially shot videos.

<Modified example>
In the above description, the first time code added to the first frame image by the first mobile terminal 20A and the second time code added to the second frame image by the second mobile terminal 20B are used as information indicating the frame synchronization deviation. An example using the time code No. 2 has been explained. However, information indicating a shift in frame synchronization is not limited to time codes.

For example, as information indicating a frame synchronization shift, the frame number of the first frame image (hereinafter referred to as "first frame number") in the first video 201A generated by the first mobile terminal 20A, and the frame number of the first frame image in the first video 201A generated by the first mobile terminal 20A, A frame image (hereinafter referred to as "second frame number") in the second moving image 201B generated by the mobile terminal 20B may be used. In this case, the reference frame identifying unit 103 identifies the first frame number (hereinafter referred to as "first reference frame number") of the first reference frame image and the second frame number (hereinafter referred to as "first reference frame number") of the second reference frame image. (referred to as “second reference frame number”) may be written in the synchronization difference information 203. Then, the reference frame changing unit 104 selects the frame number of the changed first reference frame image (hereinafter referred to as "the changed first reference frame number") selected through the reference frame changing UI 320, and/or , the frame number of the changed second reference frame image (hereinafter referred to as “changed second reference frame number”) may be overwritten on the synchronization shift information 203.

(Embodiment 2)
In Embodiment 2, a ball game video is created in which a target video and a comparison video are each identified from among a plurality of videos taken of the actions of a person (player) playing a ball game, and the proficiency level of the action and points for improvement in the form can be grasped. The analysis system will be explained. In addition, in the second embodiment, the same reference numerals are given to the same components as in the first embodiment, and the description thereof may be omitted.

Furthermore, in the second embodiment, a case will be described using an example in which the player's action practice is volleyball serve practice. However, the second embodiment is not limited to this example, and can be used for various purposes such as tennis serve or volley practice, badminton serve or smash practice, soccer heading practice, baseball batting practice, and golf swing practice. Applicable to action practice.

<Configuration of ball game video analysis system>
FIG. 11 is a diagram showing a configuration example of a ball game video analysis system 10 according to the second embodiment.

A ball game video analysis system 10 according to the second embodiment is similar to the ball game video analysis system 10 according to the first embodiment shown in FIG. However, as shown in FIG. 11, the first mobile terminal 20A (the camera thereof) may be installed near the net on one side of the court, facing the player who serves. The second mobile terminal 20B (its camera) may be installed near the net on the other side of the court, facing the player who serves.

Furthermore, the ball game video analysis system 10 described in the second embodiment may be configured so that synchronization deviations and the like are corrected by the method described in the first embodiment.

<Configuration of ball game video analysis device>
FIG. 12 is a block diagram showing a configuration example of a ball game video analysis device 100 according to the second embodiment.

As shown in FIG. 12, the ball game video analysis device 100 includes a video reception section 101, a trajectory analysis section 105, a posture analysis section 131, a sub-action analysis section 132, a record management section 133, and a display control section 134.

The video receiving unit 101 receives the first video 201A from the first mobile terminal 20A, and receives the second video 201B from the second mobile terminal 20B. For example, a scene in which a player serves is photographed in the first video 201A and the second video 201B.

The trajectory analysis unit 105 identifies the movement trajectory of the ball from the first video 201A and the second video 201B, using the technology disclosed in Patent Document 1, for example, and generates trajectory information 220. The trajectory information 220 includes a plurality of pieces of information indicating the correspondence between a time code at a certain point in the video and a three-dimensional position of the ball at the time code.

The posture analysis unit 131 generates posture model information 221 indicating the posture of the player during the period in which the action was performed, based on the first video 201A and the second video 201B. For example, the posture analysis unit 131 determines the three-dimensional position of each predetermined joint of the player (hereinafter referred to as "joint position") from the first video 201A and the second video 201B using a known joint estimation technique. The movement of each joint position is specified. The posture analysis unit 131 generates information indicating each identified joint position and the movement of each joint position as posture model information 221. Note that details of the posture model information 221 will be described later (see FIG. 14).

The sub-action analysis unit 132 analyzes each sub-action that constitutes the action captured in the first video 201A and the second video 201B, based on the trajectory information 220 and/or the posture model information 221. Then, the sub-action analysis unit 132 generates sub-action information 222 indicating the timing of occurrence of each of the analyzed sub-actions. For example, a jumping serve, which is an example of an action in volleyball, has a series of sub-actions such as ``run-up,'' ``toss,'' ``jump,'' and ``hit,'' which means that the hand that served hit the ball. It consists of Note that details of the subactions will be described later (see FIG. 15).

The recording management unit 133 manages a plurality of pieces of recorded information 200. The recorded information 200 is generated from the shooting date and time of the first video 201A and the second video 201B, the first video 201A and the second video 201B, and the first video 201A and the second video 201B. This is information indicating a correspondence relationship with trajectory information 220, posture model information 221, and sub-action information 222. That is, the recorded information 200 is information indicating the correspondence between a video of an action performed by a player on a certain date and time, and trajectory information 220, posture model information 221, and sub-action information 222 generated from the video. You can say that.

The display control unit 134 causes the display terminal 30 to display a UI for selecting recorded information 200 based on the trajectory information 220 from among the plurality of recorded information 200 managed by the recording management unit 133. Further, the display control unit 134 causes the display terminal 30 to display the first moving image 201A or the second moving image 201B that have a corresponding relationship in the recorded information 200 selected through the UI.

In addition, the display control unit 134 selects, for example, the first recorded information 200A (see FIG. 13) and the If No. 2 recorded information 200B (see FIG. 13) is selected, the following process may be performed. That is, the display control unit 134 displays the first moving image 201A or the second moving image 201B that has a corresponding relationship in the first recorded information 200A, and the first moving image 201A or the second moving image 201B that has a corresponding relationship in the second recorded information 200B. The display terminal 30 may display a UI 400A (hereinafter referred to as "video comparison UI") 400A that displays the videos 201B of 2 and 2 side by side. Note that details of the video comparison UI 400A will be described later (see FIGS. 16 and 17).

Further, the display control unit 134 selects, for example, the first recorded information 200A and the second recorded information 200 from among the plurality of recorded information 200 managed by the recording management unit 133 through the video comparison UI 400A displayed on the display terminal 30. If the recorded information 200B is selected, the following process may be performed (see FIG. 14). That is, the display control unit 134 generates the first posture model 231A generated from the posture model information 221 that has a correspondence relationship in the first recorded information 200A, and the posture model information 221 that has a correspondence relationship in the second recorded information 200B. The display terminal 30 may display a UI (hereinafter referred to as "posture comparison UI") 400C that displays the second posture model 231B generated from the second posture model 231B side by side or in an overlapping manner. Note that details of the posture comparison UI 400C will be described later (see FIGS. 18 to 21).

The display control unit 134 also displays a movement trajectory 241A of the moving object (ball) generated from the trajectory information 220 that has a corresponding relationship in the first recorded information 200A, and trajectory information that has a corresponding relationship in the second recorded information 200B. 220 may be displayed on the display terminal 30 along with the posture models 231A and 231B in the posture comparison UI 400C (see FIGS. 18 to 21).

Further, the display control unit 134 may cause the display terminal 30 to display a sub-action selection UI (447) that allows selection of one sub-action among the plurality of sub-actions. Then, the display control unit 134 causes the display terminal 30 to display the posture models 231A and 231B at the occurrence timing of one sub-action selected through the sub-action selection UI (447) on the posture comparison UI (400D). Good (see Figure 20).

<Display terminal configuration>
FIG. 13 is a block diagram showing a configuration example of the display terminal 30 according to the second embodiment.

The display terminal 30 includes an information transmitting/receiving section 151 and a UI control section 152.

The information transmitting and receiving unit 151 transmits and receives information to and from the ball game video analysis device 100 via the communication network N. For example, the information transmitting/receiving unit 151 receives a plurality of recorded information 200 (for example, first recorded information 200A and second recorded information 200B) from the ball game video analysis device 100.

The UI control unit 152 cooperates with the display control unit 134 of the ball game video analysis device 100 to display the video comparison UI 400A, the posture comparison UI 400C, etc. on a display that is an example of the output device 1002 (see FIG. 23). Further, the UI control unit 152 detects input from the user to the UI through a touch panel, which is an example of the input device 1001 (see FIG. 23). When the UI control unit 152 detects an input to the UI, the UI control unit 152 transmits the input content to the ball game video analysis device 100 through the information transmission and reception unit 151.

<Details of posture model>
FIG. 14 is a diagram for explaining an example of a posture model.

The posture model information 221 includes information indicating each joint position of the player and the movement of each joint position.

For example, the display control unit 134 or the UI control unit 152 refers to the posture model information 221 that has a correspondence relationship in the first recorded information 200A, and refers to the position of each joint at the timing of the sub-action “jump” in the action period (see FIG. 14). (white circles and black circles). Similarly, the display control unit 134 or the UI control unit 152 refers to the corresponding posture model information 221 in the second recorded information 200B to specify the position of each joint at the timing of the sub-action “jump” in the action period. do. Then, the display control unit 134 generates player posture models 231A and 231B as shown in FIG. 14 by connecting each identified joint position with a line based on the structure of the human body. Since the posture models 231A and 231B are three-dimensional models, the display control unit 134 or the UI control unit 152 can arbitrarily perform operations such as 3D rotation on the posture models to change the viewpoint in the 3D space and display them. can.

Further, as shown in FIG. 14, the display control unit 134 or the UI control unit 152 converts the first posture model 231A and the second posture model 231B to the neck joint position 232A at the timing of the sub-action “jump”. , 232B can be displayed in an overlapping manner.

The first posture model 231A and the second posture model 231B may be generated based on videos of serves performed by the same player on different dates and times. In this case, the user can visually recognize the difference in posture during the sub-action "jump" performed by the same player on different dates and times from the superimposedly displayed first posture model 231A and second posture model 231B. . Therefore, the user can easily check the improvement of the same player's serve, for example.

The first posture model 231A and the second posture model 231B may be generated based on videos of serves performed by different players. In this case, the user can visually recognize the difference in posture during the sub-action "jump" performed by different players from the first posture model 231A and second posture model 231B that are displayed in a superimposed manner. Therefore, the user can easily confirm, for example, the difference in serving posture between the model player and the player practicing.

Furthermore, by referring to the posture model information 221, it is possible to specify the timing at which the player jumped and the highest point reached when the player jumped. For example, the sub-action analysis unit 132 may detect the timing when the joint positions of both ankles of the player are separated from the ground by a certain distance or more as the timing when the player jumps. For example, the sub-action analysis unit 132 may detect the timing when the joint positions 232A and 232B of the player's neck are at the highest point as the timing when the player's jump has reached the highest point.

Note that the posture models 231A and 231B are not limited to models in which joint positions are connected with lines as shown in FIG. 14. For example, the posture models 231A and 231B may be texture-mapped three-dimensional CG (Computer Graphics) models.

<Analysis of subactions>
FIG. 15 is a diagram for explaining sub-actions that constitute a jumping serve.

As shown in Figure 15, a jumping serve, which is an example of a volleyball action, can be interpreted as a "run-up," "toss," "jump," and a "hit," which means that the hand that hit the serve hit the ball. It consists of a series of sub-actions.

The sub-action analysis unit 132 identifies a frame image at the timing at which each sub-action occurs, based on the ball trajectory information 220 and/or the player's posture model information 221.

For example, the sub-action analysis unit 132 refers to the ball trajectory information 220 to identify a frame image at a timing when the ball's movement trajectory changes from a stationary state to an upward movement. Then, the sub-action analysis unit 132 writes the timing (for example, time information) at which the identified frame image was photographed into the sub-action information 222 as the timing of occurrence of the sub-action "toss".

For example, the sub-action analysis unit 132 refers to the player's posture model information 221 and identifies a frame image at a time when the joint positions of both ankles of the player are separated from the ground by a certain distance or more. Then, the sub-action analysis unit 132 writes the timing at which the identified frame image was photographed into the sub-action information 222 as the timing at which the sub-action "jump" occurs.

For example, the sub-action analysis unit 132 refers to the ball trajectory information 220 to identify a frame image at a timing when the amount of movement of the ball in the direction approaching the net changes to a predetermined threshold value or more. Then, the sub-action analysis unit 132 writes the timing at which the identified frame image was photographed into the sub-action information 222 as the timing at which the sub-action "hit" occurs.

As a result, information (for example, time information) indicating the timing at which each sub-action constituting the action occurs is written in the sub-action information 222. Therefore, by referring to the sub-action information 222, a frame image at the time of a desired sub-action can be specified from the video. Further, by referring to the sub-action information 222, the posture model for a desired sub-action can be specified from among the posture model information 221.

<Details of video comparison UI>
FIG. 16 is a diagram showing a display example of the video comparison UI 400A. The video comparison UI 400A is an example of a comparison UI, and is a UI for comparing videos related to a plurality of mutually different recorded information 200.

As shown in FIG. 16, the video comparison UI 400A includes selection lists 401A and 401B, display areas 402A and 402B, an add button 403, a seek bar 410, a play button 411, a pause button 412, a slow play button 413, a repeat button 414, It includes a arranging button 415, an enlargement button 416, a camera switching button 417, an image reversing button 418, and a 3D switching button 419.

In the selection lists 401A and 401B, information specified from the trajectory information 220 related to the recorded information 200 is displayed in a list format. The information specified from the trajectory information 220 includes, for example, information indicating the height, speed, and angle of the ball when it passes the net during a serve. One row of the selection list 401, 401B may correspond to one piece of recorded information 200.

Furthermore, the recorded information 200 may include information indicating an evaluation of an action (for example, a serve). In this case, the evaluation of the action may also be displayed in the selection lists 401A and 401B, as shown in FIG. 16. The evaluation of the action may be manually input by the user. Alternatively, the action rating may be automatically determined based on the height, speed, and angle of the ball as it passes the net. In this case, the evaluation of the action may be determined based on whether the angle of the ball when passing through the net is greater than or equal to a predetermined threshold. Here, the threshold value referred to in the evaluation of the action may be set arbitrarily by the user.

In the display area 402A, a first moving image 201A or a second moving image 201B related to the first recorded information 200A (for example, a line filled with dots in FIG. 16) selected from the selection list 401A is displayed. In the display area 402B, the first moving image 201A or the second moving image 201B related to the second recorded information 200B (for example, the line filled with dots in FIG. 16) selected from the selection list 401B is displayed.

As shown in FIG. 16, in the video comparison UI 400A, there is a set of a selection list 401A and a display area 402A (hereinafter referred to as a "first set"), and a set of a selection list 401B and a display area 402B (hereinafter referred to as a "second set"). '') may be displayed side by side. Thereby, the user selects the rows (recorded information 200) to be compared from the selection list 401A and the selection list 401B, respectively, and compares the videos related to the selected rows displayed in the display areas 402A and 402B, respectively. You can watch while watching.

Note that the selection list 401A included in the first set includes records generated from the first video 201A and second video 201B shot on the day using the first mobile terminal 20A and the second mobile terminal 20B. Information 200 may be displayed at any time.

When the add button 403 is pressed, the display control unit 134 adds a new set including a new selection list and display area to the video comparison UI 400A. In this case, the UI control unit 152 may inquire of the user about the date of the recorded information 200 to be displayed in the new selection list. Then, the UI control unit 152 cooperates with the display control unit 134 to acquire one or more pieces of record information 200 including the date input by the user from the record management unit 133, and based on the acquired record information 200. , a selection list of new sets added may be displayed.

The seek bar 410 is a bar that indicates the time during which the moving images displayed in the display areas 402A and 402B are being played. When the user selects a certain position on the seek bar 410, the UI control unit 152 may play back the video from the time corresponding to the selected position.

When the play button 411 is pressed, the UI control unit 152 starts playing the video displayed in the display areas 402A and 402B.

When the pause button 412 is pressed, the UI control unit 152 pauses the playback of the video displayed in the display areas 402A and 402B.

When the slow playback button 413 is pressed, the UI control unit 152 plays back the videos displayed in the display areas 402A and 402 in slow motion (eg, at 1/2 speed).

When the repeat button 414 is pressed, the UI control unit 152 repeatedly plays back the videos displayed in the display areas 402A and 402B. Note that the slow playback button 413 and repeat button 414 may be used together.

When a plurality of sets in the video comparison UI 400A are selected and the arrange button 415 is pressed, the UI control unit 152 controls the subaction video comparison UI, which is a UI for comparing the subactions of the selected plurality of sets. Switch to the display of UI 400B (see FIG. 17). Note that details of the sub-action video comparison UI 400B will be described later (see FIG. 17).

When at least one set in the video comparison UI 400A is selected and the enlarge button 416 is pressed, the UI control unit 152 enlarges the range including the player of the video displayed in the display area of the selected set. indicate. Note that the range to be enlarged and displayed may be arbitrarily designated by the user. Further, the display magnification rate may be such that the user can arbitrarily specify the magnification rate.

When at least one set in the video comparison UI 400A is selected and the camera switching button 417 is pressed, the UI control unit 152 selects the video (for example, the first video) displayed in the display area of the selected set. 201A) to a video (for example, second video 201B) shot with another mobile terminal. Thereby, for example, the videos displayed in each display area 402A, 402B can be aligned with videos shot from the same direction.

When at least one set in the video comparison UI 400A is selected and the video inversion button 418 is pressed, the UI control unit 152 horizontally reverses the video displayed in the display area of the selected set. This allows, for example, to easily compare the actions of a right-handed player and the actions of a left-handed player.

When at least one set in the video comparison UI 400A is selected and the 3D switching button 419 is pressed, the UI control unit 152 displays the posture comparison UI 400C (see FIG. 18) for comparing the posture models of the selected set. ). Note that details of the posture comparison UI 400C will be described later (see FIG. 18).

FIG. 17 is a diagram showing a display example of the sub-action video comparison UI 400B.

For example, in the video comparison UI 400A shown in FIG. 16, when the first set and the second set are selected and the arrange button 415 is pressed, the UI control unit 152 Display UI400B.

The sub-action video comparison UI 400B includes display areas 421A, 421B, and a plurality of sub-action selection buttons 420 expanded from the arrange button 415. Note that the sub-action selection button 420 is an example of a sub-action selection UI.

The display areas 421A and 421B are arranged side by side as shown in FIG. 17. The UI control unit 152 enlarges the range including the player of the video displayed in the first set and displays it in the display area 421A. Similarly, the UI control unit 152 enlarges the range including the player of the video displayed in the second set and displays it in the display area 421B. At this time, the UI control unit 152 may adjust the display of the video in the display areas 421A, 421B so that the joint position of the player's neck is located at the center of the display areas 421A, 421B.

The plurality of sub-action selection buttons 420 are buttons for selecting sub-actions, and correspond to sub-actions of "toss", "jump", and "hit", for example.

For example, when the user selects the sub-action selection button 420 "Hit", the UI control unit 152 performs the following process. That is, the UI control unit 152 refers to the sub-action information 222 related to the first recorded information 200A, and selects a frame at the timing when the sub-action "hit" is captured from among the videos displayed in the first set. Identify images. Then, the UI control unit 152 displays the specified frame image in the display area 421A. Similarly, the UI control unit 152 refers to the sub-action information 222 related to the second recorded information 200B, and selects the timing at which the sub-action "Hit" was shot from among the videos displayed in the second set. Identify the frame image. Then, the UI control unit 152 displays the specified frame image in the display area 421B. If the play button 411 is pressed in this state, the UI control unit 152 may start playing the video from the frame image at the timing of the specified sub-action "hit". This allows the user to easily compare, for example, images of the player at the time of the sub-action "hit" of serves performed on different dates and times. Note that if the timing of the frame images of the sub-action "hit" is aligned, the video may be played including "run-up", "toss", etc.

<Details of posture comparison UI>
FIG. 18 is a diagram illustrating an example of a posture comparison UI when posture models are displayed side by side. FIG. 19 is a diagram illustrating an example of a posture comparison UI when posture models are displayed in an overlapping manner. The posture comparison UI 400C is an example of a comparison UI, and is a UI for comparing posture model information 221 related to a plurality of different pieces of recorded information 200.

For example, in the video comparison UI 400A shown in FIG. 16, when the first set and the second set are selected and the 3D switching button 419 is pressed, the UI control unit 152 Display the comparison UI400C.

The posture comparison UI 400C includes a display area 434, a seek bar 410, a play button 411, a pause button 412, a slow play button 413, a repeat button 414, a video switch button 441, a trajectory display switch button 442, a ball display switch button 443, and a player display. It includes a switching button 444, a superimposition/separation switching button 445, and a arranging button 446.

As shown in FIGS. 18 and 19, the UI control unit 152 controls the posture model 231A, ball 240A, and ball movement trajectory 241A corresponding to the first set, and the posture model 231B and ball 240B corresponding to the second set. and the movement trajectory 241B of the ball are displayed in the display area 434. Hereinafter, the posture model 231A, the ball 240A, and the ball movement trajectory 241A corresponding to the first set are collectively referred to as a first posture model or the like. Similarly, the posture model 231B, ball 240B, and ball movement trajectory 241B corresponding to the second set are collectively referred to as a second posture model or the like.

By performing a predetermined operation on the display area 434, the user can arbitrarily rotate the first and second posture models displayed in the display area 434 in 3D space. You can display it by changing the viewpoint. This allows the user to check and compare the postures of the players during serves performed on different dates and times from various viewpoints.

The seek bar 410 is a bar that indicates the time during which the first and second posture models, etc. displayed in the display area 434 are being reproduced. That is, the posture model etc. displayed in the display area 434 represents the posture etc. of the player at the timing of the playback time being displayed on the seek bar 410.

When the playback button 411 is pressed, the UI control unit 152 starts playing back the actions of the first and second posture models displayed in the display area 434. During playback, the first and second posture models, etc. are displayed like an animated video in the display area 434. Note that while the animation videos of the first and second posture models are being played, the first video 201A and the second video 201B, which are actual videos, are played synchronously in separate windows (not shown). Good too.

When the pause button 412 is pressed, the UI control unit 152 pauses the operations of the first and second posture models displayed in the display area 434.

When the slow playback button 413 is pressed, the UI control unit 152 plays back the motions of the first and second posture models displayed in the display area 434 in slow motion (for example, at 1/2 speed).

When the repeat button 414 is pressed, the UI control unit 152 repeatedly reproduces the actions of the first and second posture models displayed in the display area 434. Note that the slow playback button 413 and repeat button 414 may be used together.

When the video switching button 441 is pressed, the UI control unit 152 switches the display to the video comparison UI 400A shown in FIG. 16.

When the trajectory display switching button 442 is pressed, the UI control unit 152 switches between displaying and non-displaying the movement trajectories 241A and 241B in the display area 434.

When the ball display switching button 443 is pressed, the UI control unit 152 switches between displaying and non-displaying the balls 240A and 240B in the display area 434.

When the player display switching button 444 is pressed, the UI control unit 152 switches between displaying and non-displaying the posture models 231A and 231B in the display area 434.

When the superimposition separation switching button 445 is pressed, the UI control unit 152 displays a separated display of the first posture model, etc., and the second posture model, etc. as shown in FIG. 18 in the display area 434, and a display as shown in FIG. The display is switched between a display in which the first posture model, etc. and the second posture model, etc. are superimposed as shown. When displaying the first posture model 231A and the second posture model 231B in an overlapping manner, the UI control unit 152 may display them in an overlapping manner based on the neck joint positions 232A and 232B. Further, when displaying the first posture model 231A and the second posture model 231B in an overlapping manner, the UI control unit 152 controls the ball 240A according to the movement of the positions of the posture models 231A and 231B, as shown in FIG. , 240B and the positions of the ball movement trajectories 241A and 241B are also moved and displayed.

When the arrange button 446 is pressed, the UI control unit 152 displays a sub-action posture comparison UI 400D (see FIG. 20). Details of the sub-action posture comparison UI 400D will be described later (see FIG. 20).

Note that the posture comparison UI 400C may include a model inversion button (not shown). When at least one posture model in the posture comparison UI 400C is selected and the model inversion button is pressed, the UI control unit 152 displays the selected posture model horizontally inverted. Thereby, for example, the posture of a right-handed player and the posture of a left-handed player can be easily compared.

FIG. 20 is a diagram illustrating an example of a sub-action posture comparison UI.

For example, when the arrange button 446 is pressed on the posture comparison UI 400C shown in FIGS. 18 and 19, the UI control unit 152 displays the sub-action posture comparison UI 400D shown in FIG.

The sub-action posture comparison UI 400D includes display areas 451A, 451B, and a plurality of sub-action selection buttons 447 expanded from the arrange button 446. Note that the sub-action selection button 447 is an example of a sub-action selection UI.

The display areas 451A and 451B are arranged side by side as shown in FIG. 20. The UI control unit 152 enlarges the first posture model etc. (231A, 240A, 241A) and displays it in the display area 451A. Similarly, the UI control unit 152 enlarges the second posture model etc. (231B, 240B, 241B) and displays it in the display area 451B. At this time, the UI control unit 152 may adjust the display of the posture models 231A, 231B so that the neck joint positions 232A, 232B of the posture models are located at the center of the display areas 451A, 451B.

The plurality of sub-action selection buttons 447 are buttons for selecting sub-actions, and correspond to sub-actions such as "toss", "jump", and "hit", for example.

For example, when the user selects the sub-action selection button 447 "Jump", the UI control unit 152 performs the following process. That is, the UI control unit 152 refers to the sub-action information 222 related to the first recorded information 200A and specifies the first posture model and the like at the timing of the sub-action "jump". Then, the UI control unit 152 displays the identified first posture model and the like in the display area 451A. Similarly, the UI control unit 152 refers to the sub-action information 222 related to the second recorded information 200B and specifies the second posture model and the like at the timing of the sub-action "jump". Then, the UI control unit 152 displays the identified second posture model and the like in the display area 451B. If the playback button 411 is pressed in this state, the UI control unit 152 may start reproducing the motion of the posture model, etc. at the timing of the identified sub-action "jump". This allows the user to easily compare, for example, the player's posture and movement during the sub-action "jump" of serves performed on different dates and times. Note that if the timing of the frame images of the sub-action "jump" is aligned, the video may be played including "run-up", "toss", etc.

FIG. 21 is a diagram for explaining an example of selecting a posture model on the posture comparison UI 400C.

When the user presses the right sidebar 430 (see FIGS. 18 and 19) included in the posture comparison UI 400C, the UI control unit 152 displays a selection list 431 as shown in FIG. 21. In the selection list 431, information specified from the trajectory information 220 related to the recorded information 200 is displayed in a list format, as in the case of FIG. 16.

Furthermore, a check box 432 is displayed in each row of the selection list 431. The user sets the check box 432 of the line to be compared from the selection list 431 to ON.

The UI control unit 152 specifies the recorded information 200 for which the check box 432 is set to ON, and displays the posture model etc. generated from the posture model information 221 related to the specified recorded information 200 in the display area 434. This allows the user to display a desired posture model and the like in the display area 434 for comparison.

Additionally, a ball filtering button 433 may be displayed on the sidebar 430. When the ball filtering button 433 is pressed, the UI control unit 152 displays a condition box (not shown) to filter the recorded information 200 displayed in the selection list 431. In the condition box, at least one of ball height, speed, angle, and evaluation, which serve as filtering thresholds, may be input. Then, the UI control unit 152 displays the recorded information 200 that meets the conditions input in the condition box as a selection list 431. Thereby, the user can efficiently extract recorded information 200 to be a comparison candidate from a large amount of recorded information 200.

<Hardware configuration>
The embodiments according to the present disclosure have been described above in detail with reference to the drawings. Note that the functions of the first mobile terminal 20A, the second mobile terminal 20B, the display terminal 30, and the ball game video analysis device 100 described above can be realized by a computer program.

FIG. 22 is a diagram showing the hardware configuration of a computer that implements the functions of the ball game video analysis device 100 using a program.

This computer 1000A includes an input device 1001 such as a keyboard, mouse, or touch pad, an output device 1002 such as a display or speakers, a CPU (Central Processing Unit) 1003, a GPU (Graphics Processing Unit) 1004, a ROM (Read Only Memory) 1005, RAM (Random Access Memory) 1006, storage device 1007 such as HDD (Hard Disk Drive), SSD (Solid State Drive) or flash memory, DVD-ROM (Digital Versatile Disk Read Only Memory) or USB (Universal Serial Bus) memory, etc. A reading device 1008 that reads information from a recording medium, and a transmitting/receiving device 1009 that communicates by wire or wirelessly via a communication network N are provided, and each part is connected by a bus 1010.

Then, the reading device 1008 reads the program from the recording medium recording the program for realizing the functions of each of the above devices, and stores the program in the storage device 1007. Alternatively, the transmitting/receiving device 1009 communicates with a server device connected to the communication network N, and causes the storage device 1007 to store programs downloaded from the server device to implement the functions of each of the devices described above.

Then, the CPU 1003 copies the program stored in the storage device 1007 to the RAM 1006, and sequentially reads out and executes instructions included in the program from the RAM 1006, thereby realizing the functions of each of the devices described above.

FIG. 23 is a diagram showing the hardware configuration of a computer that implements the functions of the first mobile terminal 20A, the second mobile terminal 20B, and the display terminal 30 using programs. Note that the components already explained in FIG. 11 may be given the same reference numerals and the explanation will be omitted.

This computer 1000B includes an input device 1001, an output device 1002, a CPU 1003, a GPU 1004, a ROM 1005, a RAM 1006, a storage device 1007, a reading device 1008, a transmitting/receiving device 1009, and a camera device 1011, and each part is connected by a bus 1010.

The camera device 1011 includes an image sensor and generates captured images (moving images and still images). The generated captured image is stored in the storage device 1007. The captured image stored in the storage device 1007 may be transmitted to an external server or the like via the transmitting/receiving device 1009 and the communication network N.

Each functional block used in the description of the above embodiment is typically realized as an LSI, which is an integrated circuit. These may be integrated into one chip individually, or may be integrated into one chip including some or all of them. Although it is referred to as an LSI here, it may also be called an IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Furthermore, if an integrated circuit technology that replaces LSI emerges due to advancements in semiconductor technology or other derived technology, then of course the functional blocks may be integrated using that technology.

(Summary of this disclosure)
A ball game video analysis device (100) according to the present disclosure provides a first video (201A) and a second video that are taken from different positions of actions of a person (for example, a player) on a moving object (for example, a ball) used in a ball game. (201B), a trajectory analysis unit (105) that generates trajectory information (220) indicating the movement trajectory of the moving object, and information indicating the correspondence relationship between the first video, the second video, and the trajectory information. Displays a record management unit (133) that manages a plurality of record information (200) and a UI (User Interface) that allows the user to select record information based on trajectory information from the plurality of record information. It includes a display control unit (134) that displays a first moving image or a second moving image that has a corresponding relationship in recorded information.

According to this configuration, recorded information is selected from a plurality of pieces of recorded information managed by the recording management section based on the trajectory information, and a moving image that has a corresponding relationship in the selected recorded information is displayed. The video captures the actions of a person playing a ball game. Therefore, the user can easily find and view videos that are useful for ball game practice, such as videos of successful or unsuccessful actions, based on the trajectory information, from among a plurality of videos in which actions of people are shot.

Further, when the first recorded information and the second recorded information are selected from among the plurality of recorded information, the display control unit displays the first moving image or the second moving image that has a corresponding relationship in the first recorded information. The moving image and the first moving image or the second moving image that have a corresponding relationship in the second recorded information may be displayed side by side.

According to this configuration, the moving image related to the selected first recorded information and the moving image related to the selected second recorded information are displayed side by side. Therefore, the user can watch videos of the actions displayed side by side and use them to practice ball games.

The ball game video analysis device may further include a posture analysis unit (131) that generates posture model information (221) indicating the posture of the person during the action based on the first video and the second video. . The recorded information may be information indicating a correspondence relationship between the first moving image, the second moving image, trajectory information, and posture model information. When the first recorded information and the second recorded information are selected from among the plurality of recorded information, the display control unit displays a posture model ( 231A) and the posture model (231B) generated from the corresponding posture model information in the second recorded information may be displayed side by side or overlappingly.

According to this configuration, the posture model related to the selected first recorded information and the posture model related to the selected second recorded information are displayed side by side or overlappingly. Therefore, the user can view the posture models at the time of action displayed side by side or overlapping and use them for practicing a ball game.

The display control unit also displays a movement trajectory (241A) of the moving object generated from trajectory information that has a corresponding relationship in the first recorded information, and a movement trajectory (241A) of the moving object that is generated from trajectory information that has a corresponding relationship in the second recorded information. The movement trajectory (241B) of the body may be displayed together with the posture model.

According to this configuration, the movement trajectory of the moving object is also displayed together with the posture model. Therefore, the user can see the movement of the moving object during the action displayed together with the posture model, and can use it for practicing a ball game.

The ball game video analysis device also includes a sub-action analysis unit (132) that generates sub-action information (222) indicating the timing of occurrence of each of a plurality of sub-actions that constitute an action, based on trajectory information and posture model information. You can prepare even more. The recorded information may be information indicating a correspondence relationship between the first moving image and the second moving image, trajectory information, posture model information, and sub-action information. The display control unit displays a sub-action selection UI (420) from which one sub-action among a plurality of sub-actions can be selected, and displays a posture model at the occurrence timing of one sub-action selected through the sub-action selection UI. May be displayed.

According to this configuration, a posture model at the timing of occurrence of one sub-action selected through the sub-action selection UI is displayed. Therefore, the user can use the sub-action selection UI to display a posture model for a desired sub-action, which can be used to practice a ball game.

Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can come up with various changes, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims, and It is understood that it falls within the technical scope of the present disclosure. Further, each of the constituent elements in the embodiments described above may be combined as desired without departing from the spirit of the invention.

The technology of the present disclosure can be used to analyze sports using moving objects.

10 Ball game video analysis system 20A First mobile terminal 20B Second mobile terminal 30 Display terminal 100 Ball game video analysis device 101 Video reception unit 102 Image conversion unit 103 Reference frame identification unit 104 Reference frame change unit 105 Trajectory analysis unit 106 Result display Section 131 Posture analysis section 132 Sub-action analysis section 133 Record management section 134 Display control section 151 Information transmission/reception section 152 UI control section 200 Recorded information 200A First recorded information 200B Second recorded information 201A First moving image 201B Second Video 202A First frame image group 202B Second frame image group 203 Synchronization deviation information 204 Analysis result information 210A First image display area 210B Second image display area 211A, 211B Ball detection area 220 Trajectory information 221 Posture model Information 222 Sub-action information 231A First posture model 231B Second posture model 232A, 232B Neck joint positions 240A, 240B Ball 241A, 241B Ball movement trajectory 310 Reference frame confirmation UI
312 OK button 313 Change button 314 Redo button 320 UI for changing reference frame
322 OK button 323 Return button 324A, 324B 1 frame return button 325A, 325B N frame return button 326A, 326B 1 frame advance button 327A, 327B N frame advance button 330 Analysis result screen 331 Ball movement trajectory 400A Video comparison UI
400B Sub-action video comparison UI
400C posture comparison UI
400D sub action posture comparison UI
401A, 401B Selection list 402A, 402B Display area 403 Add button 410 Seek bar 411 Play button 412 Pause button 413 Slow play button 414 Repeat button 415 Arrange button 416 Enlarge button 417 Camera switch button 418 Image flip button 419 3D switch button 420 Sub action Selection buttons 421A, 421B Display area 430 Sidebar 431 Selection list 432 Check box 433 Ball filtering button 434 Display area 441 Video switching button 442 Trajectory display switching button 443 Ball display switching button 444 Player display switching button 445 Superimposition separation switching button 446 Arrangement button 447 Sub action selection button 451A, 451B Display area 1000 Computer 1001 Input device 1002 Output device 1003 CPU
1004 GPU
1005 ROM
1006 RAM
1007 Storage device 1008 Reading device 1009 Transmitting/receiving device 1010 Bus 1011 Camera device

Claims (7)

a trajectory analysis unit that generates trajectory information indicating a movement trajectory of the moving object based on a first video and a second video taken from different positions of a person's action on a moving object used in a ball game;
a recording management unit that manages a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information;
Displaying a UI (User Interface) for selecting first recorded information and second recorded information from among the plurality of recorded information based on the trajectory information, and in the first recorded information selected through the UI. A display control unit that displays the first moving image or the second moving image in a corresponding relationship, and the first moving image or the second moving image in a corresponding relationship in the second recorded information. Ta,
Ball game video analysis device. further comprising a posture analysis unit that generates posture model information indicating the posture of the person at the time of the action based on the first video and the second video,
The recorded information is information indicating a correspondence relationship between the first moving image and the second moving image, the trajectory information, and the posture model information,
When the first recorded information and the second recorded information are selected from among the plurality of recorded information, the display control unit generates posture model information from corresponding posture model information in the first recorded information. displaying the posture model generated from the posture model information corresponding to the second recorded information side by side or overlapping each other;
The ball game video analysis device according to claim 1. The display control unit is configured to display a movement trajectory of the moving body generated from corresponding trajectory information in the first recorded information and a movement trajectory generated from corresponding trajectory information in the second recorded information. displaying the movement trajectory of the body together with the posture model;
The ball game video analysis device according to claim 2 . further comprising a sub-action analysis unit that generates sub-action information indicating the timing of occurrence of each of a plurality of sub-actions constituting the action, based on the trajectory information and the posture model information,
The recorded information is information indicating a correspondence relationship between the first moving image and the second moving image, the trajectory information, the posture model information, and the sub-action information,
The display control section includes:
Displaying a sub-action selection UI that allows selection of one sub-action among the plurality of sub-actions;
displaying the posture model at the occurrence timing of one subaction selected through the subaction selection UI;
The ball game video analysis device according to claim 2 . A ball game video analysis system including a first mobile terminal with a camera, a second mobile terminal with a camera, a ball game video analysis device, and a display terminal,
The first mobile terminal generates a first video by photographing a person's action toward a moving object used in a ball game from a first position,
The second mobile terminal photographs the action of the person from a second position to generate a second video,
The ball game video analysis device includes:
Generating trajectory information indicating a trajectory of the moving object based on the first video and the second video,
managing a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information;
displaying on the display terminal a UI that allows selection of first recorded information and second recorded information from among the plurality of recorded information based on the trajectory information;
The first moving image or the second moving image that has a corresponding relationship in the first recorded information selected through the UI, and the first moving image or the second moving image that has a corresponding relationship in the second recorded information. displaying the video on the display terminal;
Ball game video analysis system. The device is
Generating trajectory information indicating a movement trajectory of the moving object based on a first video and a second video of a person's action on a moving object used in a ball game taken from different positions,
managing a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information;
Displaying a UI for selecting first recorded information and second recorded information from among the plurality of recorded information based on the trajectory information, and having a corresponding relationship in the first recorded information selected through the UI. displaying the first moving image or the second moving image and the first moving image or the second moving image that have a corresponding relationship in the second recorded information;
Ball game video analysis method. Generating trajectory information indicating a movement trajectory of the moving object based on a first video and a second video of a person's action on a moving object used in a ball game taken from different positions,
managing a plurality of recorded information that is information indicating a correspondence relationship between the first moving image and the second moving image and the trajectory information;
Displaying a UI for selecting first recorded information and second recorded information from among the plurality of recorded information based on the trajectory information, and having a corresponding relationship in the first recorded information selected through the UI. displaying the first moving image or the second moving image and the first moving image or the second moving image that have a corresponding relationship in the second recorded information;
A computer program that causes a computer to perform a process.

Images