JP7765533B2 - Distribution device, distribution method, and program - Google Patents

Description

Article 30, Paragraph 2 of the Patent Act applies. Date of website publication: January 22, 2024. Website address: https://www.winticket.jp/

The present invention relates to a distribution device, a distribution method, and a program.

Patent Document 1 discloses a television station server and a judging device that distributes video footage of sports matches and provides users with the results of judging the merits and demerits of teams in the matches.

JP 2016-168258 A

Some sports and publicly managed games are difficult for beginners to understand due to complex rules, etc. However, there is a problem in that it is not possible to distribute video footage of these games so that even beginners can easily understand them.

In light of the above, the present invention aims to provide a distribution device, distribution method, and program that can distribute video footage of a sport so that even beginners can easily understand the sport.

One aspect of the present invention is a distribution device comprising: a tracking unit that tracks the position of one or more players for each player based on the coordinates of the player in a time series of frames of a video captured of the players competing in a stadium; a measurement unit that measures the movement speed of the player for each player based on the difference in the tracked positions of the players; an energy estimation unit that estimates the energy exerted by the player for each player based on the movement speed of the player; a physical fitness estimation unit that estimates the remaining physical fitness of the player for each player based on the cumulative value of the energy exerted by the player; an image processing unit that associates a physical fitness gauge image representing the remaining physical fitness of the player with each player and superimposes it on the time series of frames; and a communication unit that distributes the time series of frames on which the physical fitness gauge image is superimposed to one or more user terminals.

One aspect of the present invention is a distribution method executed by a distribution device, the distribution method including: a tracking step of tracking the position of one or more players for each player based on the coordinates of the player in a time series of frames of a video captured of the players competing in a stadium; a measurement step of measuring the movement speed of the player for each player based on the difference in the tracked positions of the players; an energy estimation step of estimating the energy exerted by the player for each player based on the movement speed of the player; a stamina estimation step of estimating the remaining stamina of the player for each player based on the cumulative value of the energy exerted by the player; an image processing step of associating a stamina gauge image representing the remaining stamina of the player with the player and superimposing it on the time series of frames; and a communication step of distributing the time series of frames on which the stamina gauge image is superimposed to one or more user terminals.

One aspect of the present invention is a program for causing a computer to execute the following steps: a tracking step for tracking the position of one or more players based on the coordinates of the players in time-series frames of a video captured of the players competing in a stadium; a measurement step for measuring the movement speed of the players for each player based on the difference in the tracked positions of the players; an energy estimation step for estimating the energy exerted by the players for each player based on the movement speed of the players; a stamina estimation step for estimating the remaining stamina of the players for each player based on the cumulative value of the energy exerted by the players; an image processing step for associating a stamina gauge image representing the remaining stamina of the players with the player and superimposing it on the time-series frames; and a communication step for distributing the time-series frames on which the stamina gauge image is superimposed to one or more user terminals.

This invention makes it possible to distribute video footage of the sport, making it easier for even beginners to understand the sport.

FIG. 1 is a diagram illustrating an example of the configuration of a distribution system in a first embodiment. FIG. 10 is a diagram showing an example of definition of the relative position of a player in the direction of travel in the first embodiment. FIG. 10 is a diagram showing an example of definition of the relative position of a player in a direction perpendicular to the direction of travel in the first embodiment. FIG. 10 is a diagram showing an example of the definition of coordinates of a stadium in the first embodiment. FIG. 10 is a diagram showing an example of definition of coordinates of a player running along a lane in a rectangular area of a stadium in the first embodiment. FIG. 10 is a diagram showing an example of definition of coordinates of a player running along a lane in a semicircular area of a stadium in the first embodiment. FIG. 10 is a diagram illustrating an example of a formation effect in the traveling direction in the first embodiment. FIG. 10 is a diagram illustrating an example of the formation effect in a direction perpendicular to the traveling direction in the first embodiment. FIG. 10 is a diagram showing an example of past positions of other players in the first embodiment. FIG. 10 is a diagram showing an example of the formation effect of a plurality of players in the first embodiment. FIG. 10 is a diagram showing a first example of the formation effect between players in the first embodiment. FIG. 10 is a diagram showing a second example of the formation effect between players in the first embodiment. FIG. 10 is a diagram showing an example of a stamina gauge image in the first embodiment. 10 is a flowchart illustrating an example of the operation of the distribution device in the first embodiment. FIG. 10 is a diagram showing a first example of a display of a stamina gauge image in the first embodiment. FIG. 10 is a diagram showing a second example of a display of a stamina gauge image in the first embodiment. FIG. 10 is a diagram showing a third example of a display of a stamina gauge image in the first embodiment. FIG. 10 is a diagram showing a fourth example of a display of a stamina gauge image in the first embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram showing an example of the configuration of a distribution system 1 in a first embodiment. The distribution system 1 is a system that distributes images (video) captured of athletes competing in a competition to a user terminal. The images may be moving images or one or more still images. The still images to be distributed may be one or more frames from a moving image (a time series of frames) to be distributed. In the following, the competition is not limited to a specific sport (e.g., soccer) or a publicly managed race (e.g., bicycle racing). In the first embodiment, the competition is bicycle racing as an example.

The distribution system 1 includes an imaging device 2, a relay device 3, a communication line 4, and a distribution device 5. One or more user terminals 6 are connected to the communication line 4 so as to be able to communicate with the distribution device 5.

The distribution device 5 includes a communication unit 51, a storage device 52, a decoding unit 53, a tracking unit 54, a measurement unit 55, an energy estimation unit 56, a physical strength estimation unit 57, an image processing unit 58, and an encoding unit 59. The user terminal 6 includes a terminal communication unit 61, a terminal decoding unit 62, and a terminal display unit 63.

Some or all of the functional units of the distribution device 5 and user terminal 6 are realized as software by a processor such as a CPU (Central Processing Unit) executing a program stored in a storage device having a non-volatile storage medium (non-transitory storage medium). The program may be recorded on a computer-readable storage medium. Examples of computer-readable storage media include portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as non-transitory storage media such as hard disks built into computer systems.

Some or all of the functional units of the distribution device 5 and the user terminal 6 may be realized using hardware including electronic circuits (electronic circuits or circuitry) using, for example, LSIs (Large Scale Integrated circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), etc.

The imaging device 2 generates a time series of video frames capturing one or more athletes (participating athletes) competing in the racetrack 100 (velodrome). The video frame rate is, for example, 30 frames per second. The imaging device 2 captures images of each athlete, for example, from before the start to after the finish. The imaging device 2 may also capture images of the race guides along with each keirin athlete. Multiple imaging devices 2 may be installed at the racetrack 100. The field of view of the frame (the captured range) can be estimated, for example, based on parameters of the imaging device 2 (for example, the orientation of the imaging device 2 and the imaging magnification).

The relay device 3 transmits the time-series frames of the captured video to the communication unit 51 using the communication line 4. The communication line 4 is, for example, the Internet. The distribution device 5 distributes images (for example, video (time-series frames)) captured of athletes competing in a competition to the user terminal 6.

The communication unit 51 acquires time-series frames of the captured video from the relay device 3. The time-series frames of the video acquired from the relay device 3 have been subjected to compression encoding processing by the relay device 3 in advance. The communication unit 51 records the acquired time-series frames of the video in the storage device 52.

The communication unit 51 acquires time-series frames on which each player's stamina gauge image is superimposed from the image processing unit 58 or the encoding unit 59. The encoding unit 59 has previously performed compression encoding processing on the time-series frames on which the stamina gauge image is superimposed. The communication unit 51 distributes the time-series frames on which the stamina gauge image is superimposed to one or more user terminals 6.

The decoding unit 53 performs a decoding process on the time-series frames of the video that have been compressed and encoded. The decoding unit 53 outputs the time-series frames of the video to the tracking unit 54. The tracking unit 54 tracks the position of each player based on the coordinates of the player in the time-series frames of the video in which one or more players competing in the stadium 100 are captured.

The measurement unit 55 measures the movement speed of each player based on the tracked player's position. For example, the measurement unit 55 measures the movement speed of each player based on the frame rate and the difference between the position on the stadium 100 corresponding to the coordinates of the player captured in the nth frame and the position on the stadium 100 corresponding to the coordinates of that player captured in the (n-1)th frame.

The energy estimation unit 56 estimates the energy exerted by each athlete based on the athlete's moving speed. The energy estimation unit 56 estimates the energy of each resistance force element acting on the moving object (= athlete + bicycle) as it moves. The energy (driving energy) exerted by the athlete is used to move the moving object (= athlete + bicycle). Four types of resistance force act on a moving moving object: acceleration resistance force, climbing resistance force, air resistance force, and rolling resistance force. Therefore, if the energy of each of these four types of resistance force can be estimated, the energy exerted by the athlete can be estimated.

Incidentally, at the speed of the moving object (= cyclist + bicycle) in a Keirin race, the Reynolds number of the air around the moving object is relatively large, so it is thought that the inertial resistance force, which is a type of air resistance, is dominant over the viscous resistance force. Therefore, as an example, in the following, we will assume that the inertial resistance force, which is proportional to the square of the moving object's speed, is the main air resistance force acting on the moving object.

If the speed of a moving object is constant, the energy of the acceleration resistance force is zero. It is possible to estimate the energy of the acceleration resistance force using an equation that represents the physical laws of kinetic energy. For example, it is possible to estimate the energy of the acceleration resistance force based on the difference between the speed of the athlete captured in the nth frame and the speed of the athlete captured in the (n-1)th frame. The mass of the moving object is determined in advance based on, for example, the weight listed in the athlete's profile and the weight listed in the bicycle specifications.

When a moving object is moving on a flat surface, the energy of the climbing resistance force (the force resisting gravity) is 0. It is possible to estimate the energy of the climbing resistance force using an equation that represents the physical law of potential energy. For example, it is possible to estimate the energy of the climbing resistance force based on the difference between the potential energy of a player captured in the nth frame and the potential energy of that player captured in the (n-1)th frame.

The energy of air resistance can be estimated using an equation that expresses the physical laws of air resistance. The air density and the frontal projection area (running posture) of the moving body may each be constant values that are predetermined as parameters used in estimating the energy of air resistance. The energy estimation unit 56 may also estimate the energy of air resistance taking into account the formation effect (wind-blocking effect). For example, the energy estimation unit 56 determines whether other players are running in a low-pressure space that has occurred directly behind a running player, based on the relative positions of the players. The energy estimation unit 56 reflects this determination result in the process of estimating the energy of air resistance.

A low-pressure space created directly behind a moving vehicle creates a suction force, which can pull other vehicles chasing that vehicle into the low-pressure space. This effect is called the formation effect (slipstream). In Keirin, the method in which riders ride in formation is called "drafting." By riding in formation, the riders behind experience less air resistance than normal. When a rider rides directly behind another rider, the effect of reducing the energy of air resistance is greatest. The further a rider is from directly behind another rider, the smaller the effect of reducing the energy of air resistance that rider experiences.

The energy estimation unit 56 determines whether there are other players in front of the player based on the coordinates of each player. If there are other players in front of the player, the energy estimation unit 56 estimates the direction and distance of the other players relative to the player. Here, to determine whether the direction in which the other players are located is in front of the player, the direction of travel of each player in the stadium 100 must be determined in advance. Therefore, by defining the general shape (approximate shape) of the stadium 100, the direction of travel of each player in the time-series frames is defined. In addition, the energy estimation unit 56 estimates the formation effect of the players based on the direction and distance of the other players.

The energy of rolling resistance can be estimated using an equation that represents the physical laws of friction resistance between the tires and the lane (road surface) of the stadium 100. The coefficient of friction due to the materials of the road surface and tires may be a predetermined constant value. The energy estimation unit 56 estimates the energy of rolling resistance for each athlete.

The physical strength estimation unit 57 estimates the amount of physical strength consumed per predetermined time based on the energy exerted by the player. For example, the physical strength estimation unit 57 estimates the amount of physical strength consumed per 1/30th of a second (the display time of one frame) based on the frame rate of the video. The physical strength estimation unit 57 estimates the remaining physical strength (amount of remaining energy) of each player based on the cumulative value of the energy exerted by the player (amount of physical strength consumed per hour). Here, the physique and basic physical strength of each player are determined in advance based on the profile and past performance of each player.

A rider's remaining stamina (remaining energy) is expressed as the result of subtracting the cumulative energy exerted by the rider from their maximum stamina (MAX HP). Maximum stamina is determined for each rider based on various factors such as past performance and tactics. Generally, the better a rider's performance, the stronger their maximum stamina. Also, because keirin is based on the concept of teams, a rider's maximum stamina (maximum energy) varies depending on their role on the team. Generally, younger riders have relatively strong maximum stamina, so they ride at the front of the pack, cutting through the wind. Veteran riders have relatively weaker maximum stamina, so they ride further back in the pack to conserve their remaining stamina. Veteran riders also make full use of tactics (techniques) such as blocking other teams.

The image processing unit 58 associates a stamina gauge image representing the player's remaining stamina with the player and superimposes it on the time-series frames. For example, the image processing unit 58 superimposes the player's stamina gauge image near the image of the player in the time-series frames. The encoding unit 59 performs compression encoding on the time-series frames of the video on which the stamina gauge image is superimposed. The encoding unit 59 outputs the time-series frames of the video on which compression encoding has been performed to the communication unit 51.

Next, the distribution device 5 will be described in detail.
The energy estimation unit 56 estimates the energy of each resistance force, including acceleration resistance force, climbing resistance force, air resistance force, and rolling resistance force, at a predetermined period (for example, frame period) based on equations that represent the physical laws of each resistance force.

When the moving object is accelerating, the acceleration resistance force is a positive value; when the moving object is moving at a constant velocity, the acceleration resistance force is 0; and when the moving object is decelerating, the acceleration resistance force is a negative value. The energy estimation unit 56 estimates the energy of the acceleration resistance force at the nth frame based on the difference between the kinetic energy at the nth frame and the kinetic energy at the (n-1)th frame. Here, the energy estimation unit 56 estimates the energy of the acceleration resistance force at the nth frame based on the mass "m" of the moving object (= athlete + bicycle), the velocity "V _sn " at the nth frame, and the velocity "V _sn-1 " at the (n-1)th frame. The athlete's mass in the estimation process is determined in advance based on the weight listed in the athlete's profile.

The energy estimation unit 56 estimates the energy of the climbing resistance force at the time of the nth frame based on the difference between the potential energy at the time of the nth frame and the potential energy at the time of the (n-1)th frame. The energy estimation unit 56 estimates the energy of the rolling resistance force at the time of the nth frame and the energy of the air resistance force at the time of the nth frame on a frame cycle. Here, the energy estimation unit 56 first estimates the energy of the air resistance force without taking the formation effect into account, and then estimates the final energy of the air resistance force based on the energy of the air resistance force thus estimated, taking the formation effect into account.

Regarding kinetic energy, the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. The wind speed " _vs " (unit: m/s) is set to a constant value as a numerical value based on the environment. The speed " _Vs " (unit: m/s) is set for each athlete based on the measurement results of the speed of the moving object. The symbol "n" represents the frame number in the time series. The kinetic energy is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the kinetic energy based on equation (1). The energy estimation unit 56 may use the moving average of the kinetic energy over the most recent 30 frames (1 second) as the kinetic energy of the current frame.

Regarding potential energy, the mass "m" (unit: kg), which corresponds to the athlete's body weight and bicycle weight, is set to a constant value associated with the athlete. The coefficient of gravitational acceleration "g" (unit: m/ ^s2 ) is set to a constant value of "9.8." The coordinates (x, y, z) are the measurement results of the athlete's coordinates in the stadium 100. Here, "z" represents the coordinate in the vertical direction (unit: m). The velocity " _Vs " (unit: m/s) is the measurement result of the speed of the moving object (= athlete + bicycle). The symbol "n" represents the frame number in the time series. The potential energy is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the potential energy based on equation (2). The energy estimation unit 56 may use the moving average of the potential energy over the most recent 30 frames (1 second) as the potential energy for the current frame.

Regarding the rolling resistance, the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value associated with the athlete. The coefficient of gravitational acceleration "g" (unit: m/ ^s2 ) is set to a constant value of "9.8." The coefficient of rolling resistance " _Ur " is set to a constant value of "0.004." The coordinates (x, y, z) are the measurement results of the athlete's coordinates in the stadium 100. The symbol "n" represents the frame number in the time series. The energy of the rolling resistance (the energy of the driving force of the moving object) corresponds to the result of multiplying the rolling resistance " _Ur mg" by the speed "_Vs" of the moving object per display time of one frame. The energy of the rolling resistance is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the energy of the rolling resistance based on equation (3). Here, the energy estimation unit 56 may determine the moving average of the energy of the rolling resistance force in the most recent 30 frames (1 second) as the energy of the rolling resistance force in the current frame.

Regarding air resistance (excluding the formation effect), the mass "m" (unit: kg) corresponding to the athlete's weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. The coefficient of air resistance " _Cd ", the frontal projection area of the moving object "A", the air density "R", and the wind speed " _vs " (unit: m/s) are each set to a constant value. The speed " _Vs " (unit: m/s) is the measurement result of the speed of the moving object (= athlete + bicycle). The symbol "n" represents the frame number in the time series. In bicycle racing, the main air resistance acting on a moving body is inertial resistance, which is proportional to the square of the speed. Therefore, the air resistance is proportional to the result of multiplying the square of the sum of the speed " _Vs " and the wind speed "vs _" per frame display time (" _Vs + _vs" ) by the air resistance coefficient " _Cd ", the air density "R", and the frontal projection area "A" of the moving body. The energy of the air resistance is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the air resistance (excluding the formation effect) based on equation (4).

The energy of the air resistance force (excluding the formation effect) corresponds to the product of the energy of the air resistance force (excluding the formation effect) multiplied by the speed " _Vs " of the moving object (= player + bicycle) per display time of one frame. The energy estimation unit 56 estimates the energy of the air resistance force (excluding the formation effect) based on equation (5).

Figure 2 is a diagram (bird's-eye view) showing an example of the definition of the relative positions of players in the direction of travel in the first embodiment. Figure 3 is a diagram (bird's-eye view) showing an example of the definition of the relative positions of players in the direction perpendicular to the direction of travel in the first embodiment. The formation effect is the effect of a low-pressure space (air current) occurring directly behind the moving body traveling at the front of the formation reducing the air resistance force on the moving body at the rear of the formation. When a low-pressure space occurs directly behind a moving body, surrounding moving bodies are drawn into that low-pressure space.

The measurement unit 55 measures the distance (relative distance) from the player to other players in the player's direction of travel "X." The measurement unit 55 also measures the distance (relative distance) from the player to other players in the direction "Y" (horizontal direction) perpendicular to the player's direction of travel. The energy estimation unit 56 estimates the formation effect that the player receives from other players based on the distance from the player to the other players. The energy estimation unit 56 also estimates the formation effect that the player receives from all other players based on the estimated formation effect that each other player has on the player.

Next, the method for estimating the platoon effect (slipstream) will be described in detail.
(A) Definition of the general shape (approximate shape) of the stadium (definition of the direction of movement of athletes)
4 is a diagram (bird's-eye view) showing an example of how coordinates of a stadium are defined in the first embodiment. In order to measure the distance in the player's forward direction and the distance to the player's side, it is necessary to measure the direction in which the player is moving on the stadium 100 based only on the coordinates in the captured frame. Therefore, the shape of the lanes of the stadium 100 is defined as being divided into straight lines and semicircles (corners).

In linear areas, the moving objects (= athletes + bicycles) move in a straight line. In semicircular areas, the moving objects (= athletes + bicycles) move in the tangent direction of the semicircle. The shape of the stadium 100 may also be determined based on transition curves.

For the coordinates (x_a, y_a) of player 101 and the coordinates (x_b, y_b) of player 102, the y coordinate is negative on the home stretch side of the stadium 100. The y coordinate is positive on the back stretch side of the stadium 100. The shape of the stadium 100 is divided into two semicircular areas and a rectangular area (-40≦x≦40).

(B) Measurement of the distance from one player to another (distance in the direction of travel "v", distance in the direction perpendicular to the direction of travel "h" (horizontal))
5 is a diagram (bird's-eye view) showing an example of the definition of coordinates of players running along a rectangular lane in a stadium in the first embodiment. The distance from player 101 to player 102 (another player) on the rectangular lane is measured based on the difference in coordinates. One of player 101 or player 102 may run on the rectangular lane, while the other runs on a semicircular lane.

Figure 6 is a diagram (bird's-eye view) showing an example of the definition of the coordinates of a player running along a semicircular lane on a stadium in the first embodiment. When either player 101 or player 102 is running along a semicircular lane, the tangent direction of the semicircular lane is defined as the direction of travel. In this case, distance is measured using the four basic arithmetic operations, using the dot product and cross product.

(C) Estimating the energy of the formation effect based on each distance The energy estimation unit 56 estimates the energy of the formation effect based only on the distance "v" in the direction of travel, and corrects this result based on the distance in the lateral direction. Basically, the longer the lateral distance "h" between one player and another player, the lower the formation effect.

Figure 7 is a diagram (side view) showing an example of the formation effect in the direction of travel in the first embodiment. Even if player 102 is positioned ahead of player 101, if the distance in the direction of travel is less than 0.5 m, it is unlikely that player 101 will be in a position where the formation effect can be obtained, even if the length of the bicycle (bicycle body length) is taken into account, and there will be no formation effect in the direction of travel (slipstream(v) = 0, (v < 0.5)).

When the distance in the direction of travel is greater than 0.5 cm and less than 3 m, air resistance decreases to 50%, so the formation effectiveness in the direction of travel is 50% (= 0.5) (slipstream(v) = 0.5, (0.5 <= v < 3.0)). When the distance in the direction of travel is 3 m or greater, the formation effectiveness in the direction of travel decreases gradually (slipstream(v) = 1/2/exp(v-3)^(1/4) (v >= 3.0)).

The formation effect based only on the direction of travel can be expressed as follows, for example:

For example, the platooning effectiveness in the direction of travel is 50% (maximum effect) at a distance of 0.5 vehicle lengths, 43% at a distance of 1 vehicle length in the direction of travel, 27% at a distance of 2 vehicle lengths in the direction of travel, and 10% at a distance of 4 vehicle lengths in the direction of travel.

Figure 8 is a diagram (bird's-eye view) showing an example of the formation effect in the direction perpendicular to the direction of travel (horizontal direction) in the first embodiment. When the horizontal distance "h" is between 0 and 0.15 m, the formation effect in the direction of travel is 100%. Therefore, being up to a distance of 0.15 m behind another player is considered to be the same as being directly behind that player (0 m from behind the other player).

When the lateral distance is 1m or more (h = 1.15m or more), there is no formation effect "slipstream(v)" in the direction of travel (slipstream(v, h) = 0, (h >= 1.0)). When the lateral distance is less than 1m (less than h = 1.15m), the longer the lateral distance, the less effective the formation in the direction of travel becomes (slipstream(v, h) = slipstream(v)*(1-h)^3, (0.0 <= h < 1.0)).

The formation effect that takes into account the lateral direction in addition to the formation effect in the direction of travel, i.e., the formation effect in the direction perpendicular to the direction of travel (lateral direction), can be expressed as, for example, in equation (9).

For example, when player 101 is positioned directly behind player 102, the horizontal formation effect is 50% (= 0.5) of the formation effect in the forward direction (maximum formation effect). Even if player 101 is positioned just 0.1 m to the side of player 102, the horizontal formation effect is 50% (maximum effect). When player 101 is positioned just 0.25 m to the side of player 102, the horizontal formation effect is 45%. When player 101 is positioned more than 1.15 m to the side of player 102, the horizontal formation effect is 0% (no formation effect).

(D) Improvements to ensure estimation accuracy even for semicircular lanes and lane slopes Even in cases where players are far apart on a semicircular lane (corner), the coordinates of player 102 (another player) at a second time earlier than the first time are taken into consideration in addition to the coordinates of player 102 at the first time relative to the coordinates of player 101 at the first time. This makes it possible to compare the trajectory of the position of player 102 with the trajectory of the position of player 101. In other words, it is possible to compare the relative positions of players in the past.

Figure 9 is a diagram (bird's-eye view) showing an example of the past positions of other players in the first embodiment. The tracking unit 54 tracks the position of each player for multiple frames, including past frames in a time series. In Figure 9, the player 102 designated by the symbol "102-(n-4)" represents the player 102 four frames in the past relative to the player 102 designated by the symbol "102-n."

The measurement unit 55 measures the distance (v0, v1, v2, v3, v4) in the direction of travel from player 101 to player 102 for a total of five frames, from the present (nth frame) to the past ((n-4)th frame). The measurement unit 55 measures the distance (h0, h1, h2, h3, h4) in the lateral direction from player 101 to player 102 for those five frames.

(E) Estimate the formation effect that a player receives from all other players based on the formation effect that a player receives from each other player (integrated processing)
10 is a diagram (bird's-eye view) showing an example of the formation effect of multiple players in the first embodiment. Even when the players in the formation are lined up in a single file, the formation effect decreases the further back in the formation. The formation effect of players other than the lead player is uniformly reduced by about 50%, with no significant difference between players. The energy estimation unit 56 estimates the energy of the air resistance force, including the formation effect, that a player receives from all other players for each frame (time), based on the energy of the air resistance force, including the formation effect, that a player receives from each of the other players.

Figure 11 is a diagram (bird's-eye view) showing a first example of the formation effect between players in the first embodiment. In this example, the formation effect decreases by 50% or more when player 101 is surrounded by multiple players 102.

Figure 12 is a diagram (bird's-eye view) showing a second example of the formation effect between riders in the first embodiment. Cases in which rider 101 is surrounded by multiple riders 102 are unlikely in keirin. Therefore, when multiple riders 102 are causing a formation effect on rider 101, the maximum formation effect of the multiple riders 102 is used as the integrated value of the formation effect received by rider 101 (slipstream_total = max(slipstream(v, h))). The energy estimation unit 56 multiplies the energy of the air resistance force (excluding the formation effect) by the formation effect coefficient "1 - slipstream_total" (unit: dimensionless).

The energy estimation unit 56 estimates the energy of the formation effect that player 101 receives from the current and past positions of player 102. For example, the energy estimation unit 56 estimates the energy of the formation effect based on a total of five frames, the current (0th frame) and past (1st to 4th frames), based on the distance in the forward direction (v0, v1, v2, v3, v4) based on a total of five frames, and the distance in the lateral direction (h0, h1, h2, h3, h4) based on a total of five frames.

Here, when the energy estimation unit 56 estimates the energy of the formation effect based on the distance "v" in the direction of travel based on past frames of the player 102, if any of the distances from "v1" to "v4" (for example, "v3") is a negative value, the energy estimation unit 56 estimates that there is no formation effect (0%), as shown in equation (10), even if the distance "v0" is a positive value.

The energy of the formation effect (slipstream(v3)) estimated based on, for example, distance "v3" in the direction of travel only may be corrected using the formation effect based on the lateral distance "h3" exemplified in equation (11).

When multiple players 102 are causing a formation effect on player 101, the maximum formation effect among the formation effects of those multiple players 102 is used as the integrated value of the formation effect received by player 101, as shown in equation (12).

In this way, the energy estimation unit 56 estimates the energy (amount of physical strength consumed) exerted by the athlete for each frame (time). The physical strength estimation unit 57 subtracts the energy estimated for each frame from the athlete's maximum physical strength (maximum energy) as time passes. The result of this subtraction is displayed on the terminal display unit 63 as a physical strength gauge image. Here, the maximum physical strength is determined in advance based on past performance, such as race scores (for example, results over the past four months).

Figure 13 is a diagram showing an example of a stamina gauge image (life gauge image) in the first embodiment. The stamina gauge image is an image that shows what percentage of maximum stamina (maximum energy) is remaining, with maximum stamina (maximum energy) being 100%. The length of the stamina gauge image may be normalized so that it is the same length for all riders. A standard maximum stamina may be determined for each rider based on measurements from past Keirin races. The difference in maximum stamina between riders may be determined based on race results.

Even after a Keirin race, riders have not yet used up all of their actual maximum physical strength. Therefore, maximum physical strength may be estimated for each rider based on their performance in past FTP (Functional Threshold Power) tests and past races.

A player's maximum physical strength (MAX HP) can be expressed, for example, as in equation (13) using the player's maximum physical strength deviation value. Here, the maximum physical strength deviation value can be expressed as in equation (14).

Next, an example of the operation of the distribution device 5 will be described.
FIG. 14 is a flowchart showing an example of the operation of the distribution device 5 in the first embodiment. The tracking unit 54 tracks the position of each player based on the player's coordinates in time-series frames of the video in which the player is captured (step S101). The measurement unit 55 measures the player's movement speed for each player based on the difference in the player's position (step S102). The energy estimation unit 56 estimates the energy exerted by each player based on the player's movement speed (step S103). The physical strength estimation unit 57 estimates the remaining physical strength of each player based on the cumulative value of the energy exerted by the player (step S104). The image processing unit 58 superimposes a physical strength gauge image on the time-series frames in association with the player (step S105). The communication unit 51 distributes the time-series frames on which the physical strength gauge image is superimposed to the user terminal 6 (step S106).

As described above, the tracking unit 54 tracks the position of each player based on the player's coordinates in time-series frames of a video of one or more players competing in a competition (race, soccer match, etc.) in the stadium 100. The measurement unit 55 measures the player's movement speed for each player based on the difference in the tracked player's position. The energy estimation unit 56 estimates the energy exerted by each player based on the player's movement speed. The stamina estimation unit 57 estimates the player's remaining stamina for each player based on the cumulative value of the energy exerted by the player. The image processing unit 58 associates a stamina gauge image representing the player's remaining stamina with the player and superimposes it on the time-series frames. The communication unit 51 distributes the time-series frames on which the stamina gauge image is superimposed to one or more user terminals 6.

In this way, a stamina gauge image is associated with each athlete and superimposed on a time-series frame of the video. This makes it possible to distribute video footage of the sport so that even beginners can easily understand the sport. Because the athlete's stamina is estimated based on the video footage, there is no need for the athlete to wear a wearable device to measure their stamina, and this does not affect the athlete's performance.

The image processing unit 58 may superimpose a stamina gauge image near the image of the player in the time-series frames. The image processing unit 58 may superimpose a stamina gauge image in a color associated with the player on the time-series frames.

The physical strength estimation unit 57 may estimate the maximum physical strength of a player based on at least one of the player's physique, weight, and past performance. The physical strength estimation unit 57 may estimate the player's remaining physical strength by subtracting the cumulative value of the energy exerted by the player from the player's maximum physical strength.

The tracking unit 54 may track the position of each player based on the coordinates of the player in a time series of frames of a video in which multiple players are captured. The energy estimation unit 56 may estimate the air resistance force for each player based on the positions of the multiple players in the formation and the player's movement speed. The energy estimation unit 56 may estimate the energy exerted by the player based at least on the air resistance force.

The tracking unit 54 may divide the stadium 100 into multiple areas according to its shape, and track the positions of players in each divided area. The pattern in which the tracking unit 54 divides the stadium 100 is not limited to a specific pattern. The tracking unit 54 may divide the stadium 100 into semicircular areas and rectangular areas, for example.

Next, a display example of a stamina gauge image (HP (Hit Point) gauge image) will be described.
FIG. 15 is a diagram showing a first example of the display of a stamina gauge image in the first embodiment. The device communication unit 61 acquires the distributed video (video of a bicycle race) from the communication unit 51. The device decoding unit 62 decodes the video distributed from the communication unit 51. The device display unit 63 displays the video distributed from the communication unit 51. The image processing unit 58 superimposes (adds) a stamina gauge image near an image of the player in a time-series frame. In FIG. 15 , the image processing unit 58 superimposes a stamina gauge image 201 near (within a predetermined distance from) a player who has been previously associated with the stamina gauge image 201. If the player is at the front of the formation, the image processing unit 58 may include the character string "It's windy" in the stamina gauge image 201.

Figure 16 is a diagram showing a second example of the display of a stamina gauge image in the first embodiment. In Figure 16, the image processing unit 58 superimposes a stamina gauge image 201 near the player who has been previously associated with the stamina gauge image 201. The image processing unit 58 superimposes a stamina gauge image 202 near the player who has been previously associated with the stamina gauge image 202. The image processing unit 58 superimposes a stamina gauge image 203 near the player who has been previously associated with the stamina gauge image 203. The image processing unit 58 may include the string "60%" in the stamina gauge image 203 of a player whose stamina gauge is 60% or less.

Figure 17 is a diagram showing a third example of the display of a stamina gauge image in an embodiment. In Figure 17, the image processing unit 58 superimposes (adds) a stamina gauge image 204 near the player who has been previously associated with the stamina gauge image 204. The image processing unit 58 superimposes a stamina gauge image 205 near the player who has been previously associated with the stamina gauge image 205. The image processing unit 58 superimposes a stamina gauge image 206 near the player who has been previously associated with the stamina gauge image 206. The image processing unit 58 may include the string "Conserve OK" in the stamina gauge image 204 of a player whose stamina gauge exceeds 60%. The same applies to the stamina gauge image 205 and the stamina gauge image 206.

Figure 18 is a diagram showing a fourth example of the display of a stamina gauge image in the first embodiment. In Figure 18, the image processing unit 58 superimposes the stamina gauge image 207 near the player who is pre-associated with the stamina gauge image 207. The image processing unit 58 superimposes the stamina gauge image 208 near the player who is pre-associated with the stamina gauge image 208. If the distance between the player who is pre-associated with the stamina gauge image 207 and the player who is pre-associated with the stamina gauge image 208 is less than a threshold value, and the difference in the stamina gauges (remaining stamina) of those players is equal to or greater than a predetermined difference, the image processing unit 58 may include the string "CHANCE!!" in the stamina gauge image 208.

Second Embodiment
In track events, athletes move along tracks (lanes) that are defined along the outer edge of a stadium (field). The second embodiment differs from the first embodiment mainly in that the event is not a track event. The second embodiment will be described focusing on the differences from the first embodiment.

As mentioned above, the competition is not limited to a specific sport or public competition. The competition may be other than track events, such as soccer, as mentioned above. The tracking unit 54 may divide a rectangular stadium into, for example, two rectangular areas, or into, for example, four triangular areas.

In soccer, fouls are more likely to occur when multiple players are crowded together in a small area on the outer edge of the field or near the goalposts, competing for the ball. Therefore, the closer the players are to each other in such a small area of the field, the more likely they are to tire. Furthermore, the longer players are crowded together, the more likely they are to tire.

In this way, not only the elements of a player's travel distance and travel speed (air resistance), but also other elements have a significant impact on a player's remaining physical strength (fatigue level). Therefore, the tracking unit 54 may track the positions of players moving within each divided area of a soccer or other sports field. The divided areas may be weighted. Small areas near the outer edge of the sports field or the goal posts may be weighted relatively more heavily than other areas.

The physical strength estimation unit 57 may estimate the remaining physical strength of a player for each divided area based on at least one of the player's travel distance, the player's travel distance, the density of multiple players, and a weighting (coefficient) set for the divided area.

The physical strength estimation unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players in areas with high weighting (for example, small areas close to the goal posts) greater than the cumulative value of energy exerted by players in areas with low weighting. The physical strength estimation unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players in areas with high player density greater than the cumulative value of energy exerted by players in areas with low player density. Furthermore, the physical strength estimation unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players who spend a long time crowded together greater than the cumulative value of energy exerted by players who spend a short time crowded together.

As described above, the physical strength estimation unit 57 estimates the remaining physical strength of a player for each divided area based on at least one of the player's movement distance, the player's movement distance, the density of multiple players, and the weighting (coefficient) set for the divided area.

This makes it possible to distribute video footage of sports other than track events so that they are easy for beginners to understand. Based on the video footage, the athlete's physical strength is accurately estimated by, for example, weighting each divided area. This eliminates the need for the athlete to wear a wearable device to measure their physical strength, and does not affect the athlete's performance.

The above describes in detail an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to this embodiment and includes designs that do not deviate from the gist of the present invention.

1...Distribution system, 2...Image capture device, 3...Relay device, 4...Communication line, 5...Distribution device, 6...User terminal, 51...Communication unit, 52...Storage device, 53...Decoding unit, 54...Tracking unit, 55...Measurement unit, 56...Energy estimation unit, 57...Stamina estimation unit, 58...Image processing unit, 59...Encoding unit, 61...Terminal communication unit, 62...Terminal decoding unit, 63...Terminal display unit, 100...Stadium, 101...Athlete, 102...Athlete, 201...Stamina gauge image, 202...Stamina gauge image, 203...Stamina gauge image, 204...Stamina gauge image, 205...Stamina gauge image, 206...Stamina gauge image, 207...Stamina gauge image, 208...Stamina gauge image

Claims (9)

a tracking unit that tracks the position of each of one or more athletes competing in a stadium based on coordinates of the athletes in time-series frames of a video in which the athletes are captured;
a measuring unit that measures the movement speed of each player based on the difference in the tracked positions of the players;
an energy estimation unit that estimates the energy exerted by each player based on the movement speed of the player;
a physical strength estimation unit that estimates the remaining physical strength of each player based on the cumulative value of energy exerted by the player;
an image processing unit that associates a stamina gauge image representing the remaining stamina of the player with the player and superimposes the image on the time series of frames;
a communication unit that distributes the time-series frames on which the stamina gauge image is superimposed to one or more user terminals. The distribution device according to claim 1, wherein the image processing unit superimposes the stamina gauge image near the image of the player in the time series of frames. The distribution device described in claim 2, wherein the image processing unit superimposes the stamina gauge image in a color associated with the player onto the time-series frames. The distribution device described in claim 1, wherein the physical strength estimation unit estimates the player's maximum physical strength based on at least one of the player's physique, weight, and past performance, and estimates the player's remaining physical strength by subtracting the cumulative value of the energy exerted by the player from the player's maximum physical strength. the tracking unit tracks the position of each player based on coordinates of the player in the time-series frames of a video in which a plurality of players are captured;
2. The distribution device according to claim 1, wherein the energy estimation unit estimates air resistance energy for each of the players based on the positions of the players constituting the formation and the movement speeds of the players, and estimates energy exerted by the players based on at least the air resistance energy. The distribution device according to claim 1, wherein the tracking unit divides the stadium into multiple areas according to the shape of the stadium and tracks the position of the player for each divided area. The distribution device according to claim 1, wherein the tracking unit divides the stadium into semicircular and rectangular areas. A distribution method executed by a distribution device,
a tracking step of tracking the positions of one or more athletes playing in a stadium based on coordinates of the athletes in time-series frames of a video captured of the athletes;
a measuring step of measuring a moving speed of each player based on a difference in the tracked positions of the players;
an energy estimation step of estimating the energy exerted by each player based on the movement speed of the player;
a physical strength estimation step of estimating the remaining physical strength of each of the players based on the cumulative value of the energy exerted by the players;
an image processing step of superimposing a stamina gauge image representing the remaining stamina of the player on the time series frames in association with the player;
a communication step of distributing the time-series frames on which the stamina gauge image is superimposed to one or more user terminals. On the computer,
a tracking step of tracking the positions of one or more athletes playing in a stadium based on coordinates of the athletes in time-series frames of a video captured of the athletes;
a measuring step of measuring a moving speed of each player based on a difference in the tracked positions of the players;
an energy estimation step of estimating, for each player, the energy exerted by the player based on the player's movement speed;
a physical strength estimation step of estimating the remaining physical strength of each player based on the cumulative energy exerted by the player;
an image processing step of superimposing a stamina gauge image representing the remaining stamina of the player on the time series frames in association with the player;
a communication procedure for distributing the time-series frames on which the stamina gauge image is superimposed to one or more user terminals.