JP6526731B2 - Information distribution device - Google Patents

Description

Embodiments of the present invention relates to information distribution equipment.

  Conventionally, a technology for distributing large-capacity content such as moving pictures to a terminal device is known (see, for example, Patent Document 1). Further, there is known a technique for generating a three-dimensional model of an object included in the image from a two-dimensional image (see, for example, Patent Document 2 and Non-Patent Document 1).

JP, 2013-115440, A JP, 2016-126425, A

Paul E. Debevec, Camillo J. Taylor, Jitendra Malik, Modeling and Rendering Architecture from A hybrid geometry- and image-based approach, "SIGGRAPH '96 Proceedings of the 23rd annual conference on computer graphics and interactive techniques", (United States of America ), 1996, P. 11-20.

Here, there is a need for real-time delivery of content to a large number of terminal devices. For example, in a stadium or the like where a soccer game is played, there is a need to distribute in real time the content related to the game to the terminal devices of the spectators watching the game. The content is, for example, a moving image of a viewpoint designated by the audience.
However, when individual information is distributed in real time to a large number of terminal devices, the traffic flowing through the communication line for information distribution becomes extremely large, and problems such as information distribution delay and information loss may occur.

The present invention aims to provide information distribution equipment which is capable of real time delivery of content.

One embodiment of the present invention can be obtained by performing image processing on a target image included in the video data based on a video data acquisition unit that acquires video data and the video data acquired by the video data acquisition unit. A motion information generation unit that generates motion information indicating a motion of a model for each of the models, performance information of a terminal device to which the motion information is to be distributed, or communication quality information indicating communication quality with the terminal device; A terminal information acquisition unit acquired as information, an adjustment degree information acquisition unit acquiring from the terminal device adjustment degree information indicating the degree of adjustment of the information amount of the motion information requested by the terminal device, and the terminal information acquisition unit and said terminal information but to obtain, based on said adjustment about information the adjustment about information acquiring unit acquires, to change the density of the motion information per the model A motion information amount adjustment unit that adjusts the information amount of the motion information generated by the motion information generation unit; and a distribution unit that distributes the information amount adjusted motion information adjusted by the motion information amount adjustment unit to the terminal device An information delivery apparatus comprising:

  Further, in the information distribution device according to one embodiment of the present invention, the model includes a plurality of elements and joints connecting the elements, and the motion information amount adjustment unit is included in the model. By adjusting the number of elements, the information amount of the motion information generated by the motion information generation unit is adjusted.

  The information distribution apparatus according to an embodiment of the present invention further includes a target image information acquisition unit for acquiring information on the target image, the distribution unit including the movement information after adjustment of the information amount of the target image. Information relating to the target image acquired by the target image information acquisition unit is distributed to the terminal device.

  Further, in the information delivery apparatus according to one embodiment of the present invention, the information on the target image is information indicating an attribute of the model to be delivered.

  Further, the information distribution apparatus according to the embodiment of the present invention is a movement for storing a movement information distribution request acquisition unit that acquires a movement information distribution request output from the terminal device, and the movement information generated by the movement information generation unit. The information processing apparatus further comprises an information storage unit, and the distribution unit is configured to receive the motion information stored in the motion information storage unit based on the motion information distribution request acquired by the motion information distribution request acquisition unit. The distribution request is distributed to the terminal that has output the request.

  Further, in the information distribution apparatus according to one embodiment of the present invention, the distribution unit performs the movement when the movement information after information amount adjustment has been distributed to the terminal device that has output the movement information distribution request. The motion information stored in the information storage unit is distributed to the terminal device.

According to the present invention, it is possible to provide information distribution equipment which is capable of real time delivery of content.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the outline | summary of a structure of the information delivery system of 1st Embodiment. It is a figure which shows an example of the image displayed on the display part of the terminal device of this embodiment. It is a figure which shows an example of the model which the information delivery apparatus of this embodiment produces | generates. It is a figure which shows an example of a function structure of the information delivery system of this embodiment. It is a figure which shows an example of operation | movement of the information delivery system of this embodiment. It is an example of the moving image displayed on the terminal device of this embodiment. It is a figure which shows an example of a function structure of the information delivery system of 2nd Embodiment. It is a figure which shows an example of the non-real-time delivery operation | movement by the information delivery apparatus of this embodiment. It is a figure which shows an example of arrangement | positioning of an imaging device in case several imaging devices are installed in the stadium.

First Embodiment
Hereinafter, the information delivery system 1 of the first embodiment will be described with reference to the drawings.
FIG. 1 is a diagram showing an outline of the configuration of the information distribution system 1 of the first embodiment.

[System overview]
The information delivery system 1 includes an information delivery apparatus 10. The information distribution device 10 is connected to the terminal device 20 via the network N.
The information distribution device 10 performs image processing on the video data VD captured by the imaging device CAM, and distributes the information after the image processing to the terminal device 20. Hereinafter, a case where the information distribution system 1 is provided in a stadium SD such as soccer and distributes information of a soccer match will be described as an example. In the present embodiment, the case where one imaging device CAM is provided will be described.

The terminal device 20 is, for example, a smartphone or a tablet terminal, and includes a display unit D and an operation unit TP. The operation unit TP includes, for example, a touch panel, and is operated by the user of the terminal device 20. The display unit D includes, for example, a liquid crystal display, and displays various images. The image displayed on the display unit D includes an image generated by the terminal device 20 based on the information distributed from the information distribution device 10.
Here, referring to FIG. 2, an example based on the information distributed by the information distribution device 10 will be described.

[One Example of Image Displayed on Terminal Device 20]
FIG. 2 is a view showing an example of an image P displayed on the display unit D of the terminal device 20 of the present embodiment.
The image P1 shown in FIG. 2 (A) is an image showing the trajectory of the movement of the player in the state of looking over the stadium SD from above.
The image P2 shown to FIG. 2 (B) is an image which shows arrangement | positioning of the player seen from a certain viewpoint. In the image P2, the player is deformed and displayed by a combination of relatively simple figures such as a ball and a cylinder.
The image P3 shown in FIG. 2 (C) is an image showing the arrangement of the players viewed from a certain viewpoint. In the image P3, the player is constituted by a combination of a large number of polygons (for example, polygons), and further shadowing based on the direction of light rays and texture mapping of the polygon surface are performed and displayed.
Here, the amounts of information of the images P2 and P3 are different from each other. The amount of information of an image here is determined by the size of the image, the resolution of the image, the number of gradations of color tone, and the like, and is determined by the number of motion vectors for generating a moving image. For example, the fact that the density of motion vectors used to generate a moving image is high and the frequency of update of motion vectors is high is referred to as "a large amount of information of image". In this example, since the image P3 has a larger number of polygons than the image P2, the player's motion can be expressed more smoothly. In this example, the density of motion vectors used to generate the image P3 is higher than the density of motion vectors used to generate the image P2. That is, the image P2 shown in this example has a relatively small amount of information. The image P3 has a relatively large amount of information. In the following description, an image having a relatively large amount of information is also referred to as a “fine-grained image”. In addition, an image with a relatively small amount of information is also referred to as a "grain size coarse image".

[Adjustment of Information Amount of Image Displayed on Terminal Device 20]
The granularity of the image P displayed on the display unit D of the terminal device 20 is determined by various factors. The factors include (1) image processing performance of the information distribution device 10, (2) image processing performance of the terminal device 20, and (3) communication quality between the information distribution device 10 and the terminal device 20, ((1) 4) The communication speed and the upper limit value of the communication amount based on the contents of the communication line contract of the terminal device 20, and (5) the request of the user of the terminal device 20, and the like. For example, when the image processing performance of the information distribution device 10 or the terminal device 20 is relatively high, the image P with a fine granularity can be displayed.

[Generation of video by model and model motion information]
The information distribution apparatus 10 performs image processing on the video data VD captured by the imaging device CAM to extract an image of a player included in the video data VD. Furthermore, the information distribution device 10 performs image processing on the image of the extracted player, and models the player's image by calculating the player's skeleton, center of gravity position, and the like by known means.
Here, if model information indicating a player's skeleton or the like is stored in advance in the terminal device 20, the terminal device 20 can generate a moving image of the model by giving motion information to the model information. .
In this case, the information distribution device 10 distributes the motion information of the model to the terminal device 20. The terminal device 20 generates a moving image of a model based on the distributed motion information and model information stored in advance, and displays the generated moving image on the display unit D.

FIG. 3 is a diagram showing an example of a model M generated by the information distribution device 10 of the present embodiment. This model M includes an element E and a joint J. In this example, the element E is, for example, the trunk, head, arms, and legs of the player. Further, in this example, the joint J is a joint portion connecting the trunk and the head, the trunk and the arms, the trunk and the legs, and the like.
In general, when modeling one player, the more the number of elements E and the number of joints J, the finer the player's motion can be expressed. In addition, as the number of elements E and the number of joints J increase, the amount of information of the model M increases.
For example, in the model MA shown in FIG. 3A, the number of elements E (or joints J) is 1, and the amount of information is the smallest. Model MB, model MC, and model MD shown to FIG. 3 (B)-(D) have many elements E and the joint J in a description order. The model MD has the largest amount of information in the illustrated model M. Hereinafter, the model MA is also referred to as a “point model”. The models MB to MD are also referred to as multi-joint models.

  The information distribution device 10 adjusts the amount of information of the model M and the amount of information of the motion information of the model M based on the various factors described above. Hereinafter, with reference to FIG. 4, an example of a specific functional configuration of the information distribution system 1 will be described.

[Functional Configuration of Information Distribution System 1]
FIG. 4 is a diagram showing an example of a functional configuration of the information distribution system 1 of the present embodiment. Hereinafter, the functional configuration of the terminal device 20 will be described, and then, the functional configuration of the information distribution device 10 will be described.

[Functional Configuration of Terminal Device 20]
The terminal device 20 includes a control unit 200 and a storage unit 250 in addition to the operation unit TP and the display unit D described above. The control unit 200 includes, for example, a CPU, and transmits and receives various operations and information. The control unit 200 includes a function detection unit 201, a terminal information transmission unit 202, a communication quality information generation unit 203, a motion information reception unit 204, a moving image generation unit 205, and a display control unit 206. Prepare as.

  The operation detection unit 201 detects a user's operation on the operation unit TP. In this example, the operation detection unit 201 detects a “motion information distribution request operation” by the user.

Communication quality information generation unit 203 generates communication quality information CQ. The communication quality information CQ is information indicating the communication quality between the information distribution device 10 and the terminal device 20. For example, when the radio wave condition between the information distribution device 10 and the terminal device 20 is bad, the communication quality information generation unit 203 generates communication quality information CQ indicating that the communication quality is bad.
Specifically, the communication quality information generation unit 203 determines the communication quality in consideration of the communication speed in communication with the information distribution apparatus 10, in some cases, the radio wave intensity, the number of times of occurrence of retransmission, and the like. The communication quality information generation unit 203 generates communication quality information CQ based on the determination result of communication quality.

The terminal information transmission unit 202 transmits the terminal information TI to the information distribution device 10. The terminal information TI includes the above-mentioned communication quality information CQ and performance information PI. Here, the performance information PI is the image processing performance of the terminal device 20. Here, the image processing performance is determined by the type and operating frequency of the arithmetic device provided in the terminal device 20, the access speed and capacity of the memory, the type and operating frequency of the image processing engine, and the like.
In this example, when the user performs a distribution request operation, the terminal information transmission unit 202 transmits the terminal information TI to the information distribution apparatus 10 together with the distribution request of the motion information.

  When transmitting a distribution request for movement information, the terminal information transmission unit 202 may request only movement information of a specific player or an athlete belonging to a specific team. In this case, the terminal information transmission unit 202 may request motion information based on the user's operation. For example, when the user performs an operation to specify a specific player, the terminal information transmission unit 202 requests only movement information of the player specified by the user. In addition, when the user performs an operation of specifying a specific team, the terminal information transmission unit 202 requests only movement information of players belonging to the team specified by the user.

  The user can also select the granularity of the displayed image (ie the granularity of the motion information to be delivered). In this case, the terminal information transmission unit 202 selects the information amount of the motion information to be requested according to the selection of the granularity by the user. For example, when the user selects display of an image having a coarse granularity like the image P2 described above, the terminal information transmission unit 202 requests motion information with a relatively small amount of information. In addition, when the user selects the delivery of an image with fine granularity, such as the image P3 described above, the terminal information transmission unit 202 requests motion information with a relatively large amount of information.

  Note that the terminal information transmission unit 202 may select the amount of motion information to be requested based on the communication quality information CQ and the performance information PI, not only by the instruction from the user. In this case, for example, when the communication quality indicated by the communication quality information CQ is relatively high, the terminal information transmission unit 202 requests motion information with a relatively large amount of information. Further, when the communication quality indicated by the communication quality information CQ is relatively low, the terminal information transmission unit 202 requests motion information with a relatively small amount of information.

  The storage unit 250 includes a model information storage unit 251. The model information storage unit 251 stores information of the element E of the model M and the joint J.

The motion information receiving unit 204 receives motion information distributed from the information distribution device 10.
The moving image generation unit 205 generates a moving image of the model M based on the motion information received by the motion information receiving unit 204 and the model information stored in the model information storage unit 251.
The display control unit 206 controls the display unit D, and causes the display unit D to display a moving image generated by the moving image generation unit 205.

  Here, “motion information” received by the motion information receiving unit 204 is motion information after the information distribution device 10 has adjusted the amount of information. That is, the motion information receiving unit 204 receives the motion information MIA after information amount adjustment from the information distribution device 10. The functional configuration of the information distribution device 10 will be described.

[Functional Configuration of Information Distribution Device 10]
The information distribution device 10 includes a control unit 100. The control unit 100 includes, for example, a CPU, and transmits and receives various operations and information. The control unit 100 includes a video data acquisition unit 101, a motion information generation unit 102, a terminal information acquisition unit 103, a motion information amount adjustment unit 104, and a distribution unit 105 as its functional units.

The video data acquisition unit 101 acquires video data VD from the imaging device CAM. The video data VD includes, for example, an image of a player in the game.
The motion information generation unit 102 performs image processing on the target image TGT included in the video data VD based on the video data VD acquired by the video data acquisition unit 101. In this example, the target image TGT is an image of a player.

  Specifically, the motion information generation unit 102 performs image processing on the video data VD to separate the player's image and the background image, and extracts the target image TGT, that is, the player's image from the video data VD. . The motion information generation unit 102 models the extracted target image TGT. When the image data VD includes images of a plurality of players, the motion information generation unit 102 extracts an image for each player and models each player. The motion information generation unit 102 analyzes the motion of the model M and generates motion information MI of the model M. The motion information MI is, for example, a motion vector of the model M. The motion vector is obtained by analyzing the player's motion between the plurality of frames included in the video data VD by a known means.

  The terminal information acquisition unit 103 acquires, from the terminal device 20, information for adjusting the amount of information of the motion information MI, that is, the terminal information TI.

  The motion information amount adjustment unit 104 adjusts the amount of information of the motion information MI generated by the motion information generation unit 102. As described above, the granularity of the image that can be displayed on the terminal device 20 is restricted by the image processing performance of the terminal device 20, the communication quality between the information distribution device 10 and the terminal device 20, and the like. The motion information amount adjustment unit 104 adjusts the information amount of the motion information MI by acquiring these constraint conditions from the terminal device 20.

There are the following two examples of a mechanism in which the motion information amount adjustment unit 104 adjusts the amount of information.
(1) The case where the motion information generation unit 102 generates motion information MI of various information amounts in parallel.
The motion information generation unit 102 generates, in parallel, motion information MI for models M with various amounts of information as shown in FIGS. 3A to 3D described above.
In this case, the motion information amount adjustment unit 104 selects, from among the plurality of pieces of motion information MI generated by the motion information generation unit 102, the motion information MI of the information amount satisfying the restriction condition acquired from the terminal device 20.
(2) When the motion information generation unit 102 generates motion information MI with a large amount of information.
Among the models M with various information amounts as shown in FIGS. 3A to 3D described above, the motion information generation unit 102 applies to the model M having a relatively large number of elements E and the number of joints J, Generate motion information MI. For example, the motion information generation unit 102 generates motion information MI for the model MD.
In this case, the motion information amount adjustment unit 104 thins out information from the motion information MI generated by the motion information generation unit 102 to generate motion information MI of the amount of information satisfying the constraint condition acquired from the terminal device 20.

  In any of the examples, the motion information amount adjustment unit 104 adjusts the information amount of the motion information MI generated by the motion information generation unit 102 by adjusting the number of elements E included in the model M. You may do so.

  Also, the motion information amount adjustment unit 104 may adjust the amount of information of the motion information MI based on the reproduction delay time at the terminal device 20. In this case, the terminal device 20 periodically transmits the reproduction time of the moving image generated based on the motion information MI to the information distribution device 10. The motion information amount adjustment unit 104 obtains the degree of deviation between the distribution time of the motion information MI and the reproduction time of the moving image generated based on the motion information MI. The motion information amount adjustment unit 104 adjusts the amount of information of the motion information MI according to the calculated degree of deviation. For example, when the deviation degree between the delivery time of the motion information MI and the reproduction time of the moving image is larger than a predetermined threshold, the motion information amount adjustment unit 104 thins out the motion information MI to obtain the information amount of the motion information MI. Reduce. In addition, the motion information amount adjustment unit 104 may further thin out the motion information MI when the degree of deviation between the distribution time of the motion information MI and the reproduction time of the moving image is larger than the predetermined threshold described above. In this case, the motion information amount adjustment unit 104 changes the degree of thinning of the motion information MI continuously or stepwise according to the degree of deviation between the distribution time of the motion information MI and the reproduction time of the moving image. Thus, the amount of motion information MI is reduced.

The motion information amount adjustment unit 104 outputs the motion information MI whose information amount has been adjusted, that is, the motion information adjusted information amount MIA to the distribution unit 105.
The distribution unit 105 distributes the information amount adjusted motion information MIA to the terminal device 20.

[Operation of information distribution system 1]
Next, the operation of the information distribution system 1 will be described with reference to FIG.
FIG. 5 is a diagram showing an example of the operation of the information distribution system 1 of the present embodiment.

(Step S110) The video data acquisition unit 101 acquires video data VD from the imaging device CAM. In this example, the imaging device CAM images the stadium SD at a viewpoint that looks over the entire stadium SD in a soccer game. In this case, the video data VD includes the images of all players participating in the game.
(Step S120) The motion information generation unit 102 generates motion information MI of the model M of the player based on the video data VD acquired by the video data acquisition unit 101.
In this example, the motion information generation unit 102 generates the motion information MI regardless of the presence or absence of the distribution request of the motion information from the terminal device 20. Also, in this example, the motion information generation unit 102 generates motion information MI for each of the models M shown in FIGS. 3 (A) to 3 (D).

(Step S210) The terminal device 20 transmits a distribution request for motion information to the information distribution device 10.
(Steps S130 to S140) The information distribution apparatus 10 waits for transmission of a distribution request from the terminal device 20. When the information delivery apparatus 10 receives a delivery request from the terminal device 20 (step S140; YES), the information delivery apparatus 10 performs the process of steps S150 to S170.

(Step S220) The terminal information transmission unit 202 of the terminal device 20 transmits the terminal information TI to the information distribution device 10.
(Step S150) The terminal information acquisition unit 103 of the information distribution device 10 acquires the terminal information TI from the terminal device 20.
In this example, the terminal device 20 is described as sequentially transmitting the motion information distribution request and the terminal information TI, but the present invention is not limited thereto. The terminal device 20 may simultaneously transmit a distribution request for motion information and the terminal information TI.

  (Step S160) The motion information amount adjustment unit 104 adjusts the amount of information of the motion information MI generated in step S120 based on the terminal information TI acquired in step S150. Specifically, when the terminal information TI requests the distribution of the “point model” instead of the “multi-joint model”, the motion information amount adjustment unit 104 generates each model M generated by the motion information generation unit 102. Among the motion information MI, the model MA shown in FIG. 3A, that is, the motion information MI of the point model is selected as the motion information MI to be transmitted. The motion information amount adjustment unit 104 outputs the selected motion information MI as the information amount adjusted motion information MIA to the distribution unit 105.

  The terminal information TI may include information for specifying the type (granularity) of the model to be distributed. Further, the motion information amount adjustment unit 104 may determine the type (granularity) of the model to be distributed based on the terminal information TI. In the case of the former configuration in which “the terminal information TI includes the information for specifying the type (granularity) of the model to be distributed”, the motion information amount adjustment unit 104 selects the model of the type specified by the terminal information TI. Choose In the case of the latter configuration in which the motion information amount adjustment unit 104 determines the type (granularity) of the distribution target model based on the terminal information TI, the motion information amount adjustment unit 104 includes the motion information generation unit 102. The type of the model to be distributed is selected by comparing the amount of information of the movement information MI of each model M that is generated with the amount of distributable information indicated by the terminal information TI.

(Step S170) The distribution unit 105 distributes the information amount adjusted movement information MIA to the terminal device 20.
(Steps S230 to S250) The motion information receiving unit 204 of the terminal device 20 receives the distributed information-adjusted motion information MIA. The moving image generation unit 205 generates a moving image of the model M based on the received information amount adjusted movement information MIA. The display control unit 206 causes the display unit D to display the generated moving image.

  After acquiring the distribution request for the movement information and the terminal information TI from the terminal device 20 in steps S130 and S150, the information distribution device 10 generates movement information after adjustment of the information amount MIA each time the movement information MI is generated. It distributes to the terminal device 20. That is, the information distribution device 10 distributes the information amount adjusted movement information MIA in real time. Here, in real-time distribution, the motion information MI is generated each time the information distribution apparatus 10 acquires the video data VD, and the information amount adjusted motion information MIA in which the information amount of the generated motion information MI is adjusted is distributed. It is.

FIG. 6 is an example of a moving image displayed on the terminal device 20 according to the present embodiment. The information distribution device 10 generates motion information MI each time the video data VD is acquired. The information distribution device 10 adjusts the amount of information of the movement information MI based on the terminal information TI acquired from the terminal device 20, and distributes the information amount after adjustment information movement information MIA in real time.
The figure shows, as an example, the case where the terminal device 20 requests movement information MIA after adjustment of the information amount of the “point model”.
At time t1, the information distribution device 10 distributes information indicating the current positions of the player PL1, the player PL2 and the player PL3 to the terminal device 20 as movement information after adjustment information amount MIA. The terminal device 20 displays an image P11 indicating the current position of the player PL1, the player PL2 and the player PL3 on the display unit D based on the distributed information-adjusted movement information MIA (FIG. 6A).
At time t2, the information distribution device 10 distributes the information indicating the movements of the player PL1, the player PL2 and the player PL3 to the terminal device 20 as movement-adjusted information movement information MIA. The terminal device 20 displays an image P12 indicating the movement locus and the current position of the player PL1, the player PL2 and the player PL3 on the display unit D based on the distributed information-adjusted movement information MIA (FIG. 6B). ).
Also at time t3, as in the case of time t2, the terminal device 20 displays an image P13 indicating the movement loci of the player PL1, the player PL2 and the player PL3 and the current position on the display unit D (FIG. ).
The above-mentioned information indicating the movements of the player PL1, the player PL2 and the player PL3 may be information indicating the direction and amount of displacement of the player from time t1 to time t2, and the player's at time t2 It may be information indicating a position.

As described above, the information distribution system 1 according to the present embodiment does not distribute moving images such as the video data VD to the terminal device 20 to be distributed as it is, but a model of a player included in the video data VD. Deliver M's motion information. Here, the amount of information of motion information of the model M is smaller than the amount of information of moving images such as the video data VD. For this reason, according to the information distribution system 1, it is possible to distribute a smaller amount of information than moving images such as the video data VD as they are.
Therefore, according to the information distribution system 1 of the present embodiment, content can be distributed in real time.
Here, the case where the user of the terminal device 20 is a spectator at the stadium and the information distribution device 10 distributes the information of this game will be described. In this case, the user of the terminal device 20 can view the game being deployed in front of the user in real time on the smartphone or tablet terminal at hand. The information distribution system 1 can display the position of the player, its movement trajectory, and the like on the terminal device 20 in real time, and can thus provide the user with information for further enjoying the game. Further, according to the information distribution system 1, the terminal device 20 can generate the content according to the preference of the user based on the motion information of the model M. For example, the specific player may be enlarged and displayed based on the terminal device 20 and the motion information of the model M to be distributed, or voice commentary based on the motion information may be performed. That is, according to the information delivery system 1, it is possible to provide the content according to the preference of the user.
Further, in the case of information distribution in a stadium or the like, the number of terminal devices 20 also increases according to the number of spectators, and in the case of the conventional technique of distributing video data VD as it is, communication of a wireless communication line used for information distribution. There is a problem that it is difficult to provide sufficient quality content because the capacity is tight. According to the information distribution system 1 of the present embodiment, since the communication amount of information distribution is small compared to the case of distributing the video data VD as it is, it is possible to reduce the extent to which the communication capacity of the wireless communication line is tight.

  Further, the information distribution apparatus 10 of the present embodiment adjusts the amount of information of the movement information MI based on the terminal information TI transmitted from the terminal device 20 to be distributed, and distributes the information amount adjusted movement information MIA. Therefore, according to the information distribution device 10, the amount of information of the motion information MI to be distributed can be reduced according to the performance of the terminal device 20 and the communication status between the information distribution device 10 and the terminal device 20. That is, according to the information distribution device 10, the communication amount of information distribution can be further reduced compared to the case of distributing the motion information MI without adjusting the amount of information.

  The terminal device 20 may update the communication quality information CQ in accordance with a change in the state of communication with the information distribution device 10, and repeatedly transmit the terminal information TI to the information distribution device 10. In this case, each time the information distribution device 10 receives the terminal information TI, the information distribution device 10 selects movement information of an amount of information according to the communication quality. That is, the information distribution device 10 may change the amount of information of the post-adjustment amount of movement information MIA to be distributed in real time.

  Moreover, in the above-mentioned, although it demonstrated that information, such as the element E of the model M and the joint J, was beforehand stored in the terminal device 20, it is not restricted to this. The information delivery apparatus 10 may deliver the information of the model M to be delivered to the terminal device 20 when the delivery of the motion information is requested.

Second Embodiment
Hereinafter, the information distribution system 1a of the second embodiment will be described with reference to the drawings.
FIG. 7 is a diagram showing an example of a functional configuration of the information distribution system 1a of the second embodiment. The information distribution system 1a has (1) a display function according to the state of the terminal device 20a, (2) a function of changing the amount of distribution information by user operation, (3) non-real time distribution function, (4) multi-angle motion information MI Alternatively, it differs from the first embodiment described above in that it has a distribution function of free viewpoint movement information MI, (5) a moving image display function according to a watching position, and (6) an additional information display function such as player information. Each function will be described below. The same components and operations as those of the first embodiment described above are denoted by the same reference numerals and description thereof is omitted.

[(1) Display Function According to Status of Terminal Device 20a]
In the terminal device 20a, the control unit 200a includes a moving image generation unit 205a and a state acquisition unit 207 as its functional units. The state acquisition unit 207 acquires the state of the terminal device 20a. Here, the state of the terminal device 20a includes the processing load of the terminal device 20a, the remaining battery capacity of the terminal device 20a, the brightness around the terminal device 20a, and the like.

[Change of display according to processing load of terminal device 20a]
The state acquisition unit 207 acquires the state of the processing load of the terminal device 20a. For example, when the control unit 200a is executing a plurality of processes simultaneously, the processing load may increase, and processing delay may occur in real time generation of a moving image. In such a case, the state acquisition unit 207 notifies the moving image generation unit 205a that the processing load of the control unit 200a has increased.
The moving image generation unit 205a generates a moving image of the model M according to the state based on the movement information MI received by the movement information reception unit 204a and the state acquired by the state acquisition unit 207. Here, the moving image generation unit 205a may change parameters that can be changed in the terminal device 20a, such as resolution. When the processing load of the control unit 200a increases as in the above-described example, for example, the moving image generation unit 205a reduces the resolution of the moving image to be generated and maintains real time generation of the moving image.

[Change of display according to the remaining battery capacity of the terminal device 20a]
The state acquisition unit 207 acquires the state of the remaining battery capacity of the terminal device 20a. For example, when the battery remaining amount of the terminal device 20a is lower than a certain threshold, the state acquisition unit 207 notifies the moving image generation unit 205a that the battery remaining amount of the control unit 200a is small.
When the battery remaining amount is low, the moving image generation unit 205a reduces the power consumption by reducing the particle size of the moving image to be generated, thereby extending the generation time of the moving image.

[Change of display according to the brightness around the terminal device 20a]
The state acquisition unit 207 acquires the illuminance (brightness) around the terminal device 20a. The moving image generation unit 205a generates a moving image according to the brightness acquired by the state acquisition unit 207. For example, the illuminance around the terminal device 20a may be different between when the soccer game is played in the daytime and in the nighttime. In this case, the moving image generation unit 205a makes the contents, color arrangement, font size, resolution, and the like of the moving image to be generated different between daytime and nighttime.

[(2) Function to change the amount of distribution information by user operation]
The information delivery system 1a can change the amount of information to be delivered by the operation of the user. For example, the user of the terminal device 20a may request distribution of a "point model" instead of a "multijoint model". In this case, the operation detection unit 201 detects an operation for requesting distribution of the “point model” by the user. The terminal information transmission unit 202a transmits adjustment degree information AI indicating the degree of adjustment of the information amount to the information distribution device 10a based on the user's operation. In this example, the terminal information transmission unit 202a transmits adjustment degree information AI indicating the “point model” to the information distribution device 10a.

The control unit 100a includes a motion information amount adjustment unit 104a, a distribution unit 105a, and an adjustment degree information acquisition unit 106 as its functional units.
The adjustment degree information acquisition unit 106 acquires the adjustment degree information AI transmitted by the terminal device 20a. The movement information amount adjustment unit 104 a adjusts the information amount of the movement information MI further based on the adjustment degree information AI acquired by the adjustment degree information acquisition unit 106. In the example described above, the motion information amount adjustment unit 104a selects the motion information MI of the “point model” as the motion information MI to be distributed. The movement information amount adjustment unit 104 a outputs the movement information MI of the “point model” to the distribution unit 105 a as the information amount adjusted movement information MIA.

[(3) Non real time delivery function]
Although the case where the information delivery apparatus 10 and the information delivery apparatus 10a perform real-time delivery has been described above as an example, the information delivery apparatus 10a can also perform non-real time delivery.

  Non-real time delivery is not to deliver motion information MI immediately after generation, but to temporarily store motion information MI, and to deliver motion information MI when there is a request from the terminal device 20a.

Specifically, the information distribution device 10a includes a storage unit 150. The storage unit 150 includes a motion information storage unit 151. The motion information storage unit 151 sequentially stores motion information MI generated by the motion information generation unit 102 in the order of generation. That is, the motion information storage unit 151 stores motion information MI having a large amount of information before the motion information amount adjustment unit 104a adjusts the amount of information.
The control unit 100 a includes a motion information distribution request acquisition unit 107. Upon acquiring the motion information distribution request MRQ from the terminal device 20a, the motion information distribution request acquisition unit 107 notifies the distribution unit 105a of acquisition of the motion information distribution request MRQ.
When the acquisition of the motion information distribution request MRQ is notified, the distribution unit 105a reads out the motion information MI stored in the motion information storage unit 151, and distributes the read out motion information MI to the terminal device 20a.

  The information delivery system 1a can deliver motion information MI having a larger amount of information than in the case of real-time delivery by performing non-real-time delivery when the wireless communication line used for information delivery is relatively open. it can.

  The information distribution device 10a can also selectively perform non-real-time distribution to the terminal device 20a that has been distributed in real time. In this case, when there is a request for non-real-time distribution from the terminal device 20a that is not performing real-time distribution, the information distribution device 10a rejects the non-real-time distribution request. An example of the non-real time delivery operation by the information delivery apparatus 10a will be described with reference to FIG.

FIG. 8 is a diagram showing an example of non-real time delivery operation by the information delivery apparatus 10a of the present embodiment.
(Step S170, Step S230 to Step S240) The information distribution system 1a performs the above-mentioned real time distribution. The information distribution device 10a stores a terminal device 20a to be distributed in real time.
(Step S410, Step S310) The terminal information transmission unit 202a of the terminal device 20a makes a distribution request of motion information by non-real time distribution. Specifically, the terminal information transmission unit 202a transmits the motion information distribution request MRQ to the information distribution device 10a. The motion information distribution request acquisition unit 107 of the information distribution device 10a acquires a motion information distribution request MRQ transmitted by the terminal device 20a.
(Step S320) The distribution unit 105a determines whether the motion information MIA after information amount adjustment has been distributed to the terminal device 20a that has transmitted the motion information distribution request MRQ, that is, whether real-time distribution has been performed. More specifically, the distribution unit 105a determines whether the terminal device 20a that has transmitted the motion information distribution request MRQ is the terminal device 20a stored as the terminal device 20a to be distributed in real time in step S170.
(Step S330) The distribution unit 105a distributes the motion information MI stored in the motion information storage unit 151 to the terminal device 20a.
(Step S420 to Step S450) When receiving the motion information MI, the terminal device 20a stores the motion information MI in the storage unit 250. The terminal device 20a sequentially reads the stored motion information MI to generate a moving image. The display control unit 206 causes the display unit D to display the generated moving image.
With such a configuration, the information distribution device 10a can give the user a motive to perform real-time distribution.

  In addition, although the case where the information delivery apparatus 10a delivers motion information MI was demonstrated as an example here, it is not restricted to this. In non-real-time distribution, after the information stored in the storage unit 150 of the information distribution device 10a is transferred to the storage unit 250 of the terminal device 20a, the information is read from the storage unit 250 and displayed. Therefore, real-time property is not required for transmission and reception of information between the information distribution device 10a and the terminal device 20a. For this reason, in the case of non-real time delivery, even with information having a large amount of information, it is possible to deliver with the storage capacity of the storage unit 150 or the storage capacity of the storage unit 250 as the upper limit. For example, in the case of non-real time delivery, the information delivery device 10a may deliver the video data VD to the terminal device 20a instead of or in addition to the motion information MI.

[(4) Delivery function of multi-angle motion information MI or free viewpoint motion information MI]
Although the case where one imaging device CAM has been described above has been described, the present invention is not limited to this. There may be a plurality of imaging devices CAM that supply the video data VD to the information distribution system 1a. For example, if there are three or more imaging devices CAM, the motion information MI can be generated from the video data VD.

FIG. 9 is a diagram showing an example of the arrangement of the imaging devices CAM when a plurality of imaging devices CAM are installed in the stadium SD. In the case of this example, eight imaging devices CAM of imaging devices CAM1 to CAM8 are installed at the stadium SD. In this case, the video data acquisition unit 101 of the information distribution device 10a acquires video data VD captured by the imaging device CAM, that is, video data VD1 to video data VD8. The motion information generation unit 102 generates motion information MI for each video data VD. With such a configuration, the information distribution device 10a can distribute multi-angle motion information MI.
For example, when the terminal device 20a requests distribution of the movement information MI based on the viewpoint position of the imaging device CAM2, the information distribution device 10a distributes the movement information MI based on the viewpoint position of the imaging device CAM2.

  The information distribution device 10a can also generate motion information MI in which the viewpoint position is an arbitrary position, that is, free viewpoint motion information MI, based on the video data VD at a plurality of viewpoint positions. In this case, the information distribution device 10a generates and distributes motion information MI based on the viewpoint position instructed by the terminal device 20a.

  In addition, although the case where the information delivery apparatus 10a performs coordinate transformation of a viewpoint position here was demonstrated, it is not restricted to this. The terminal device 20a may switch the viewpoint position. In this case, the information distribution device 10a distributes coordinate information indicating the position of the player on the stadium SD to the terminal device 20a together with the movement information MI. The moving image generation unit 205a of the terminal device 20a generates a free viewpoint moving image by performing coordinate conversion of the information distributed from the information distribution device 10a based on the viewpoint position information and the coordinate information instructed by the user.

[(5) Movie display function according to watching position]
The state acquisition unit 207 of the terminal device 20a acquires the current position of the terminal device 20a as a state. The motion information generation unit 102 of the information distribution device 10a performs coordinate conversion of the motion information MI with the current position of the terminal device 20a acquired by the state acquisition unit 207 as the viewpoint position. By configuring in this manner, the information distribution system 1a can cause the terminal device 20a to display a moving image according to the current position of the terminal device 20a, that is, the watching position of the user. Thereby, the information distribution system 1a can display a moving image according to the user's watching position without the user specifying the viewpoint position.
As described above, the moving image generation unit 205a of the terminal device 20a may perform coordinate conversion of the viewpoint position.

[(6) Additional information display function such as player information]
The information distribution system 1a can also distribute information related to the target image TGT in addition to the motion information MI. In this case, in the information distribution device 10a, the control unit 100a includes the target image information acquisition unit 108, and the storage unit 150 includes the target image information storage unit 152.
The target image information acquisition unit 108 acquires information on the target image TGT. Specifically, the motion information generation unit 102 extracts the target image TGT from the video data VD. The target image TGT is, for example, a full-length image of the player. The target image information acquisition unit 108 identifies a player extracted as the target image TGT based on information included in the extracted target image TGT, such as a design of a uniform, a profile number, a physical feature, and the like. The target image information acquisition unit 108 reads out information related to the specified player from the target image information storage unit 152, and supplies the information to the distribution unit 105a. The distribution unit 105a distributes the information on the target image TGT together with the motion information.

Here, the information related to the target image TGT includes, for example, in the case of a soccer game, information such as the player's name, face photograph, battle history, position, and the number of goals. Further, in the case of horse racing, the information related to the target image TGT includes information such as the horse pillar, the comment of the trainer, the state of the horse in the paddock, the odds, the current ranking in competition, and the like.
That is, the information related to the target image TGT is information indicating an attribute of the target image TGT.

  Further, the information related to the target image TGT may include identification information for identifying a player, a racehorse or the like. In this case, when the user selects an image such as a player or a racehorse, the information distribution system 1a selects the player's history or results based on the identification information of the target image TGT corresponding to the selected player or racehorse. Related information may be acquired. The information delivery system 1a can present the history and results of the player selected by the user to the user by distributing the acquired related information such as the history and results to the terminal device 20a.

  Further, the information related to the target image TGT may include information indicating a team, a position, and the like belonging to the player. In this case, the information distribution system 1a performs color-coded display for each team belonging to the target image TGT (for example, an image of a player) and display of a mark for identifying a position.

In addition, the target image TGT is not limited to the image of the player, and may be an image of equipment or tools used in the game. For example, the target image TGT may be a ball or goal in a soccer game, a referee, a card or a flag possessed by the referee, or the like. Further, the target image TGT may be a racehorse, a car, a gate, a starter on the gate, a flag possessed by the starter, a goal, an arrival display board, or the like, in addition to the jockey in horse racing. Further, the target image TGT may be a racing vehicle, a crew in a pit, a replacement tire, a flag on the side of the course, or the like in addition to the driver in the car race.
For example, when the target image TGT is a ball in a soccer game, the information distribution system 1a can distribute motion information MI indicating a movement trajectory of the ball.

  In addition, the terminal device 20a may display the moving image at timing after the timing at which the motion information MI is distributed by storing the distributed motion information MI. In this case, when the user performs a reproduction operation, the terminal device 20a reads the motion information MI and displays a moving image. In addition, the terminal device 20a may display a slide bar indicating the reproduction required time of the moving image and the reproduction position on the display unit D, and may display the moving image at any reproduction position by the operation of the slide bar by the user.

  Also, the information distribution system 1a may grasp the current position of the terminal device 20a, and select the terminal device 20a within a predetermined range as a distribution target. For example, the information distribution system 1a acquires the position information of the terminal device 20a of the base station of the mobile communication network, or the position information of the terminal device 20a acquired by GPS or near field communication. The information distribution system 1a determines, based on the acquired position information of the terminal device 20a, whether the terminal device 20a is a distribution target. For example, in the case where distribution is performed only to the user who is in the stadium where the soccer game is being performed, the information distribution system 1a distributes this terminal device 20a when the current position of the terminal device 20a is the corresponding stadium. Make it

  Further, when storing the motion information MI distributed by the terminal device 20a, the information distribution system 1a may control whether to delete the motion information MI stored in the terminal device 20a. In this case, for example, when the terminal device 20a moves out of the predetermined distribution range, the information distribution system 1a erases the movement information MI stored in the terminal device 20a.

  Further, when displaying a moving image based on the movement information MI, the information distribution system 1a may distribute the video data VD which is the original image of the movement information MI based on the user's operation. For example, the terminal device 20a stores the distributed motion information MI, and the user tries to reproduce a moving image based on the motion information MI. In this case, the user may desire to reproduce the video data VD which is the original image instead of the moving image based on the movement information MI. In this case, the information distribution system 1a distributes the video data VD which is the original image of the motion information MI based on the user's operation.

  While the embodiments of the present invention and the modifications thereof have been described above, these embodiments and the modifications thereof are presented as examples, and are not intended to limit the scope of the invention. These embodiments and their modifications can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

  Each of the above-described devices has a computer inside. The process of each process of each device described above is stored in a computer readable recording medium in the form of a program, and the above process is performed by the computer reading and executing this program. Here, the computer readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory or the like. Alternatively, the computer program may be distributed to a computer through a communication line, and the computer that has received the distribution may execute the program.

Further, the program may be for realizing a part of the functions described above.
Furthermore, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1 ... Information delivery system, 10 ... Information delivery apparatus, 100 ... Control part, 101 ... Video data acquisition part, 102 ... Motion information generation part, 103 ... Terminal information acquisition part, 104 ... Motion information amount adjustment part, 105 ... Delivery part 106 Adjustment level information acquisition unit 107 Motion information distribution request acquisition unit 108 Target image information acquisition unit 150 Storage unit 151 Motion information storage unit 152 Target image information storage unit 20 Terminal device 200: control unit 201: operation detection unit 202: terminal information transmission unit 203: communication quality information generation unit 204: motion information reception unit 205: moving image generation unit 206: display control unit 207: state Acquisition unit, 250 ... storage unit, 251 ... model information storage unit

Claims (6)

A video data acquisition unit for acquiring video data;
A motion information generation unit that generates, for each model, motion information indicating a motion of a model obtained by performing image processing on a target image included in the video data based on the video data acquired by the video data acquisition unit; ,
A terminal information acquisition unit that acquires, as terminal information, performance information of a terminal apparatus to which the motion information is to be distributed or communication quality information indicating communication quality with the terminal apparatus;
An adjustment degree information acquisition unit that acquires, from the terminal device, adjustment degree information indicating the degree of adjustment of the information amount of the motion information requested by the terminal device;
The motion information generation unit by changing the density of the motion information per model based on the terminal information acquired by the terminal information acquisition unit and the adjustment degree information acquired by the adjustment degree information acquisition unit A motion information amount adjustment unit that adjusts the information amount of the motion information generated by
A distribution unit that distributes, to the terminal device, movement information after adjustment of the amount of information adjusted by the movement information amount adjustment unit. The model includes a plurality of elements and joints connecting the elements,
The motion information amount adjustment unit
The information distribution device according to claim 1 , wherein the information amount of the motion information generated by the motion information generation unit is adjusted by adjusting the number of the elements included in the model. A target image information acquisition unit that acquires information on the target image;
The distribution unit, together with the information amount adjusted motion information of the target image, information about the target image the target image information acquisition unit acquires, according to claim 1 or claim 2 distributes to the terminal device Information distribution device. The information distribution device according to claim 3 , wherein the information related to the target image is information indicating an attribute of the model to be distributed. A motion information distribution request acquisition unit that acquires a motion information distribution request output from the terminal device;
A motion information storage unit for storing the motion information generated by the motion information generation unit;
The delivery unit
The motion information stored in the motion information storage unit is distributed to the terminal device that has output the motion information distribution request based on the motion information distribution request acquired by the motion information distribution request acquisition unit. The information delivery apparatus according to any one of claims 1 to 4 . The delivery unit
When the motion information after adjusting the amount of information has been distributed to the terminal device that has output the motion information distribution request, the motion information stored in the motion information storage unit is distributed to the terminal device The information delivery apparatus according to claim 5 .

Images