JP5609147B2 - Video distribution system, relay device, and video distribution program - Google Patents

Description

  The present invention relates to a video distribution system, a relay device, and a video distribution program.

  Conventionally, real-time distribution has been performed in which video data is distributed in real time via a network. There is a relay device that temporarily accumulates video received from a server when performing such real-time distribution. This relay apparatus classifies the accumulated video into regions, reconfigures the quality of the accumulated video according to the classification result, and transmits the reconstructed video to the terminal. As described above, the quality is adjusted for each video area, reconstructed, and the reconstructed video is transmitted to the terminal, so that the terminal can display a video in which a specific area is selectively adjusted to high quality.

  However, when the network is in a congested state or when encoding is performed many times by the relay device, the quality of the video may be significantly degraded. For this reason, even if the quality of a specific area is selectively adjusted by the relay device, there is a limit to the degree of improvement. Therefore, the viewer who browses the video on the terminal cannot correctly understand the meaning of the video from the video reproduced in real time. In preparation for such a situation, the terminal collects video from the server afterwards by causing the server to retransmit the video with degraded quality in the video delivered in real time.

Japanese translation of PCT publication No. 2008-527810 International Publication No. 2003/84225 JP 2000-175016 A

  However, in the above-described prior art, when retransmitting real-time distributed video data, it is necessary to transmit the video data between the server, the terminal, and a relay device that relays these devices. If retransmission is performed retroactively to a server that is a video distribution source in this manner, there is a problem that the retransmission efficiency of video data is deteriorated as a result of squeezing the network and increasing the retransmission time.

  The disclosed technology has been made in view of the above, and provides a video distribution system, a relay device, and a video distribution program capable of improving retransmission efficiency of video data even when real-time distribution of video data is performed. With the goal.

  The video distribution system disclosed in the present application is a video distribution system including a relay device that relays video data distributed from a distribution source device to a distribution destination device. The relay device includes a video data storage unit that stores the video data. When the relay device receives the video data, the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device A reliability calculation unit for calculating. The relay device includes a storage control unit that stores the video data in the video data storage unit when the reliability of the video data calculated by the reliability calculation unit is a predetermined threshold or more. The relay apparatus includes a transmission control unit that transmits the video data to another relay apparatus or the distribution destination apparatus. When the reliability of the video data calculated by the reliability calculation unit is equal to or greater than a threshold, the transmission control unit transmits the video data together with address information of the own device, and the reliability of the video data is a threshold The video data is transmitted if it falls below.

  According to one aspect of the video distribution system disclosed in the present application, it is possible to improve the retransmission efficiency of video data even when performing real-time distribution of video data.

FIG. 1 is a block diagram illustrating a configuration of a relay device included in the video distribution system according to the first embodiment. FIG. 2 is a system configuration diagram of the video distribution system according to the second embodiment. FIG. 3 is a block diagram of the configuration of the receiving apparatus according to the second embodiment. FIG. 4 is a block diagram illustrating the configuration of the transcoder according to the second embodiment. FIG. 5 is a block diagram illustrating the configuration of the router according to the second embodiment. FIG. 6 is a block diagram of the configuration of the playback apparatus according to the second embodiment. FIG. 7 is a flowchart illustrating a procedure of distribution processing according to the second embodiment. FIG. 8 is a flowchart illustrating the procedure of the relay process according to the second embodiment. FIG. 9 is a flowchart illustrating the procedure of the reproduction process according to the second embodiment. FIG. 10 is a flowchart of the retransmission request processing procedure according to the second embodiment. FIG. 11 is a flowchart illustrating a procedure of relay point retransmission processing according to the second embodiment. FIG. 12 is a flowchart illustrating a procedure of a distribution source retransmission process according to the second embodiment. FIG. 13 is an explanatory diagram for explaining a transmission destination to which a distribution destination apparatus requests retransmission. FIG. 14 is a diagram illustrating an example of video data distributed by the encoder. FIG. 15 is a diagram illustrating an example of video data that has been interpolated by error correction by a decoder. FIG. 16 is a diagram illustrating an example of video data before error correction is performed by the decoder. FIG. 17 is a schematic diagram illustrating an example of a computer that executes a video distribution program according to the third embodiment.

  Embodiments of a video distribution system, a relay device, and a video distribution program disclosed in the present application will be described below in detail with reference to the drawings. Note that this embodiment does not limit the disclosed technology.

  First, the video distribution system according to the first embodiment will be described. FIG. 1 is a block diagram illustrating a configuration of a relay device included in the video distribution system according to the first embodiment. As shown in FIG. 1, the relay device 1 includes a video data storage unit 2, a reliability calculation unit 3, a storage control unit 4, and a transmission control unit 5. The relay device 1 is included in a video distribution system that distributes video data from a distribution source device to a distribution destination device.

  Among these, the video data storage unit 2 stores video data. When the video data is received, the reliability calculation unit 3 determines the video data reliability indicating the degree to which the quality of the video data has changed due to the network status between the distribution source device or another relay device and the own device. calculate. In addition, the storage control unit 4 stores the video data in the video data storage unit 2 when the reliability of the video data calculated by the reliability calculation unit 3 is equal to or greater than a predetermined threshold.

  The transmission control unit 5 transmits the video data to another relay device or a distribution destination device. When the reliability of the video data calculated by the reliability calculation unit 3 is equal to or greater than the threshold, the transmission control unit 5 transmits the video data together with the address information of the own device, and the reliability of the video data falls below the threshold. If it is, video data is transmitted.

  As described above, the relay device 1 according to the present embodiment transmits the video data to another relay device or a distribution destination device regardless of the reliability value of the video data. Therefore, the distribution destination device transmits the video data. Receive in real time. Furthermore, when the reliability of the video data is equal to or higher than a predetermined threshold, the relay device 1 according to the present embodiment transmits the video data together with the address information of the relay device to another relay device or a distribution destination device. For this reason, the distribution destination apparatus can retransmit the video data to the nearest relay apparatus in the address information received from the relay apparatus 1, and there is no need to retransmit back to the distribution source apparatus. Further, since the relay device 1 according to the present embodiment stores the video data in the video data storage unit 2 when the reliability of the video data is equal to or higher than a predetermined threshold value, the video data with degraded video quality is delivered. It is not retransmitted to the previous device. Therefore, according to the relay device 1 according to the present embodiment, it is possible to improve the retransmission efficiency of the video data while ensuring real-time delivery of the video data.

  Next, a video distribution system according to the second embodiment will be described. Here, the system configuration of the video distribution system will be described, the configuration of each device included in the video distribution system will be described, and then the processing flow of the video distribution system will be described.

[System configuration]
FIG. 2 is a system configuration diagram of the video distribution system according to the second embodiment. As shown in FIG. 2, the video distribution system accommodates a receiving device 20, a transcoder 10, a router 30 </ b> A, a router 30 </ b> B, and a playback device 40. In the example shown in FIG. 2, it is assumed that video data is distributed from the receiving device 20 to the playback device 40 in real time. In the example of FIG. 2, the case where one transcoder is accommodated and two routers are exemplified, but it is sufficient that at least one of the transcode and the router is accommodated. Can accommodate multiple transcoders and routers.

  The receiving device 20 and the transcoder 10, the transcoder 10 and the router 30 </ b> A, and the router 30 </ b> B and the playback device 40 are connected so as to be communicable by wire or wirelessly. The router 30A and the router 30B are connected to each other via a network 50 so as to communicate with each other. The network 50 may be any type of communication network such as the Internet, a LAN (Local Area Network), or a VPN (Virtual Private Network).

  Among these, the receiving device 20 is a device that receives video data from the camera 200, and can be a server that distributes the video data. The receiving device 20 encodes the video data input from the camera 200 and distributes the video data to the playback device 40 in real time. The camera 200 may be an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

  The transcoder 10 is a device that transcodes video data distributed from the receiving device 20. The transcoder 10 functions as a relay device that relays video data from the receiving device 20 to the playback device 40.

  The transcoder 10 can transcode data into a data format required by the playback device 40 that is the distribution destination of the video data. For example, it is assumed that the receiving device 20 encodes video data for moving picture experts group phase (MPEG) 2 for a personal computer. At this time, if the distribution destination device is a terminal accommodated in a communication network having a smaller transmission path capacity than the Internet, such as a mobile phone or PHS (Personal Handyphone System), transcoding from MPEG2 to MPEG4 is performed. To do. As a result, the image can be made compact in accordance with the distribution destination apparatus.

  The router 30A and the router 30B are communication devices that connect networks using a routing table (not shown). The router 30A and the router 30B function as a relay device that relays video data from the receiving device 20 to the playback device 40.

  The playback device 40 is a device that plays back video data distributed from the receiving device 20 on the display unit 400. As an example, a mobile terminal such as a mobile phone or a PHS can be employed in addition to a fixed terminal such as a personal computer or a digital television. The display unit 400 can be applied to all display devices such as a monitor, a display, and a touch panel.

[Configuration of Receiving Device 20]
FIG. 3 is a block diagram of the configuration of the receiving apparatus according to the second embodiment. 3 includes an image analysis unit 21, an evaluation score calculation unit 22a, a threshold setting unit 22b, a group information generation unit 23, an encoding unit 24, a transmission unit 25a, and a reception unit. 25b. Furthermore, the receiving device 20 includes an image storage unit 26a, a storage control unit 26, a retransmission control unit 27, and an image erasing unit 28.

  Among these, the image analysis unit 21 is a processing unit that divides images input from the camera 20 into groups and analyzes the images for each group. As an example, the image analysis unit 21 is a unit that can divide an image stream, and a set of P pictures (Predictive picture) and B pictures (Bi-directional predictive) reproduced by one I picture (Intra picture). One group. When a plurality of images are grouped in this way, an ID (identifier) that uniquely identifies the group is assigned to the group. Hereinafter, a series of images included in a group is referred to as a group image.

  The image analysis unit 21 analyzes an image from components such as luminance, frequency, and motion vector (vector) of images included in one group. As an example, the image analysis unit 21 detects a background portion and a human portion included in an image by detecting a portion where the change in luminance of the image is larger than a predetermined value as an edge component. To do. Further, the image analysis unit 21 detects a portion where the change in shading is fine and a portion where the change in shading is rough depending on whether the frequency of the image is a high frequency component or a low frequency component. In addition, the image analysis unit 21 divides a motion vector between a plurality of frames into a portion where a motion vector is detected and a portion where there is no difference and a motion vector is not detected. Such image analysis can be realized by using a function of a known encoder.

  The evaluation score calculation unit 22a is a processing unit that calculates an evaluation score for each group using an external parameter used by the encoding unit 24 described later. Here, the “evaluation point” is an index for evaluating the quality of an image. For example, the evaluation score includes the degree of change in image quality before and after encoding or transcoding. Also, the evaluation score includes the degree of change in image quality due to the network status between devices when an image is transmitted between the device on the distribution source side and the device itself. In the case of the receiving device 20, the receiving device 20 is a device from which video data is distributed, and an unencoded image serving as a distribution original is supplied from the camera 200. For this reason, in the case of the receiving device 20, it is only necessary to evaluate the degree to which the image quality changes before and after encoding.

  For example, the evaluation score calculation unit 22a can calculate an evaluation score using a bit rate as an external parameter. As an example, the evaluation score calculation unit 22a calculates an evaluation score by substituting a known value into the evaluation score calculation formula (1) “evaluation score = original evaluation score * β * (α * compression ratio)”. To do.

  Here, the compression rate is obtained by dividing the bit rate of the compressed image by the bit rate of the original image. “Α” indicates a degree of resistance against deterioration of a specific type of image defined for each encoder vendor. This tolerance “α” is defined by “0 ≦ α ≦ 1”, and indicates that the larger the numerical value, the less likely the deterioration occurs. Further, “β” indicates a tolerance for deterioration defined for each type of subject of the image. This tolerance “β” is defined by “0 ≦ α ≦ 1”, and the greater the value, the greater the tolerance. Note that the original evaluation score is an evaluation score of the image when it is input to the evaluation score calculation unit 22a, and the image input to the receiving device 20 is an image before encoding that is an original of distribution. The evaluation score is a perfect score.

  As described above, when the evaluation score is calculated using the bit rate, the evaluation score lower than the original evaluation score is calculated as the degree to which the bit rate after encoding decreases from the bit rate before encoding is larger. This is because as the compression ratio increases, the image becomes rough and the quality of the image deteriorates. Also, an evaluation point closer to the original evaluation point is calculated as the change between the bit rate after encoding and the bit rate before encoding is smaller. For example, if the compression rate is 0%, the calculated evaluation score is the same as the original evaluation score.

  Also, the reason why the tolerance “α” is used in the evaluation point calculation formula (1) is because the performance of the encoder differs depending on the vendor. For example, the encoder provided by vendor A has a high degree of resistance for weather observation images, but has a low degree of resistance for traffic monitoring images. The encoder provided by the vendor B has a low tolerance for weather observation and traffic monitoring images, but has a high tolerance for astronomical observation images. In this way, different vendors have different encoder performance. For this reason, if the evaluation score is calculated using the tolerance “α”, an accurate evaluation score can be calculated in evaluating the image even when the performance of the encoder varies.

  Further, the reason why the tolerance “β” is used in the evaluation point calculation formula (1) is that the importance for the viewer differs depending on the type of subject of the image. As an example, a case is assumed in which a viewer monitors illegal activities with an image obtained by capturing the periphery of an ATM (automated teller machine). In this case, the importance for the viewer is greatly different between the case where the subject is a person and the case where the subject is the background. In other words, the appearance of the person and the actions that the person is performing are important, and the background itself is meaningless. Thus, the tolerance for background degradation is greater than the tolerance for human degradation. Therefore, if the evaluation score is calculated using the tolerance “β”, an accurate evaluation score can be calculated in evaluating the image even when the importance for the viewer varies depending on the type of subject of the image. Note that, when the tolerance “β” is used here, the result of classifying the person and the background by the image analysis unit 21 can be used.

  Further, the evaluation score calculation unit 22a can calculate an evaluation score using a frame rate as an external parameter. As an example, the evaluation score calculation unit 22a calculates an evaluation score by substituting a known value into the evaluation score calculation formula (2) “evaluation score = original evaluation score * frame rate change rate * γ”. The frame rate change rate is obtained by dividing the frame rate after encoding by the frame rate before encoding. “Γ” indicates the intensity of image movement. This “γ” is defined by “0 ≦ γ ≦ 1 * (1 / reciprocal of frame rate change rate)”, and the larger the value, the less the movement.

  As described above, when the evaluation score is calculated using the frame rate, the evaluation score lower than the original evaluation score is calculated as the degree to which the frame rate after encoding decreases from the frame rate before encoding is larger. This is because as the rate of change in frame rate increases, the number of images per unit time decreases and the image quality deteriorates. Also, the smaller the change between the frame rate after encoding and the frame rate before encoding, the closer to the original evaluation score is calculated. For example, if the frame rate change rate is 0%, the calculated evaluation score is the same as the original evaluation score. If the frame rate after encoding is lower than before encoding, the playback time is set longer than before encoding and the video is played without changing the number of frames before encoding, and the playback time before and after encoding are changed. Without reducing the number of frames.

  The reason why the intensity of motion “γ” is used in the evaluation point calculation formula (2) is that the portion where the image moves is important for the viewer. For example, in the case of an image such as a color bar that has no change between frames, the amount of information given to the viewer is equal to zero, and only needs to be noticed by the viewer when there is movement. From this, it is defined that the value of the intensity of motion “γ” is determined so as to approach the original evaluation point as the motion of the image decreases. In other words, in the case of an image that does not move between frames, it becomes “1 / reciprocal of frame rate change rate”, and the original evaluation score no matter how many frames are thinned out or the reproduction time is extended. Is obtained as a calculation result. As a result, the evaluation score, which is a criterion for retransmission, is prevented from being lowered, and an image having no motion between frames is not retransmitted. Here, when the tolerance “γ” is used, the detection result of the motion vector by the image analysis unit 21 is used.

  As described above, the evaluation score calculation unit 22a calculates the evaluation score using the evaluation score calculation formula (1) and the evaluation score calculation formula (2), and thereby the degree of change in image quality before and after encoding is performed. To evaluate. Here, the evaluation score is calculated by substituting the parameters into the two evaluation score calculation formulas, but the evaluation score may be calculated using only one of them. Also, the order of assignment to the evaluation score calculation formula may be any of the formulas before or after, and the calculation result of one calculation formula is substituted as the “original evaluation score” into the other calculation formula. The evaluation point calculation unit 22a can also perform statistical processing such as arithmetic mean or weighted average on the calculation result of the evaluation point calculation formula (1) and the calculation result of the evaluation point calculation formula (2).

  The threshold setting unit 22b is a processing unit that sets a threshold for comparison with an evaluation score for each group using a threshold setting table (not shown). This threshold value setting table is a table in which the degree of complexity of an image and a threshold value are stored in association with each other. The more complex the image is, the higher the threshold value is defined.

  Here, the “complexity of the image” is an index representing the richness of the content. For example, the presence or absence of a person in the image, the intensity of change in shading in the image, and the intensity of movement between frames. It is defined by the strength. For example, the threshold setting unit 22b defines a higher degree of “image complexity” when a person is detected in the image by the image analysis unit 21 or as the number of detected persons is larger. Further, the threshold setting unit 22b defines the degree of “image complexity” as the image analysis unit 21 detects more portions where the change in shading is fine. Further, the threshold setting unit 22b defines the degree of “image complexity” higher as the number of motion vectors detected by the image analysis unit 21 increases and as the size of the motion vector increases.

  The threshold setting unit 22b extracts a threshold corresponding to the degree of “image complexity” defined from the analysis result by the image analysis unit 21 from the threshold setting table. Then, the threshold setting unit 22b sets the threshold extracted in this way as the threshold of the evaluation score of the group image.

  The group information generation unit 23 is a processing unit that generates information to be assigned to a group. Explaining this, the group information generation unit 23 determines whether or not the evaluation score of the group image calculated by the evaluation score calculation unit 22a is greater than or equal to the threshold set by the threshold setting unit 22b. At this time, when the evaluation score of the group image is equal to or greater than the threshold value, the group information generation unit 23 generates group information in which three items of information of the threshold value, the evaluation score, and the base information are associated with the group ID. On the other hand, when the evaluation score of the group image falls below the threshold value, the group information generation unit 23 associates the group ID with four items of information indicating that the threshold value, evaluation score, base information, and evaluation score are below the threshold value. Generate group information. Then, the group information generation unit 23 writes the group information generated in this way to the user area of each image included in the group. The group information generation unit 23 writes the IP address (IP address), the host name (host name), and the like of the receiving device 20 as the base information.

  As described above, the group information is written in the user area of each image included in the group even if information loss occurs in some of the images included in the group. This is because the group information can be referred to. Here, the group information is written in the user area of each image included in the group, but it is not always necessary to write all the images to the target. For example, group information may be written in the user area of any one or a plurality of images in the group.

  The encoding unit 24 is a processing unit that encodes an image input from the camera 200. The encoding unit 24 determines external parameters such as a bit rate and a frame rate using the analysis result of the image analysis unit 21, and encodes video data input from the camera 200 according to the external parameters. For example, the encoding unit 24 encodes the video data input from the camera 200 into MPEG2 for a personal computer or MPEG4 for a mobile phone. Here, since the group of images is formed by the minimum unit that can divide the image stream by the image analysis unit 21, the group is encoded for each group image.

  The transmission unit 25a is a processing unit that transmits data from the reception device 20 to an external device. The receiving unit 25b is a processing unit that receives data from an external device. The image storage unit 26a is a buffer for storing images.

  For example, the transmission unit 25a transmits the video data encoded by the encoding unit 24 to the transcoder 10 at the subsequent stage. The receiving unit 25b receives an image retransmission request from the subsequent transcoder 10 or receives an image erasing instruction. Here, the location near the distribution destination side in the distribution path between the receiving device 20 as the distribution source and the playback device 40 as the distribution destination is referred to as the latter stage, and the location near the distribution source is referred to as the previous stage. And

  The storage control unit 26 is a processing unit that stores the image encoded by the encoding unit 24 in the image storage unit 26a. The storage control unit 26 stores the image distributed from the camera 200 in the image storage unit 26a until the image deletion unit 28 described later deletes the image, but the image storage unit 26a stores the evaluation score of the group image. Only group images having a value equal to or greater than the threshold value are stored. The reason why only the group images whose evaluation points are equal to or higher than the threshold value is stored in the image storage unit 26a is to prevent the video data with degraded image quality from being retransmitted.

  Explaining this, when the group information generating unit 23 does not write in the user area of the image that the evaluation score has fallen below the threshold value, the storage control unit 26 uses the group image encoded by the encoding unit 24 as it is. The image is stored in the image storage unit 26a. On the other hand, when the fact that the evaluation score has fallen in the user area of the image has been written by the group information generation unit 23, the storage control unit 26 discards the group image encoded by the encoding unit 24. Then, the storage control unit 26 corrects the external parameters so that the evaluation score of the group image is equal to or greater than the threshold value, and causes the encode 24 to re-encode the group image. When the evaluation score is equal to or greater than the threshold value, the storage control unit 26 deletes information indicating that the evaluation score is below the threshold value from the group information of each image included in the group, and the group image is stored in the image storage unit 26a. To store.

  The retransmission control unit 27 is a processing unit that performs group image retransmission control. For example, when a retransmission request is received by the receiving unit 25b, the retransmission control unit 27 extracts a group image corresponding to the group ID received together with the retransmission request from the image storage unit 26a. Then, the retransmission control unit 27 transmits the group image extracted from the image storage unit 26a to the transcoder 10 at the subsequent stage.

  The image erasing unit 28 is a processing unit that erases the image stored in the image storage unit 26a. For example, when the erasing command is received by the receiving unit 25b, the image erasing unit 28 deletes the group image corresponding to the group ID received together with the erasing command from the image storage unit 26a. The deletion of the group image is a process performed to prevent the storage capacity of the image storage unit 26a from being compressed. However, if the administrator of the receiving device 20 determines that the deletion is not necessary, the deletion function is disabled. A series of video data can be stored without being deleted.

  The functional units such as the image analysis unit 21, the evaluation score calculation unit 22a, the threshold setting unit 22b, the group information generation unit 23, the encoding unit 24, the storage control unit 26, the retransmission control unit 27, and the image deletion unit 28 include various types. Integrated circuits and electronic circuits can be used. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  For the image storage unit 26a, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory is used. The image storage unit 26a may employ a storage device such as a hard disk or an optical disk.

[Configuration of Transcoder 10]
Next, the configuration of the transcoder according to the present embodiment will be described. FIG. 4 is a block diagram illustrating the configuration of the transcoder according to the second embodiment. As shown in FIG. 4, the transcoder 10 includes a reception unit 11a, a transmission unit 11b, an image analysis unit 12, an evaluation score calculation unit 13, an image storage unit 14a, a storage control unit 14, and a group information update unit. 15. Furthermore, the transcoder 10 includes an erasure command unit 16, a transcode unit 17, a retransmission control unit 18, and an image erasure unit 19.

  Among these, the receiving unit 11a is a processing unit that receives data from an external device. The transmission unit 11b is a processing unit that transmits data from the transcoder 10 to an external device. The image storage unit 14a is a buffer for storing images.

  For example, the reception unit 11a receives a distribution image distributed by the reception device 20 at the preceding stage, receives an image retransmission request from the subsequent router 30A, or receives an image deletion instruction. Further, the transmission unit 11b transmits the video data encoded by the transcoding unit 17 described later to the subsequent router 30A, transmits an image retransmission request to the previous reception device 20, and transmits an image erasure instruction. To do.

  The image analysis unit 12 is a processing unit that analyzes an image from components such as luminance, frequency, and motion vector (vector) of images included in one group. The image analysis unit 12 exhibits the same function as the image analysis unit 21 of the receiving device 20 shown in FIG.

  The evaluation score calculation unit 13 is a processing unit that calculates an evaluation score for each group. In the case of the evaluation score calculation unit 13 of the transcoder 10, the receiving device 20 exists in the previous stage. Therefore, the evaluation score calculation unit 13 evaluates the degree of change in image quality before and after transcoding, as with the evaluation score calculation unit 22a shown in FIG. The degree to which the image quality changes depending on the network status between the two is also evaluated.

  An example of evaluating the degree of change in image quality due to the network situation is to calculate an evaluation score using a packet loss rate measured by the receiving unit 11a. For example, the evaluation score calculation unit 13 calculates the evaluation score by substituting the packet loss rate measured by the reception unit 11a into the evaluation score calculation formula (3) “evaluation score = original evaluation score * packet loss rate”. To do. Note that the packet loss rate can be the one when the image distributed from the receiving device 20 is actually received, or the packet loss rate measured over 5 minutes starting from the evaluation point calculation time. You can also

  Thus, when calculating an evaluation score using the packet loss rate, an evaluation score lower than the original evaluation score is calculated as the packet loss rate increases. This is because part of the image is lost or the image itself is lost as the packet loss rate increases, and the quality of the image deteriorates. Also, the evaluation score closer to the original evaluation score is calculated as the packet loss rate is smaller. For example, if the packet loss rate is 0%, the calculated evaluation score is the same as the original evaluation score.

  Here, the evaluation point calculation unit 13 evaluates the degree of change in image quality for only the network situation, and the evaluation point N for evaluating the degree of change in image quality for both the network situation and transcoding. And calculate. The reason why the evaluation score M is calculated in addition to the evaluation score N is that it is possible to determine whether the image quality has deteriorated due to the network condition or the external parameter of transcoding. .

  As an example, the evaluation score calculation unit 13 calculates the evaluation score M for the group image by substituting the packet loss rate between the receiving device 20 and the own device into the above-described evaluation score calculation formula (3). After that, the evaluation point calculation unit 13 substitutes the compression rate, which is an external parameter of the transcode, into the above-described evaluation point calculation formula (1) with the evaluation score M of the group image as the “original evaluation point”. Thus, the evaluation score N1 of the group image is calculated. This evaluation point N1 is calculated as an evaluation point based on the network status and the bit rate. Further, the evaluation score calculation unit 13 sets the evaluation score N1 as the “original evaluation score”, and substitutes the frame rate change rate, which is an external parameter of transcoding, into the above-described evaluation score calculation formula (2). An image evaluation point N2 is calculated. This evaluation point N2 is calculated as an evaluation point based on the network status, bit rate, and frame rate. In the following, it is assumed that the evaluation point N2 is used as the evaluation point N that evaluates the degree of change in image quality for both the network status and transcoding.

  Here, the calculation result of the evaluation score calculation formula (3) is substituted into the evaluation score calculation formula (1) as “original evaluation score”, and the calculation result is further transferred to the evaluation score calculation formula (2). The case where the evaluation score N is calculated by substitution has been described. However, the method of calculating the evaluation score N by the evaluation score calculation unit 13 is not limited to this. For example, the evaluation score calculation unit 13 may calculate the evaluation score N by performing statistical processing such as arithmetic mean or weighted average on the evaluation score calculation formula (1) to the evaluation score calculation formula (3). it can.

  The storage control unit 14 is a processing unit that stores the group image received by the reception unit 11a in the image storage unit 14a when the evaluation score N2 of the group image calculated by the evaluation score calculation unit 13 is equal to or greater than a threshold value. .

  Here, just because the evaluation score N2 of the group image falls below the threshold value, the storage control unit 14 does not immediately give up storing in the image storage unit 14a. That is, when the evaluation score M of the group image is equal to or greater than the threshold value, the group image received by the reception unit 11a is stored in the image storage unit 14a. The reason why the group image is stored when the evaluation score M of the group image is equal to or greater than the threshold value is that the cause of the image quality deterioration is not the network condition but the setting of the external parameter of transcoding.

  That is, a group image whose evaluation score is at a level equal to or higher than the threshold is received by the receiving unit 11a, and if external parameters such as a bit rate and a frame rate are corrected, a group image whose image quality is not deteriorated is stored in the image storage unit 14a. This is because there is room for storage. However, when the evaluation score M of the group image is lower than the threshold value, the group image having the image quality deteriorated so as not to be solved by the transcoder 10 alone is received by the receiving unit 11a. The image is not stored in the image storage unit 14a.

  The group information update unit 15 is a processing unit that updates group information. Explaining this, when the evaluation score N2 of the group image is equal to or greater than the threshold value, the group information update unit 15 generates group information in which the threshold value, the evaluation score, and the base information are associated with the group ID. If the evaluation score M of the group image is equal to or greater than the threshold value, the group information update unit 15 associates the threshold value, the evaluation score, the base information, the fact that the evaluation score is below the threshold value, and the cause of the drop in the group ID. Group information is generated. In this case, the item “Cause of lowering” is set as “transcoding external parameter”. When the evaluation score M of the group image is lower than the threshold value, the group information update unit 15 associates the threshold value, the evaluation score, the base information, the fact that the evaluation score is lower than the threshold value, and the cause of the lowering to the group ID. Generate group information. In this case, the item “Cause below” is set as “Network status”.

  After generating the group information in this way, the group information update unit 15 writes the generated group information in the user area of each image included in the group. At this time, the group information updating unit 15 may write the information so that it is added to the group information previously written in the user area of each image included in the group, or may be written so as to be overwritten.

  The erasing command unit 16 is a processing unit that issues a group image erasing command to the preceding apparatus. For example, when the evaluation score N2 of the group image is equal to or greater than the threshold value, the group image stored in the image storage unit 14a of the own apparatus can be retransmitted. For this reason, the erasure command unit 16 transmits a erasure command for erasing the group image together with the group ID of the group image stored in the image storage unit 14a to the reception device 20 at the preceding stage. If the evaluation score M of the group image is equal to or greater than the threshold value, the group image stored in the image storage unit 14a of the own device can be retransmitted, but the image quality may deteriorate as it is transmitted to the subsequent device. is assumed. Therefore, the erasure command unit 16 sends the erasure command to the preceding receiving device only when the evaluation score M of the group image is the same value as the evaluation score in the preceding device read from the user area of the image included in the group. Go to 20. By providing such a condition, a group image before image quality degradation, for example, a group image that is equal to or higher than the evaluation point N2 is left in the preceding apparatus.

  The transcoding unit 17 is a processing unit that transcodes the distribution image received by the receiving unit 11a. The transcoding unit 17 determines external parameters such as a bit rate and a frame rate using the analysis result of the image analysis unit 12, and transcodes the distribution image according to the external parameters. For example, the transcoding unit 17 re-encodes a distribution image encoded in MPEG2 for a personal computer into MPEG4 for a portable terminal. Here, as with the encoding unit 24 shown in FIG. 3, transcoding is performed for each group image.

  The retransmission control unit 18 is a processing unit that performs group image retransmission control. For example, when a retransmission request is received by the reception unit 11a, the retransmission control unit 18 extracts a group image corresponding to the group ID received together with the retransmission request from the image storage unit 14a. The retransmission control unit 18 corrects the external parameters so that the evaluation score N2 of the group image extracted from the image storage unit 14a is equal to or higher than the threshold value, and causes the transcoding unit 17 to re-transcode the group image. When the evaluation score N2 is equal to or greater than the threshold, the retransmission control unit 18 deletes the information indicating that the evaluation score is below the threshold and the cause of the decrease from the group information of each image included in the group, and performs transcoding. The subsequent group image is transmitted to the subsequent router 30A.

  Similar to the image erasing unit 28 shown in FIG. 3, the image erasing unit 19 is a processing unit that erases the image stored in the image storage unit 14a. For example, when the erasing command is received by the receiving unit 11a, the image erasing unit 19 deletes the group image corresponding to the group ID received together with the erasing command from the image storage unit 14a. The deletion of the group image is a process performed to reduce the storage capacity of the image storage unit 14a. However, when the administrator of the transcoder 10 determines that the deletion is not necessary, the deletion function is invalidated. The video data can be stored without being deleted.

  The functional units such as the image analysis unit 12, the evaluation score calculation unit 13, the storage control unit 14, the group information update unit 15, the deletion command unit 16, the transcoding unit, the 17 retransmission control unit 18, and the image deletion unit 19 include Various integrated circuits and electronic circuits can be employed. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  For the image storage unit 14a, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory is used. The image storage unit 14a may employ a storage device such as a hard disk or an optical disk.

[Configuration of Router 30]
Next, the configuration of the router according to the present embodiment will be described. Note that both the router 30A and the router 30B shown in FIG. 2 have the same configuration. Therefore, hereinafter, the router 30A and the router 30B will be collectively referred to as the router 30 and will be described.

  FIG. 5 is a block diagram illustrating the configuration of the router according to the second embodiment. As shown in FIG. 5, the router 30 includes a reception unit 31a, a transmission unit 31b, an evaluation score calculation unit 33, a storage control unit 34, a group information update unit 35, an erasure command unit 36, and a retransmission control unit. 38 and an image erasing unit 39. Note that the router 30 is not limited to the configuration shown in FIG. 5, and is assumed to have router-specific hardware and software such as a routing table, although not shown. Further, since the router 30 has substantially the same configuration as the transcoder 10 shown in FIG. 4, description of the same part is omitted, and only a different part will be described.

  The router 30 illustrated in FIG. 5 is a communication device that connects different networks, and does not perform image processing like the transcoder 10. Therefore, the router 30 does not need a function unit corresponding to the transcode unit 17 of the transcode 10 shown in FIG.

  Since the transcoding is not performed as described above, the evaluation score calculation unit 33 needs to evaluate the degree of change in image quality before and after the transcoding is performed, like the evaluation score calculation unit 13 illustrated in FIG. Absent. Therefore, the evaluation score calculation unit 33 calculates an evaluation score M that evaluates the degree of change in image quality only for the network status between the previous device and the device itself.

  Accordingly, the storage control unit 34 determines whether to store the group image in the image storage unit 34a based on whether the evaluation score M of the group image is equal to or greater than a threshold value. That is, the storage control unit 34 stores the group image in the image storage unit 34a when the evaluation score M of the group image is equal to or greater than the threshold value, but if the evaluation score M of the group image is lower than the threshold value, the group control unit 34 stores the group image. An alternative decision is made not to store the image in the image storage unit 34a.

  Derived from this, the “lower cause” written to the user area of the group image by the group information updating unit 35 when the evaluation score M of the group image is lower than the threshold value is only applicable to the network status.

  The other functional units, the erasure command unit 36, the retransmission control unit 38, and the image erasure unit 39 are substantially the same as the erasure command unit 16, the retransmission control unit 18 and the image erasure unit 19 of the transcoder 10 shown in FIG. Demonstrate the function.

[Configuration of Playback Device 40]
Next, the configuration of the playback apparatus according to the present embodiment will be described. FIG. 6 is a block diagram of the configuration of the playback apparatus according to the second embodiment. As shown in FIG. 6, the playback apparatus 40 includes a receiving unit 41a, an image analyzing unit 42, an evaluation score calculating unit 43, a decoding unit 44, a display control unit 45, a threshold resetting unit 46, and an image storage. A unit 47a, a storage control unit 47, and a retransmission request unit 48.

  The receiving unit 41a is a processing unit that receives data from an external device. For example, the receiving unit 41a receives a distribution image distributed by the distribution source receiving device 20 from the upstream router 30B, or receives a retransmitted image retransmitted from a relay device such as the upstream transcoder 10 or the router 30. Or The transmission unit 41b is a processing unit that transmits data from the playback device 40 to an external device. For example, the transmission unit 41b transmits a retransmission request for a distribution image to a relay device such as the previous transcoder 10 or the router 30. The image storage unit 47a is a buffer for storing images.

  The image analysis unit 42 is a processing unit that analyzes an image from components such as luminance, frequency, and motion vector (vector) of images included in one group. The image analysis unit 42 exhibits the same function as that of the image analysis unit 21 of the reception device 20 illustrated in FIG.

  The evaluation score calculation unit 43 is a processing unit that calculates an evaluation score for each group. In the case of the evaluation score calculation unit 43 of the playback device 40, the router 30B exists in the previous stage. For this reason, the evaluation score calculation unit 43 evaluates the degree of change in the image quality depending on the network status between the router 30B and the own device. In addition, since the playback device 40 does not perform processing such as encoding or transcoding that deteriorates image quality, it is not necessary to evaluate the degree of change in image quality due to these processing.

  From these things, the evaluation score calculation part 43 calculates the evaluation score which evaluated the degree of the change of image quality only for the network condition between the preceding apparatus and its own apparatus. For example, when the distribution image is received by the receiving unit 41a, the evaluation score calculation unit 43 substitutes the packet loss rate between the router 30B and the own device into the evaluation score calculation formula (3). The evaluation score M is calculated.

  The decoding unit 44 is a processing unit that decodes the distribution image received by the receiving unit 41a. For example, the decoding unit 44 decodes the distribution image encoded in MPEG2 by the receiving device 20, and decodes the distribution image encoded in MPEG4 by the transcoder 10.

  The display control unit 45 is a processing unit that performs display control on the display unit 400. For example, the group image decoded by the decoding unit 44 is reproduced on the display unit 400. At this time, the display control unit 45 causes the display unit 400 to reproduce the group image decoded by the decoding unit 44 and causes the display unit 400 to display the evaluation score of the group image. This evaluation point may be displayed in a region distinguished from the video on the screen of the display unit 400, or may be displayed so as to overlap the video.

  The threshold resetting unit 46 is a processing unit that resets a threshold for comparison with the evaluation score. Here, the threshold resetting unit 46 receives the analysis result of the group image to be reset for the threshold, and before and after the group image, as compared to the threshold setting unit 22b of the receiving device 20 illustrated in FIG. The difference is that the threshold is reset based on the analysis result of the group image. Using the analysis results of the group images received before and after in this way can define the image complexity globally rather than defining the image complexity only with the group image to be reset. Because. The group image received after the group image to be reset can also be used for determining whether or not the threshold reset processing performs a retransmission request. It is not necessary to make it in time.

  This threshold resetting unit 46 is set so far when the analysis result of the group image to be reset threshold and the analysis results of the group images received before and after the group image satisfy a predetermined condition. Reset a threshold value higher than the threshold value. For example, it is assumed that the change in light and shade analyzed by the image analysis unit 42 is flat throughout the preceding and following group images. In this case, a single image can be encoded with a low load. However, in the case of following a minimal object moving around in an image by a change in luminance or motion vector, a high load is imposed when encoding is performed by incorporating the front and rear of the stream. In such a case, the threshold resetting unit 45 resets the threshold higher than the threshold that has been set so far.

  Here, a case has been described where group images received before and after a group image to be reset are used. However, only group images received before may be used, or group images received later. It is good also as using only. In addition, the number of groups to be used retroactively from the group image that is the threshold resetting target may be any number of groups.

  The storage control unit 47 is a processing unit that stores the distribution image received by the reception unit 41a, that is, the image before decoding, in the image storage unit 47a. Explaining this, it is determined whether or not the evaluation score calculated by the evaluation score calculation unit 43 is equal to or greater than the threshold reset by the threshold resetting unit 46. At this time, if the evaluation score is equal to or greater than the threshold value, the storage control unit 47 stores the image before decoding in the image storage unit 47a. If the evaluation score falls below the threshold value, the storage control unit 47 discards the image before decoding without storing it in the image storage unit 47a.

  Based on the base information written in the user area of the group image discarded by the storage control unit 47, the retransmission request unit 48 requests retransmission of the group image to the nearest device among the previous devices that store the group image. Is a processing unit.

  Explaining this, the retransmission request unit 48 refers to the item “Cause of lowering” written in the user area of the group image discarded by the storage control unit 47. As a result, when the item “Cause of being below” is “external parameter”, the retransmission request unit 48 first writes the base where the evaluation score is below the threshold, that is, the item “below the threshold”. A retransmission request is transmitted to the device. When the item “Cause of being lower” is “Network Status”, the retransmission request unit 48 determines that the base immediately before the base whose evaluation score is lower than the threshold, that is, the item “Lower than the threshold”. A retransmission request is transmitted to the preceding apparatus preceding the preceding apparatus that was written first.

[Process flow]
Next, a processing flow of the video distribution system according to the present embodiment will be described. Here, (1) distribution processing, (2) relay processing, (3) reproduction processing, (4) retransmission request processing, (5) relay point retransmission processing, and (6) distribution source retransmission processing will be described in this order.

(1) Distribution Processing FIG. 7 is a flowchart illustrating a distribution processing procedure according to the second embodiment. This distribution process is a process executed by the receiving device 20 and is activated when video data is received from the camera 200.

  As shown in FIG. 7, when video data is received from the camera 200 (Yes in step S101), the threshold value setting unit 22b uses the analysis result by the image analysis unit 21 and the threshold value setting table to determine the evaluation score of the group image. A threshold for comparison is set (step S102). Subsequently, the evaluation score calculation unit 22a calculates the evaluation score of the group image using the external parameter used by the encoding unit 24 (step S103).

  Then, the group information generation unit 23 determines whether or not the evaluation score of the group image calculated by the evaluation score calculation unit 22a is greater than or equal to the threshold set by the threshold setting unit 22b (step S104).

  At this time, if the evaluation score of the group image is equal to or greater than the threshold (Yes at Step S104), the group information generation unit 23 sets the group information in which the three items of information are associated with the group ID of each image included in the group. Write to the user area (step S105). In this case, group information in which a threshold value, an evaluation score, and base information are associated with a group ID is written in the user area.

  On the other hand, when the evaluation score of the group image falls below the threshold (No at Step S104), the group information generation unit 23 sets the group information in which the four items of information are associated with the group ID to the user area of each image included in the group. (Step S106). In this case, group information in which four items of information indicating that the threshold value, the evaluation score, the base information, and the evaluation score are below the threshold value are associated with the group ID is written in the user area.

  Thereafter, the encoding unit 24 encodes the video data input from the camera 200 (step S107). Then, the transmission unit 25a transmits the video data encoded by the encoding unit 24 to the subsequent transcoder 10 (step S108).

  If the evaluation score previously written in the user area of the group image is equal to or greater than the threshold value (Yes at Step S109), the storage control unit 26 sends the group image encoded by the encoding unit 24 to the image storage unit 26a. Store (step S110).

  On the other hand, when the evaluation score previously written in the user area of the group image falls below the threshold value (No at Step S109), the storage control unit 26 discards the group image encoded by the encoding unit 24 (Step S111). Then, the storage control unit 26 corrects the external parameters so that the evaluation score of the group image is equal to or greater than the threshold value, and re-encodes the group image to the encode 24 (step S112). After that, the storage control unit 26 stores the group image re-encoded by the encoding unit 24 in the image storage unit 26a (step S110).

  Thus, as long as video data is supplied from the camera 200, the receiving device 20 repeatedly performs the processing from step S101 to step S112 for each group image.

(2) Relay Processing FIG. 8 is a flowchart illustrating a procedure of relay processing according to the second embodiment. This relay process is a process executed by the transcoder 10, and is activated when video data is received from the preceding apparatus.

  As shown in FIG. 8, when video data is received from the preceding apparatus (Yes at Step S201), the evaluation score calculation unit 13 calculates an evaluation score M and an evaluation score N2 from the network status and external parameters (Step S202).

  At this time, if the evaluation score N2 is greater than or equal to the threshold value (Yes at Step S203), the storage control unit 14 stores the group image received by the receiving unit 11a in the image storage unit 14a (Step S204). Then, the group information update unit 15 writes the group information in which the group ID is associated with the three items of the threshold value, the evaluation score, and the base information in the user area of each image included in the group (step S205). Thereafter, the erasure command unit 16 transmits a erasure command for erasing the group image together with the group ID of the group image stored in the image storage unit 14a to the preceding apparatus (step S206).

  If the evaluation score N2 is greater than or equal to the threshold (No at Step S203), the storage control unit 14 further determines whether or not the evaluation score M is greater than or equal to the threshold (Step S207).

  As a result, when the evaluation score M is equal to or greater than the threshold value (Yes at Step S207), the storage control unit 14 stores the group image received by the reception unit 11a in the image storage unit 14a (Step S208). Subsequently, the group information update unit 15 sets the group information in which the group ID is associated with the threshold value, the evaluation score, the base information, the fact that the evaluation score is lower than the threshold value, and the cause of the lowering to the group ID of each image included in the group. Writing to the user area (step S209). In this case, the item “Cause of lowering” is set as “transcoding external parameter”.

  When the evaluation score M of the group image is the same value as the evaluation score in the preceding apparatus read from the user area of the image included in the group (No in step S210), the erasure command unit 16 issues an erasure command. The process is performed on the preceding apparatus (step S206). If the evaluation score M of the group image is smaller than the evaluation score in the preceding apparatus (Yes at Step S210), the process proceeds to Step S212 as it is.

  When the digestion point M is less than the threshold (No at Step S207), the process proceeds to Step S211 without accumulating the group image received by the receiving unit 11a. In other words, the group information update unit 15 includes group information in which the group ID is associated with the threshold value, the evaluation point, the base information, the fact that the evaluation point is lower than the threshold value, and the cause that the evaluation point is lower than the group ID. Write to the area (step S211). In this case, the item “Cause below” is set as “Network status”.

  Thereafter, the transcoding unit 17 transcodes the distribution image received by the receiving unit 11a (step S212). The transmission unit 11b transmits the video data transcoded by the transcoding unit 17 to a subsequent apparatus (step S213).

  As described above, the transcoder 10 repeatedly performs the processing from step S201 to step S213 for each group image as long as video data is supplied from the preceding apparatus.

(3) Reproduction Process FIG. 9 is a flowchart illustrating a reproduction process procedure according to the second embodiment. This reproduction process is a process executed by the reproduction process 40, and is activated when video data is received from the preceding apparatus.

  As shown in FIG. 9, when the video data is received from the preceding apparatus (Yes at Step S301), the evaluation score calculating unit 43 calculates the evaluation score M from the network status between the preceding apparatus and the own apparatus (Step S301). S302).

  And the decoding part 44 decodes the delivery image received by the receiving part 41a (step S303). Subsequently, the display control unit 45 causes the display unit 400 to reproduce the group image decoded by the decoding unit 44, and causes the display unit to display the evaluation score of the group image (step S304). Note that the visibility of video data is not displayed by displaying information indicating that the evaluation score or threshold value is lower than only the video data for which the evaluation score is lower than the threshold value, instead of displaying the evaluation score for all video data. As a result, it is possible for the viewer of the video data to easily recognize when the video data being browsed is retransmitted separately, and the timing for determining the necessity of confirmation of the retransmitted data by the viewer.

  Thereafter, the threshold resetting unit 46, based on the analysis result of the group image to be reset threshold and the analysis results of the group images received before and after the group image, than the threshold value set so far. A high threshold is reset (step S305).

  Thereafter, the storage control unit 47 determines whether or not the evaluation score calculated by the evaluation score calculation unit 43 is equal to or greater than the threshold reset by the threshold resetting unit 46 (step S306). At this time, if the evaluation score is equal to or greater than the threshold value (Yes at Step S306), the storage control unit 47 stores the image before decoding in the image storage unit 47a (Step S307).

  If the evaluation score falls below the threshold (No at Step S306), the storage control unit 47 discards the image before decoding without storing it in the image storage unit 47a (Step S308). Thereafter, the retransmission request unit 48 performs “retransmission request processing” for requesting retransmission of the group image to the nearest device among the previous devices (step S309).

  As described above, the playback device 40 repeatedly performs the processing from step S301 to step S308 for each group image as long as video data is supplied from the preceding device.

(4) Retransmission Request Processing FIG. 10 is a flowchart illustrating a retransmission request processing procedure according to the second embodiment. This retransmission request process is a process executed by the playback device 40, and is started after the process of step S307 shown in FIG. 9 is performed.

  As illustrated in FIG. 10, the retransmission request unit 48 refers to the item “cause of lowering” written in the user area of the image included in the group discarded by the storage control unit 47 (step S <b> 401).

  At this time, when the item “cause of being below” is “external parameter” (Yes in step S402), the retransmission request unit 48 determines that the evaluation score is below the threshold, that is, the item “below the threshold”. A retransmission request is transmitted to the preceding apparatus written first (step S403).

  When the item “Cause of lowering” is “Network status” (No at Step S402), the retransmission request unit 48 transmits a retransmission request to the base immediately before the base whose evaluation score is lower than the threshold. (Step S404). In other words, the retransmission request unit 48 makes a retransmission request to the preceding apparatus immediately before the preceding apparatus in which the item “below the threshold value” is first written.

  Thereafter, when the retransmitted retransmitted image is received as a response to the retransmit request (Yes at step S405), the retransmit request unit 48 determines whether or not the retransmitted image evaluation score is equal to or higher than the reset threshold value (step S405). S406). At this time, if the evaluation score of the retransmitted image is equal to or greater than the reset threshold value (Yes at Step S406), the storage control unit 47 stores the retransmitted image in the image storage unit 47a (Step S409).

  On the other hand, if the evaluation score of the retransmission image is below the threshold (No at Step S406), the retransmission request unit 48 discards the retransmission image without storing it in the image storage unit 47a (Step S407). Then, the retransmission request unit 48 repeatedly transmits the retransmission request to the preceding apparatus that has requested retransmission in step S403 or step S404 (step S408), and returns to step S405 described above.

(5) Relay Point Retransmission Processing FIG. 11 is a flowchart illustrating a procedure of relay point retransmission processing according to the second embodiment. This relay point retransmission process is a process executed by the transcoder 10 and is activated when a retransmission request is received from the preceding apparatus.

  As shown in FIG. 11, when a retransmission request is received from the preceding apparatus (Yes at Step S501), the retransmission control unit 18 extracts a group image corresponding to the group ID received together with the retransmission request from the image storage unit 14a (Step S501). S502).

  Then, the retransmission control unit 18 corrects the external parameters so that the evaluation score N2 of the group image extracted from the image storage unit 14a is equal to or greater than the threshold value, and causes the transcoding unit 17 to re-transcode the group image (step S503). And step S504).

  Thereafter, the retransmission control unit 18 deletes information indicating that the evaluation score has fallen below the threshold and the cause of the drop from the group information of each image included in the group, and transmits the group image after transcoding to the subsequent apparatus ( Step S505).

(6) Distribution Source Retransmission Processing FIG. 12 is a flowchart illustrating a distribution source retransmission processing procedure according to the second embodiment. This distribution source retransmission process is a process executed by the receiving apparatus 20 and is activated when a retransmission request is received from the preceding apparatus.

  As shown in FIG. 12, when a retransmission request is received from the preceding apparatus (Yes at Step S601), the retransmission control unit 27 extracts a group image corresponding to the group ID received together with the retransmission request from the image storage unit 26a (Step S601). S602).

  Then, the retransmission control unit 27 transmits the group image extracted from the image storage unit 26a to the subsequent transcoder 10 (step S603).

[Effect of Example 2]
As described above, the relay device of the transcoder 10, the router 30A, or the router 30B according to the present embodiment transmits video data to another relay device or a distribution destination device regardless of the evaluation value of the video data. To do. Therefore, the playback device 40 can receive video data in real time. Furthermore, the relay device according to the present embodiment transmits the video data together with the address information of the relay device to another relay device or the receiving device 20 when the evaluation score of the video data is equal to or greater than a predetermined threshold. For this reason, in the playback device 40, the video data can be retransmitted to the nearest relay device among the address information received from the relay device, and there is no need to retransmit back to the receiving device 20. In addition, since the relay device according to the present embodiment stores the video data when the evaluation score of the video data is equal to or higher than a predetermined threshold value, the video data with degraded video quality is retransmitted to the playback device 40. Nor. Therefore, according to the relay device according to the present embodiment, it is possible to improve the retransmission efficiency of video data while ensuring real-time delivery of video data.

  Further, in the relay apparatus according to the present embodiment, the transcoder 10 calculates the evaluation score further including the degree to which the quality of the video data changes before and after the transcoding is executed by the transcoding unit. Then, the transcoder 10 according to the present embodiment stores the video data in the image storage unit 14a when the evaluation point of the video data falls below a threshold value due to transcoding. Further, the transcoder 10 according to the present embodiment, when the evaluation point of the video data falls below the threshold value due to the transcode, the information indicating that the address information of the own device and the transcode are below the threshold value due to the transcode. At the same time, the transcoded video data is transmitted. For this reason, in the transcoder 10 according to the present embodiment, video data whose evaluation score is equal to or greater than the threshold value is received, and if external parameters of the transcode are corrected, video data whose image quality is not deteriorated is stored in the image storage unit. 14a can be stored.

  Also, the playback device 40 according to the present embodiment requests the retransmission of the video data to the nearest relay device among the relay devices in which the video data is stored in the image storage unit, based on the address information received together with the video data. For this reason, according to the reproducing apparatus 40 according to the present embodiment, the video data can be retransmitted through the existing shortest path between the receiving apparatus 20 and the reproducing apparatus 40, and the network is prevented from being congested and the retransmission time is reduced. Can be shortened.

  Further, when the evaluation point of the video data is equal to or higher than the threshold, the relay device according to the present embodiment requests the video data distribution source relay device to delete the video data from the image storage unit. For this reason, in the relay device according to the present embodiment, it is possible to delete the video data when it is no longer needed at the distribution-source relay device, and the storage capacity of other relay devices is compressed by the video data. It is possible to prevent.

  Here, the transmission destination requested by the distribution destination apparatus will be described with reference to FIG. FIG. 13 is an explanatory diagram for explaining a transmission destination to which a distribution destination apparatus requests retransmission. In the example of FIG. 13, an encoder 61 connected to a camera 61b is provided as a distribution source device, and a decoder 69 (decoder) connected to a monitor 69b is provided as a distribution destination device. In the figure, “enc” is an abbreviation for encoder, and “dec” in the figure is abbreviated.

  Reference numerals 63, 65, and 67 shown in FIG. 13 indicate relay bases from the encoder 61 to the decoder 69, and either the transcoder 10 or the router 30 is installed at this relay base. Reference numerals 61a, 63a, 65a, 67a, and 69a shown in FIG. 13 indicate buffers that can store video data. The code | symbol 60 shown in FIG. 13 shows the video data transmitted to the encoder 61 from the camera 61b. Reference numeral 62 shown in FIG. 13 indicates video data transmitted from the encoder 61 to the relay device at the relay point 63. Reference numeral 64 shown in FIG. 13 indicates video data transmitted from the relay device at the relay point 63 to the relay device at the relay point 65. Reference numeral 66 shown in FIG. 13 indicates video data transmitted from the relay device at the relay point 65 to the relay device at the relay point 67. 13 indicates video data transmitted from the relay device at the relay point 67 to the decoder 69.

  For example, the threshold value of the evaluation score is “70 points”, the evaluation score of the video data 62 is “100 points”, the evaluation score of the video data 64 is “85 points”, the evaluation score of the video data 66 is “80 points”, and the video Assume that the evaluation score of the data 68 is “55 points”. At this time, if the cause that the evaluation score of the video data 68 falls below the threshold value 70 is “external parameter”, the decoder 69 makes a retransmission request to the relay device at the relay point 67. Accordingly, the decoder 69 can receive the retransmission of the video data 66 from the relay device at the relay point 67. On the other hand, if the cause that the evaluation score of the video data 68 falls below the threshold 70 is the “network condition”, the decoder 69 makes a retransmission request to the relay device at the relay point 65. Accordingly, the decoder 69 can receive the retransmission of the video data 64 from the relay device at the relay point 65.

  Next, the effect of retransmitting video data will be described with reference to FIGS. FIG. 14 is a diagram illustrating an example of video data distributed by the encoder. FIG. 15 is a diagram illustrating an example of video data that has been interpolated by error correction by a decoder. FIG. 16 is a diagram illustrating an example of video data before error correction is performed by the decoder. In the examples of FIGS. 14 to 16, it is assumed that weather data is monitored by a delivery destination device.

  As shown in FIG. 14, when video data 71, video data 72, and video data 73 are delivered in this order from the encoder, the decoder sequentially decodes video data 71, video data 72, and video data 73. As a result, the decoder displays the video data 81, the video data 82, and the video data 83 in this order on the monitor.

  However, among the three pieces of video data, the video data 82 is obtained by error-correcting the picture missing part of the video data 90 before error correction. That is, if the video data depends only on real-time distribution, the viewer misses the “lightning strike” included in the video data 72. Therefore, as described in the second embodiment, by retransmitting the video data when the evaluation score is lower than the threshold, the viewer can confirm “lightning strike” included in the video data 72 later. it can. Further, by displaying the calculated evaluation score together with the video data, the viewer can easily determine whether or not the retransmitted image needs to be confirmed.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Video distribution program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes an output program having the same function as that of the above embodiment will be described with reference to FIG. FIG. 17 is a schematic diagram illustrating an example of a computer that executes a video distribution program according to the third embodiment.

  As illustrated in FIG. 17, the computer 100 according to the third embodiment includes an operation unit 110a, a microphone 110b, a speaker 110c, a display 120, and a communication unit 130. The computer 100 further includes a CPU 150, a ROM 160, an HDD (Hard Disk Drive) 170, and a RAM (Random Access Memory) 180. These units 110 to 180 are connected via a bus 140.

  The ROM 160 stores in advance a control program that exhibits the same functions as those of the image analysis unit 12, the evaluation score calculation unit 13, the storage control unit 14, and the group information update unit 15 described in the second embodiment. . Further, the ROM 160 stores in advance a control program that exhibits the same functions as those of the erasing command unit 16, the transcoding unit 17, the retransmission control unit 18, and the image erasing unit 19. That is, as shown in FIG. 17, the ROM 160 stores an image analysis program 160a, an evaluation score calculation program 160b, a storage control program 160c, and a group information update program 160d. Further, the ROM 160 stores an erase command program 160e, a transcode program 160f, a retransmission control program 160g, and an image erase program 160h. Note that these programs 160a to 160h may be appropriately integrated or separated in the same manner as each component of the transcoder 10 shown in FIG.

  Then, the CPU 150 reads these programs 160a to 160h from the ROM 160 and executes them. As a result, as shown in FIG. 17, the CPU 150 functions as an image analysis process 150a, an evaluation point calculation process 150b, a storage control process 150c, and a group information update process 150d. Further, the CPU 150 functions as an erase command process 150e, a transcode process 150f, a retransmission control process 150g, and an image erase process 150h. Each of the processes 150a to 150h includes an image analysis unit 12, an evaluation score calculation unit 13, a storage control unit 14, a group information update unit 15, an erasure command unit 16, and a transcoding unit illustrated in FIG. 17, the retransmission control unit 18, and the image erasing unit 19.

  Then, the CPU 150 executes an image analysis process 150a, an evaluation point calculation process 150b, and a storage control process 150c. Further, the CPU 150 performs group image storage control by executing a group information update process 150d, an erase command process 150e, and a transcode process 150f. As a result, the image accumulation data 180 a is stored in the RAM 180. In addition, the CPU 150 executes group image retransmission control by executing a retransmission control process 150g and an image erasing process 150h.

  Note that the above-described data reproduction program is not necessarily stored in the HDD 170 or the ROM 160 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 100 may acquire and execute each program from these portable physical media. In addition, each program is stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires and executes each program from these. It may be.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1) A video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
The relay device is
A video data storage unit for storing the video data;
When the video data is received, the reliability for calculating the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device A calculation unit;
A storage control unit that stores the video data in the video data storage unit when the reliability of the video data calculated by the reliability calculation unit is equal to or greater than a predetermined threshold;
If the reliability of the video data calculated by the reliability calculation unit is greater than or equal to a threshold, the video data is transmitted to another relay device or the delivery destination device together with the address information of the own device, and the video data And a transmission control unit that transmits the video data to another relay device or the delivery destination device when the reliability of the video data falls below a threshold.

(Appendix 2) The relay device
A transcoding unit for transcoding the video data;
The reliability calculation unit includes:
Calculating the reliability of the video data further including the degree to which the quality of the video data changes before and after transcoding is performed by the transcoding unit;
The storage control unit
When the reliability of the video data calculated by the reliability calculation unit falls below a threshold value due to the transcode, the video data is stored in the video data storage unit,
The transmission control unit
When the reliability of the video data calculated by the reliability calculation unit is lower than the threshold value due to the transcode, together with address information of the own device and information indicating that the transcode is lower than the threshold value The video distribution system according to appendix 1, wherein the video data transcoded by the transcoding unit is transmitted.

(Appendix 3) The delivery destination device is:
A retransmission request unit that requests retransmission of the video data to a nearest relay device among relay devices in which the video data is stored in the video data storage unit based on address information received together with the video data. The video distribution system according to appendix 1 or 2.

(Appendix 4) The relay device
Deletion requesting the video data distribution source relay device to delete the video data from the video data storage unit when the reliability of the video data calculated by the reliability calculation unit is greater than or equal to a threshold value 4. The video distribution system according to appendix 1, 2 or 3, further comprising a request unit.

(Supplementary Note 5) A video data storage unit that stores video data distributed from a distribution source device to a distribution destination device;
When the video data is received, the reliability for calculating the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device A calculation unit;
A storage control unit that stores the video data in the video data storage unit when the reliability of the video data calculated by the reliability calculation unit is equal to or greater than a predetermined threshold;
If the reliability of the video data calculated by the reliability calculation unit is greater than or equal to a threshold, the video data is transmitted to another relay device or the delivery destination device together with the address information of the own device, and the video data And a transmission control unit that transmits the video data to another relay device or the delivery destination device when the reliability of the video data is below a threshold.

(Appendix 6) A video distribution program used in a video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
In the relay device,
When the video data is received, the reliability for calculating the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device Calculation procedure,
A storage control procedure for storing the video data in a video data storage unit for storing the video data when the reliability of the video data calculated by the reliability calculation procedure is equal to or greater than a predetermined threshold;
When the reliability of the video data calculated by the reliability calculation procedure is equal to or greater than a threshold, the video data is transmitted to another relay device or the delivery destination device together with the address information of the own device, and the video data And a transmission control procedure for transmitting the video data to another relay apparatus or the distribution destination apparatus when the reliability of the video distribution falls below a threshold.

(Appendix 7) A video distribution method used in a video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
The relay device is
When the video data is received, the reliability for calculating the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device A calculation step;
A storage control step of storing the video data in a video data storage unit that stores the video data when the reliability of the video data calculated by the reliability calculation step is equal to or greater than a predetermined threshold;
When the reliability of the video data calculated by the reliability calculation step is equal to or greater than a threshold, the video data is transmitted to another relay device or the delivery destination device together with the address information of the own device, and the video data And a transmission control step of transmitting the video data to another relay device or the delivery destination device when the reliability of the video data falls below a threshold value.

DESCRIPTION OF SYMBOLS 1 Relay device 2 Video | video data storage part 3 Reliability calculation part 4 Storage control part 5 Transmission control part 10 Transcoder 11a Reception part 11b Transmission part 12 Image analysis part 13 Evaluation point calculation part 14 Storage control part 14a Image storage part 15 Group information Update unit 16 Erasing command unit 17 Transcoding unit 18 Retransmission control unit 19 Image erasing unit 20 Receiving device 21 Image analysis unit 22a Evaluation point calculation unit 22b Threshold setting unit 23 Group information generation unit 24 Encoding unit 25a Transmission unit 25b Reception unit 26a Image Storage unit 26b Storage control unit 27 Retransmission control unit 28 Image erasure unit 30, 30A, 30B Router 31a Reception unit 31b Transmission unit 33 Evaluation point calculation unit 34 Storage control unit 35 Group information update unit 36 Deletion command unit 38 Retransmission control unit 39 Image Eraser 40 Playback device 41a Receiver 41b Transmission 42 the image analysis unit 43 evaluation point calculation unit 44 decoding unit 45 display control unit 46 threshold resetting unit 47 stores the control unit 47a an image storage unit 48 the retransmission control unit 50 network

Claims (7)

A video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
The relay device is
A video data storage unit for storing the video data;
A relay device in the preceding stage that relays video data in the preceding stage further than the relay device in the preceding stage and a relay device in the preceding stage that relays the video data in the preceding stage further than the relay device in the preceding stage together with the video data from the relay device in the preceding stage that relays the video data in the preceding stage than the relay device. A receiving unit that receives the reliability of which the degree of change in the quality of the video data is calculated from the network status between
When the reliability of the video data is received , the reliability calculation is performed to recalculate the reliability of the video data from the network status between the preceding relay device and the own device using the reliability of the video data And
A storage control unit that stores the video data in the video data storage unit when the reliability of the video data recalculated by the reliability calculation unit is equal to or greater than a predetermined threshold;
When the reliability of the video data recalculated by the reliability calculation unit is greater than or equal to a threshold value, the video data is transferred to another relay device or the video information together with the address information of the own device and the reliability of the video data after recalculation. A transmission for transmitting the video data to another relay apparatus or the distribution destination apparatus when the reliability of the video data transmitted to the distribution destination apparatus and recalculated by the reliability calculation unit falls below a threshold. A video distribution system comprising: a control unit. The relay device is
A transcoding unit for transcoding the video data;
The reliability calculation unit includes:
Wherein further including the degree of quality changes of the video data before and after transcoding is performed to recalculate the reliability of the image data by the transcoding unit,
The storage control unit
When the reliability of the video data recalculated by the reliability calculation unit falls below a threshold value due to the transcode, the video data is stored in the video data storage unit,
The transmission control unit
When the reliability of the video data recalculated by the reliability calculation unit falls below a threshold value due to the transcode, the address information of the device itself and information indicating that the transcode has fallen below the threshold value The video distribution system according to claim 1, wherein the video data transcoded by the transcoding unit is transmitted. The delivery destination device is:
A retransmission request unit that requests retransmission of the video data to a nearest relay device among relay devices in which the video data is stored in the video data storage unit based on address information received together with the video data. The video distribution system according to claim 1 or 2. The relay device is
A deletion request unit that requests the preceding relay device to delete the video data from the video data storage unit when the reliability of the video data recalculated by the reliability calculation unit is equal to or greater than a threshold; The video distribution system according to claim 1, 2 or 3. A video data storage unit that stores video data distributed from a distribution source device to a distribution destination device;
Between the preceding relay device that relays video data at a further preceding stage than the preceding relay device and the preceding relay device together with the video data from the preceding relay device that relays the video data at a preceding stage than the own device A receiving unit that receives the reliability in which the degree of change in the quality of the video data is calculated from the network status between
When the reliability of the video data is received , the reliability calculation is performed to recalculate the reliability of the video data from the network status between the preceding relay device and the own device using the reliability of the video data And
A storage control unit that stores the video data in the video data storage unit when the reliability of the video data recalculated by the reliability calculation unit is equal to or greater than a predetermined threshold;
When the reliability of the video data recalculated by the reliability calculation unit is greater than or equal to a threshold value, the video data is transferred to another relay device or the video information together with the address information of the own device and the reliability of the video data after recalculation. A transmission for transmitting the video data to another relay apparatus or the distribution destination apparatus when the reliability of the video data transmitted to the distribution destination apparatus and recalculated by the reliability calculation unit falls below a threshold. And a control unit. A video distribution program used for a video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
In the relay device,
A relay device in the preceding stage that relays video data in the preceding stage further than the relay device in the preceding stage and a relay device in the preceding stage that relays the video data in the preceding stage further than the relay device in the preceding stage together with the video data from the relay device in the preceding stage that relays the video data in the preceding stage than the relay device. A reception procedure for receiving a reliability in which a degree of change in the quality of the video data is calculated from a network situation between
When the reliability of the video data is received , the reliability calculation is performed to recalculate the reliability of the video data from the network status between the preceding relay device and the own device using the reliability of the video data Procedure and
A storage control procedure for storing the video data in a video data storage unit for storing the video data when the reliability of the video data recalculated by the reliability calculation procedure is equal to or greater than a predetermined threshold;
When the reliability of the video data recalculated by the reliability calculation procedure is greater than or equal to a threshold value, the video data is transferred to another relay device or the video information together with the address information of the own device and the reliability of the recalculated video data. A transmission for transmitting the video data to another relay device or the distribution destination device when the reliability of the video data transmitted to the distribution destination device and recalculated by the reliability calculation procedure falls below a threshold. A video distribution program characterized by causing a control procedure to be executed.   A video distribution system having a relay device that relays video data distributed from a distribution source device to a distribution destination device,
  The relay device is
  A video data storage unit for storing the video data;
  A transcoding unit for transcoding the video data;
  When the video data is received, the transcoding unit indicates the reliability of the video data indicating the degree to which the quality of the video data has changed from the network status between the distribution source device or another relay device and the own device. A reliability calculation unit that further calculates the degree of change in the quality of the video data before and after transcoding is performed by
  When the reliability of the video data calculated by the reliability calculation unit is greater than or equal to a predetermined threshold, or when the reliability of the video data calculated by the reliability calculation unit falls below the threshold due to the transcoding A storage control unit for storing the video data in the video data storage unit;
  When the reliability of the video data calculated by the reliability calculation unit is greater than or equal to a threshold value, the video data transcoded by the transcoding unit together with the address information of the own device is transmitted to another relay device or the distribution Send it to the destination device,
  When the reliability of the video data calculated by the reliability calculation unit is lower than the threshold value due to the transcode, together with address information of the own device and information indicating that the transcode is lower than the threshold value The video data transcoded by the transcode unit is transmitted to another relay device or the delivery destination device,
  When the reliability of the video data calculated by the reliability calculation unit falls below a threshold value, the video data transcoded by the transcoding unit is transmitted to another relay device or the delivery destination device. With the transmission controller
  A video distribution system comprising:

Images