JP7037331B2 - Data distribution system and distribution control device - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Law IEICE Technical Report Vol. 117 No. 33 (Publisher: Institute of Electronics, Information and Communication Engineers Publication date: May 11, 2017 [Publications, etc.] Institute of Electronics, Information and Communication Engineers Network System Study Group (NS) (Date: May 18, 2017)

The present invention relates to a data distribution system and a distribution control device for distributing data.

When distributing data (for example, image data, audio data, etc.) to a terminal device, a data distribution system that distributes data of different qualities according to the request of the terminal device is used. For example, the image monitoring system of Patent Document 1 transfers image data of different qualities (different resolution, image quality, frame rate, etc.) to different image recording devices (camera, recorder, server, etc.) for captured images taken by a camera. By storing and communicating with an image recording device that stores image data of the quality required by the monitoring terminal, it is possible to display image data of the required quality.

Japanese Unexamined Patent Publication No. 2012-209727

However, when such a distribution method is used, when a large number of monitoring devices require different quality image data, the larger the number of required image data qualities, the greater the quality of the image recording device. There has been a problem in terms of management cost that the image data of the above must be stored and a large storage area must be secured on the image recording device side.

For example, it is possible to acquire a plurality of image data having different qualities on the camera side and directly transmit the image data of the quality required by each monitoring terminal from the camera to each monitoring terminal. As the number of monitoring terminals increases, there arises a problem that the amount of communication between the camera and the monitoring terminals increases.

An object of the present invention is to provide a data distribution system and a distribution control device capable of efficiently distributing data of quality required by a distribution destination while suppressing an increase in communication volume.

In order to solve such a problem, the present invention has a data transmission device that transmits a plurality of packets including data and quality information, and a packet that receives a plurality of packets and delivers a packet based on at least a part of the plurality of packets. A first delivery control device and a second delivery control device that selects and delivers a packet based on at least a part of the packets received from the first delivery control device for each of a plurality of delivery destinations. Provide a data distribution system to have. In the data distribution system, the second distribution control device includes a storage unit that stores a plurality of required quality information indicating the quality required by each of the plurality of distribution destinations, and a storage unit that stores the required quality information. A plurality of information transmitters that transmit the highest required quality information indicating the highest quality to the first delivery control device, and a plurality of quality information and required quality information contained in each packet received from the first delivery control device. It has a second control unit that controls delivery to the delivery destination, and the first delivery control device has quality information included in the packet and a second delivery control for each packet received from the data transmission device. A second that compares with the highest required quality information received from the device and controls so that the quality indicated in the quality information does not deliver packets higher than the quality indicated in the highest required quality information to the second delivery control device. It has one control unit.

Further, in the data distribution system according to the present invention, the quality information is contained in the header part of the packet different from the data, and the first control unit is the quality included in the header part of the packet received from the data transmission device. It is preferable to extract the information.

Further, in the data distribution system according to the present invention, it is preferable that the data transmission device encrypts the data included in the packet by an encryption method that can be decrypted by a plurality of distribution destination devices.

Further, in the data distribution system according to the present invention, in the information transmission unit, the quality of the first type shown in the first required quality information, which is the required quality information of the first distribution destination, is the required quality of the second distribution destination. The quality of the second kind shown in the second required quality information is higher than the quality of the first kind shown in the second required quality information which is information, and the quality of the second kind shown in the first required quality information is shown in the second required quality information. If the quality is lower than the quality, the first required quality information and the second required quality information are transmitted to the first delivery control device, and when the first control unit receives the required quality information, each packet received from the data transmitting device. In addition, the quality information contained in the packet is compared with the first required quality information and the second required quality information, and the quality shown in the quality information is based on a packet having a quality equal to or lower than the quality shown in the first required quality information. A second delivery control is performed by generating a packet and delivering it to the second delivery control device, and generating a packet based on a packet whose quality shown in the quality information is equal to or lower than the quality shown in the second required quality information. It is preferable to control the delivery to the device.

Further, in order to solve such a problem, the present invention receives a plurality of packets including data and quality information, and obtains the quality information contained in each received packet and the quality required by each of the plurality of delivery destinations. At least one of the plurality of packets is sent to another delivery control device that selects and distributes a packet based on at least a part of each packet for each of the plurality of delivery destinations based on the required quality information shown. A delivery control device for delivering a packet based on a unit is provided. The delivery control device receives from the other delivery control device the highest required quality information indicating the highest quality among the plurality of required quality information indicating the quality required by each of the plurality of delivery destinations. For each of the plurality of received packets, the quality information contained in the packet is compared with the highest required quality information received from the other delivery control device, and the quality shown in the quality information is compared. However, it has a control unit that controls not to deliver a packet having a quality higher than that shown in the highest required quality information to the other delivery control device.

The data distribution system and the distribution control device according to the present invention can efficiently distribute data of the quality required by the distribution destination while suppressing an increase in the amount of communication.

It is a schematic block diagram of the data distribution system 1 by this embodiment. (A) to (c) are schematic diagrams showing an example of an image packet data structure, (d) is a schematic diagram showing an example of an image string format information data structure, and (e) is an image format information. It is a schematic diagram which shows an example of a data structure. It is a figure which shows the schematic structure of the data transmission apparatus 20. (A) is a schematic diagram showing an example of the data structure of the extended packet generation information 231, (b) to (d) are schematic diagrams showing an example of the data structure of the extended packet, and (e) is a schematic diagram showing an example of the data structure of the extended packet. It is a schematic diagram which shows an example of the data structure of. It is a figure which shows the schematic structure of the delivery control device 40. (A) is a schematic diagram showing an example of the data structure of the delivery destination information 422, and (b) is a schematic diagram showing an example of the data structure of the request information. (A) to (c) are schematic diagrams showing an example of the data structure of the distribution packet. It is a sequence diagram which concerns on the update process of the delivery destination information which concerns on this embodiment. It is an overall operation sequence diagram of the data distribution system 1 which concerns on this embodiment. It is a flowchart which shows the example of the operation of the extended packet generation processing. It is a flowchart which shows the example of the operation of the delivery destination specifying process.

Hereinafter, the data distribution system of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of a data distribution system 1 according to the present embodiment. As shown in FIG. 1, the data distribution system 1 includes a plurality of image transmission devices 10, a data transmission device 20, a plurality of distribution control devices 40a to 40c, a plurality of terminal devices 50a to 50d, and the like. The image transmission device 10 is arranged in the security area 2. The image transmission device 10 and the data transmission device 20 are respectively connected to the network 3 and communicate with each other via the network 3. The network 3 is, for example, a wired network such as an intranet composed of a LAN (Local Area Network). The network 3 may be a wireless network such as a wireless LAN (Local Area Network), WiMAX (Worldwide Interoperability for Microwave Access) or PHS (Personal Handy-phone System).

The distribution control device 40 is an in-network device of the telecommunications carrier network 4. The data transmission device 20 and the plurality of terminal devices 50 are each connected to the communication carrier network 4 by communication, and communicate with the distribution control device 40 arranged in the communication carrier network 4. The telecommunications carrier network 4 is a network according to a communication method such as W-CDMA (Wideband Code Division Multiple Access), CDMA2000 or LTE (Long Term Evolution). The telecommunications carrier network 4 may be a wired network such as an intranet. In the data distribution system 1 according to the present embodiment, the data transmission device 20 and the distribution control device 40a (“first distribution control device” in the present invention) are connected by communication. Further, the distribution control devices 40b and 40c (“second distribution control device” in the present invention) that are communication-connected to the distribution control device 40a are communication-connected to the terminal device 50.

The number of distribution control devices 40a for communication connection with the data transmission device 20 is not limited to one, and may be plural. Further, the distribution control devices 40b and 40c for communication connection with the distribution control device 40a are not limited to two, and may be one or three or more. Further, the distribution control devices 40b and 40c may communicate with each terminal device 50 via one or more other distribution control devices 40.

Each image transmission device 10 is mounted on, for example, a network camera or the like, and packets continuously captured images (image strings composed of a plurality of frames) captured by the network camera in a time series. For example, each image transmission device 10 is arranged in the security area 2 so that the network camera captures an image in the security area 2, and images of monitoring targets such as an intruder and a suspicious object in the security area 2 are captured. Get the column. Each image transmission device 10 transfers the acquired image sequence to H.I. 264 / MPEG (Moving Picture Experts Group) -4 Encoding (encoding) into image data having quality such as predetermined resolution, image quality, frame rate, etc. according to a predetermined coding standard such as AVC (Advanced Video Coding) standard. Generate an image packet related to the encoded image data. Each image transmission device 10 has a communication unit, and transmits the generated image packet to the data transmission device 20 via the communication unit.

Each image transmission device 10 encodes an image sequence acquired with different qualities (resolution, image quality, frame rate) into a plurality of image data having different qualities. Alternatively, each image transmitting device 10 may convert an image string acquired with a predetermined quality into an image string having a quality different from the quality, and encode it into a plurality of image data having different qualities.

The data distribution system 1 has only one image transmission device 10, and one image transmission device 10 may encode an image sequence captured by different network cameras into a plurality of image data.

2 (a) to 2 (c) are schematic views showing an example of a data structure of an image packet for transmitting encoded image data. As shown in FIGS. 2A to 2C, the image packet has a header portion and a payload portion.

The header portion has at least an image transmission device ID, a data transmission device address, and the like. The image transmission device ID is a unique value (for example, an IP address, a port number, etc.) that can uniquely identify the image transmission device 10. The data transmission device address is an address (for example, an IP address, a port number, etc.) of the data transmission device 20 that is communicated and connected to each image transmission device 10. The structure of the image packet is classified into three types according to the structure of the payload unit.

FIG. 2A is an example of an image packet in which the payload portion contains format information (image sequence format information) common to a predetermined image sequence. FIG. 2D is a schematic diagram showing an example of a data structure of image string format information. As shown in FIG. 2D, the image string format information includes resolution information indicating the resolution of the image, image quality information indicating the coding quality (quantization parameter, etc.) of the image, or the number of images included in one second. It has frame rate information and the like indicating (number of frames). For example, in the H.264 / MPEG-4 AVC standard, SPS (Sequence Parameter Set), PPS (Picture Parameter Set), and the like are set as image sequence format information.

For example, SPS frame_mbs_only_flag, pic_width_in_mbs_minus1, pic_height_in_map_units_minus1 and the like are set as resolution information. frame_mbs_only_flag is 1 if the picture contains only frame macroblocks, i.e. not interlaced, and 0 otherwise. pic_width_in_mbs_minus1 is (number of horizontal macroblocks of the picture-1) and pic_height_in_map_units_minus1 is (number of vertical macroblocks of the picture-1). When the number of pixels in the horizontal and vertical directions of the macroblock is 16, the number of pixels in the horizontal direction and the number of pixels in the vertical direction are calculated by the following equations.

Number of pixels in the horizontal direction = (pic_width_in_mbs_minus1 + 1) x 16
Number of pixels in the vertical direction = (2-frame_mbs_only_flag) × (pic_height_in_map_units_minus1 ＋ 1) × 16

For example, if the resolution is 4K (4096 x 2160), it is set to frame_mbs_only_flag = 1, pic_width_in_mbs_minus1 = 255, pic_height_in_map_units_minus1 = 134. On the other hand, when the resolution is FullHD (1920 × 1080), it is set to frame_mbs_only_flag = 0, pic_width_in_mbs_minus1 = 119, pic_height_in_map_units_minus1 = 33.

Further, as the image quality information, pic_init_qp_minus26 of PPS and the like are set. pic_init_qp_minus26 is the initial value of the quantization parameter of the slice.

As the frame rate information, num_units_in_tick, time_scale, etc. of VUI (Video Usability Information) included in SPS are set. The frame rate is calculated by dividing time_scale by num_units_in_tick. For example, if the frame rate is 60fps, it will be set to num_units_in_tick = 1000, time_scale = 60000. On the other hand, if the frame rate is 30 fps, it is set to num_units_in_tick = 1000, time_scale = 30000.

The image string format information may include information other than the above information.

In FIG. 2B, the payload portion includes format information (image format information) for a predetermined unit of image data (for example, a slice) in a predetermined image sequence and a part of the image data (partial image data). This is an example of an image packet.

FIG. 2E is a schematic diagram showing an example of a data structure of image format information. As shown in FIG. 2 (e), the image format information is format information necessary for decoding the encoded image data of a predetermined unit, and has a frame type and the like. The frame type is an I frame (reference frame) in which the frame is encoded without interframe prediction, a P frame encoded using only forward prediction, or forward prediction, backward prediction, or bidirectional prediction. Indicates which of the B frames is encoded by selecting one of the above. For example, in the H.264 / MPEG-4 AVC standard, slice_type or the like of a slice header is set as image format information. Further, the image format information may include information other than the above-mentioned information.

When the image data of a predetermined unit exceeds the size of one packet, the image data of the predetermined unit is divided into a plurality of partial image data and stored in a plurality of packets. For example, in the H.264 / MPEG-4 AVC standard, the predetermined unit is, for example, a slice. In the following, one slice will be described as one frame, but one frame may include a plurality of slices. The image format information is stored at the beginning of the image data of a predetermined unit.

FIG. 2C is an example of an image packet in which only partial image data is included in the payload portion. If the image data is small enough to be stored in one image packet, the image string format information, the image format information, and the image data may be stored in one image packet.

The data transmission device 20 receives an image packet from the image transmission device 10, generates an extended packet from the received image packet, and transmits it to the distribution control device 40. The extended packet is a packet in which quality information related to the data of the image packet is added to the header portion of the received image packet. The image packet data included in the payload portion of the extended packet is transmitted to the terminal device 50 via the distribution control device 40, and the terminal device 50 is obtained by depacketizing and reconstructing the received image packet data. The data (hereinafter referred to as "reconstruction data") is decoded to restore the image string. Here, the information indicating the quality of the reconstructed data achieved by including the data of the image packet in the reconstructed data is the quality information. The data transmission device 20 is mounted on, for example, a set-top box or the like that can execute data processing according to a predetermined application program. The data transmission device 20 and the image transmission device 10 may be mounted as one device. FIG. 3 is a diagram showing a schematic configuration of the data transmission device 20. As shown in FIG. 3, the data transmission device 20 includes a first communication unit 21, a second communication unit 22, a storage unit 23, a processing unit 24, and the like.

The first communication unit 21 has a communication interface circuit for the data transmission device 20 to communicate with the network 3 according to a communication method such as TCP / IP. The first communication unit 21 sends data such as an image packet received from the image transmission device 10 to the processing unit 24 via the network 3. Further, the first communication unit 21 transmits the data received from the processing unit 24 to the image transmission device 10 via the network 3.

The second communication unit 22 has a communication interface circuit for the data transmission device 20 to perform wireless communication between the communication carrier network 4 and a mobile phone according to a communication method such as W-CDMA, CDMA2000 or LTE. The second communication unit 22 sends the data received from the distribution control device 40 to the processing unit 24 via the communication carrier network 4. Further, the second communication unit 22 transmits the data received from the processing unit 24 to the distribution control device 40 via the communication carrier network 4.

The storage unit 23 includes a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 23 stores a computer program for controlling the data transmission device 20 and various data, and inputs and outputs these information to and from the processing unit 24. The computer program is installed in the storage unit 23 from a computer-readable portable recording medium such as a CD-ROM (compact disk read only memory) or DVD-ROM (digital versatile disk read only memory) using a known setup program or the like. May be done. Further, the storage unit 23 stores the extended packet generation information 231 and the like. The extended packet generation information 231 is information referred to when the data transmission device 20 generates an extended packet from an image packet, and is updated based on the information in the payload portion of the image packet each time the image packet is received. Will be done.

FIG. 4A is a schematic diagram showing an example of the data structure of the extended packet generation information 231. As shown in FIG. 4A, the extended packet generation information 231 includes an image transmission device ID, a resolution type, an image quality type, and a frame rate for each image string corresponding to an image packet received from each image transmission device 10. Each information such as frame number, frame type, frame number interval and frame rate type set is stored in association with each other.

The image transmission device ID is identification information of the image transmission device that generated the image sequence. As described above, the extended packet generation information 231 is recorded for each image string. Therefore, when a certain image transmission device 10 transmits a plurality of types of image strings having different qualities, the extended packet generation information 231 is used to transmit the image. A plurality of image transmission device IDs of the device 10 are registered.

The resolution type is identification information corresponding to the resolution of the image sequence. For example, if the resolution is 4K, 1 is set as the resolution type, and if the resolution is FullHD, 2 is set as the resolution type. As the resolution type, a set of the number of pixels in the horizontal direction and the number of pixels in the vertical direction may be set instead of the ID.

The image quality type is identification information corresponding to the image quality information of the image sequence. For example, when following the H.264 / MPEG-4 AVC standard, if the initial value of the quantization parameter (pic_init_qp_minus26) is equal to or less than the first threshold value (-6), 1 is set as the image quality type and quantization is performed. If the initial value of the parameter is larger than the first threshold and smaller than the second threshold (14), the image quality type is set to 2, and the initial value of the quantization parameter is equal to or higher than the second threshold. If so, 3 is set as the image quality type. As the image quality type, a set of a minimum value and a maximum value of the initial value of the quantization parameter or the initial value of the quantization parameter may be set instead of the ID.

The frame rate is a frame rate value of an image string.

The frame number is the number of the frame constituting the image string and indicates the number of the frame corresponding to the data included in the latest received image packet. The minimum value of the frame number is 1, the maximum value is the value of the frame rate, and the frame number increases monotonically from 1 for each frame and returns to 1 when the maximum value is reached.

The frame type is a type of a frame constituting an image string, and indicates a frame type corresponding to the data included in the received image packet. For example, the frame type indicates whether the frame is an I frame (reference frame), a P frame, or a B frame.

The frame number interval and frame rate type pair is information that the received image packet refers to in determining the feasible frame rate in the reconstructed data. The frame number interval indicates the frame interval that can realize the reconstruction data of a specific frame rate, and the frame rate type indicates the frame rate realized by the frame of the frame number interval. That is, when the set of the frame number interval and the frame rate type is (2, 30), it is shown that the frame interval that can realize the reconstruction data at the frame rate of 30 fps is 2.

The set of frame number interval and frame rate type is set as many as the number of possible combinations. As the frame rate type, a frame rate and a divisor of the number of reference frames generated per second (referred to as a reference frame rate) are set. For example, if the frame rate is 60 fps and the reference frame rate is 30, the frame rate type is a set consisting of 60, 30, 15, 10, 6, 5, 3, 2, 1. The reference frame rate is calculated by dividing the frame rate by the frame interval (GOP (Group Of Picture)) at which the reference frame occurs. Further, the GOP is calculated from the interval at which the image packet whose frame type is the reference frame included in the image format information is generated, but it may be set in advance. The frame number interval is calculated by dividing the frame rate by each frame rate type.

Here, a set of a frame number interval and a frame rate type in the case of complying with the H.264 / MPEG-4 AVC standard is illustrated. In this case, when the frame rate is 60 fps and the GOP is 2, the reference frame rate at which the reference frame appears is 30, and the set of frame number interval and frame rate type is {(1, 60), (2, 30). ), (4, 15), (6, 10), (10, 6), (12, 5), (20, 3), (30, 2), (60, 1)} are set.

The processing unit 24 includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), peripheral circuits, and the like. The processing unit 24 includes an extended packet generation unit 241 and an extended packet transmission unit 242 and the like.

The extended packet generation unit 241 specifies the quality information of the image packet from the information of the payload unit of the image packet received from the image transmission device 10 via the first communication unit 21. That is, the extended packet generation unit 241 specifies the quality achieved by including the image packet data in the reconstruction data, and includes the specified quality as quality information in the header part of the image packet to include the extended packet. To generate. The details of the extended packet generation process in the extended packet generation unit 241 will be described later.

The extended packet transmission unit 242 transmits the extended packet generated by the extended packet generation unit 241 to the distribution control device 40 via the second communication unit 22.

4 (b) to 4 (d) are schematic views showing an example of the data structure of the extended packet. As shown in FIGS. 4 (b) to 4 (d), the extended packet has a header part and a payload part like the image packet. However, the header unit has a distribution control device address instead of the data transmission device address, and further has quality information. That is, the quality information is included in the header part of the extended packet, which is different from the payload part in which the data is written. As a result, the delivery control device 40 can specify the delivery destination device based on the information (quality information, etc.) of only the header section without referring to the payload section of the extended packet, which improves the security and improves the security. It is possible to protect privacy.

The distribution control device address is an address of the distribution control device 40 (for example, an IP address and a port number). Similar to the configuration of the image packet, the configuration of the extended packet is also classified into three types according to the configuration of the payload unit.

FIG. 4 (e) is a schematic diagram showing an example of a data structure of quality information. As shown in FIG. 4 (e), the quality information includes information such as a resolution type, an image quality type, and a frame rate type. This quality information indicates the quality of the reconstructed data achieved by including the data included in the reconstructed data when the data included in the payload unit is reconstructed by the terminal device 50. As the quality information of the extended packet, the resolution type, the image quality type, and the frame rate type stored in the extended packet generation information 231 are set. At this time, as the frame rate type set as the quality information, the set obtained based on the current frame number is selected from the set of the frame number interval and the frame rate type of the extended packet generation information 231. And the frame rate type indicated by the set is set. The details of the frame rate type setting method will be described later.

The payload portion of the extended packet may be encrypted by an encryption method that can be decrypted by the terminal device 50. That is, since the extended packet generation unit 241 described later encrypts the payload unit of the extended packet, each distribution control device 40 cannot access the image data included in the extended packet, and the administrator of each distribution control device 40 or the like cannot access the image data. Since the content of the image data included in the extended packet cannot be viewed, it is possible to further improve the security and protect the privacy.

When the extended packet generation unit 241 receives the image packet (FIG. 2 (a)) containing the image string format information, the extended packet generation unit 241 generates an extended packet (FIG. 4 (b)) containing the image string format information. Further, when the extended packet generation unit 241 receives the image packet (FIG. 2 (b)) containing the image format information, the extended packet generation unit 241 generates the extended packet (FIG. 4 (c)) containing the image format information. Further, when the extended packet generation unit 241 receives the image string format information and the image packet (FIG. 2 (c)) that does not include the image format information, the extended packet generation unit 241 does not include the image string format information and the image format information (FIG. 4). (D)) is generated.

The distribution control device 40 receives a plurality of extended packets from the data transmission device 20 or another distribution control device 40, and controls another distribution of the extended packet based on at least a part of the received extended packets. Deliver to device 40. Further, the distribution control device 40 receives an extended packet from the data transmission device 20 or another distribution control device 40, converts at least a part of the received extended packets into distribution packets, and distributes the extended packets. It is selected and distributed for each of the plurality of terminal devices 50 mentioned above. The distribution control device 40 is mounted on, for example, a router or the like that can execute data processing according to a predetermined application program. FIG. 5 is a diagram showing a schematic configuration of the distribution control device 40. As shown in FIG. 5, the distribution control device 40 includes a communication unit 41, a storage unit 42, a processing unit 43, and the like.

The communication unit 41 has a communication interface circuit for the distribution control device 40 to communicate with the data transmission device 20, another distribution control device 40, or the terminal device 50 via the communication carrier network 4. The communication unit 41 sends data received from the data transmission device 20, another distribution control device 40, or the terminal device 50 to the processing unit 43. Further, the communication unit 41 transmits the data received from the processing unit 43 to the data transmission device 20, another distribution control device 40, or the terminal device 50.

The storage unit 42 includes a semiconductor memory such as a ROM or RAM, a magnetic recording medium and its access device, an optical recording medium, its access device, and the like. The storage unit 42 stores a computer program for controlling the distribution control device 40 and various data, and inputs and outputs these information to and from the processing unit 43. The computer program may be installed in the storage unit 42 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like. Further, the storage unit 42 stores the distribution source information 421, the distribution destination information 422, and the like. The distribution source information 421 stores identification information (for example, an IP address) of a distribution source device (data transmission device 20 or another distribution control device 40) that distributes an extended packet to its own device.

The delivery destination information 422 determines whether or not to deliver a packet based on each extended packet to a predetermined delivery destination (another delivery control device 40 or a terminal device 50) for each extended packet received by the delivery control device 40. This is the information that is referred to when doing so. FIG. 6A is a schematic diagram showing an example of the data structure of the delivery destination information 422. As shown in FIG. 6A, as the delivery destination information 422, the required quality information (required resolution type, required image quality type, and required frame rate type) is provided for each combination of the image transmission device ID and the delivery destination device address. It is stored in association with each other.

The image transmission device ID is an image transmission device 10 that is a source of an image packet corresponding to a packet (extended packet or distribution packet) to be received by the device (other delivery control device 40 or terminal device 50) that is the delivery destination. This is the image transmission device ID of. The delivery destination device address is the address of each delivery destination device. The required resolution type, required image quality type, and required frame rate type, which are the required quality information, are the resolution type, the image quality type, and the frame rate type indicating the quality of the image string required to be received in each delivery destination device, respectively. be.

The delivery destination information 422 is updated based on the request information each time the request information described later is received. However, the present invention is not limited to this, and may be set in advance by an administrator or the like using an input device (not shown), or may be set according to an instruction received from another device.

The processing unit 43 includes a CPU, an MPU, peripheral circuits, and the like. The processing unit 43 includes an information receiving unit 431, an information updating unit 432, an information transmitting unit 433, a packet receiving unit 434, a control unit 435, a packet distribution unit 436, and the like.

The information receiving unit 431 receives the request information including the required quality information indicating the quality required by each delivery destination from the terminal device 50 of the packet delivery destination via the communication unit 41. For example, the information receiving unit 431 of the distribution control device 40b receives the request information from each terminal device 50 which is the distribution destination. Further, the information receiving unit 431 includes the highest required quality information indicating the highest quality among the qualities required by each delivery destination of the other delivery control devices 40 from the other delivery control device 40 which is the delivery destination of the packet. The request information is also received via the communication unit 41. For example, the information receiving unit 431 of the distribution control device 40a includes the highest required quality information indicating the highest quality among the qualities required by each terminal device 50 to which the distribution control device 40b is distributed from the distribution control device 40b. Receive request information.

When the information receiving unit 431 receives the request information from each device of the packet delivery destination, the information update unit 432 updates the delivery destination information 422 based on the received request information. Then, the information updating unit 432 calculates the highest required quality information indicating the highest quality among the qualities (required quality information) required by each delivery destination shown in the updated delivery destination information 422.

The information transmission unit 433 transmits the request information including the highest required quality information calculated by the information update unit 432 to the distribution control device 40 of the packet transmission source via the communication unit 41.

The packet receiving unit 434 receives the extended packet delivered from the data transmitting device 20 or another distribution control device 40 via the communication unit 41.

The control unit 435 controls the delivery of the packet to each delivery destination based on the quality information and the delivery destination information 422 included in each extended packet received from the data transmission device 20 or another delivery control device 40. When the delivery destination indicated by the delivery destination device address of the delivery destination information 422 is another delivery control device 40, the control unit 435 obtains the quality information contained in the packet for each extended packet as the required quality information from the delivery destination. An extended packet based on an extended packet whose quality shown in the quality information is higher than the quality shown in the required quality information as compared with (the highest required quality information shown in the required information received from the other delivery control device 40). Is controlled so as not to be delivered to the other delivery control device 40. Further, when the delivery destination indicated by the delivery destination device address of the delivery destination information 422 is the terminal device 50, the control unit 435 requests the quality information included in the packet for each extended packet (request quality information from the delivery destination). The delivery packet based on the extended packet whose quality shown in the quality information is higher than the quality shown in the required quality information as compared with the required quality information shown in the required information received from the terminal device 50). Control so that it is not delivered to. On the other hand, when the quality information of the extended packet received from the data transmission device 20 or another delivery control device 40 is equal to or lower than the quality indicated in the corresponding required quality information of the delivery destination information 422, the control unit 435 said the extended packet. The packet (extended packet or distribution packet) based on the above is controlled to be distributed to another distribution control device 40 or terminal device 50 via the communication unit 41.

In this way, the packet distribution unit 436 distributes the expansion packet or the distribution packet based on the expansion packet received by the packet reception unit 434 to the other distribution control device 40 or each terminal device 50 via the communication unit 41.

7 (a) to 7 (c) are schematic views showing an example of the data structure of the distribution packet. As shown in FIGS. 7 (a) to 7 (c), the distribution packet has a header part and a payload part like the extended packet.

However, the header section has the delivery destination device address instead of the delivery control device address, and the quality information is removed from the header section. The delivery destination device address is the delivery destination device address in the item determined to correspond to each type of the image transmission device ID and the required quality information in the delivery destination information 422. Similar to the configuration of the extended packet, the configuration of the distribution packet is also classified into three types according to the configuration of the payload unit.

When the control unit 435 receives the extended packet (FIG. 4 (b)) containing the image string format information, the control unit 435 generates a distribution packet (FIG. 7 (a)) containing the image string format information. Further, when the control unit 435 receives the extended packet (FIG. 4 (c)) containing the image format information, the control unit 435 generates a distribution packet (FIG. 7 (b)) containing the image format information. On the other hand, when the control unit 435 receives the extended packet (FIG. 4D) that does not include the image string format information and the image format information, the control unit 435 does not include the image string format information and the image format information (FIG. 7 (FIG. 7). c)) is generated.

The terminal device 50 is a server arranged in a data center for long-term storage of the captured image transmitted by the image transmitting device 10, and a monitoring control center for monitoring the captured image transmitted by the image transmitting device 10 by a controller. It is a personal computer to be arranged, or a mobile terminal (mobile phone) owned by a person who responds to the situation by rushing to the security area 2. Each terminal device 50 has a communication unit and a display unit, and receives and receives a distribution packet based on an image packet transmitted by the image transmission device 10 from the distribution control device 40 via the communication unit. The image string obtained by decoding the reconstructed data reconstructed by the data contained in is displayed on the display unit.

FIG. 8 is a sequence diagram relating to the update processing of the distribution destination information of the data distribution system 1 according to the present embodiment, and illustrates the processing performed between the distribution control devices 40a and 40b and the terminal device 50a. .. The operation sequence described below is executed in cooperation with each element of each device mainly by the control unit of each device based on the program stored in advance in the storage unit of each device of the data distribution system 1. Will be done.

First, the terminal device 50a generates request information according to an instruction by a user using an input device (not shown) or an instruction from a management device (not shown) of the terminal device 50a via a communication unit (step). S101). The request information is generated for each image transmission device 10 (image transmission device ID) that the terminal device 50a desires to receive image data. For example, when the terminal device 50a wishes to receive image data from the three image transmission devices 10, the request information is generated for each of the three image transmission devices 10. The request information may be stored in advance in the terminal device 50a.

FIG. 6B is a schematic diagram showing an example of the data structure of the request information. As shown in FIG. 6B, the request information includes information such as a distribution source device address, an image transmission device ID, a distribution destination device address, and required quality information.

The distribution source device address is the address of the distribution source device (distribution control device 40b) that distributes the distribution packet to the terminal device 50.

As the image transmission device ID, the image transmission device ID of the image transmission device 10 for which the terminal device 50 that transmits the request information transmits the image data desired to be received is set.

The delivery destination device address is an address (IP address, port number, etc.) of the delivery destination device (terminal device 50 for transmitting request information) of the delivery packet.

The required resolution type, the required image quality type, and the required frame rate type, which are the required quality information, are information indicating the resolution, image quality, and frame rate of the image data required to be received and displayed by the terminal device 50, respectively. In the present embodiment, an example in which a plurality of types of qualities are included in the required quality information will be described, but the required quality information may include only one type of quality. As described above, the required resolution type, the required image quality type, and the required frame rate type are set with the respective values defined as the resolution type, the image quality type, and the frame rate type.

Next, in step S102 of FIG. 8, the terminal device 50 transmits the generated request information to the distribution control device 40b of the packet distribution source via the communication unit (step S102). Next, the information receiving unit 431 of the distribution control device 40b receives the request information from the terminal device 50 via the communication unit 41 (step S103). Next, the information updating unit 432 updates the delivery destination information 422 based on the request information received by the information receiving unit 431 (step S104).

Next, the information update unit 432 of the distribution control device 40b is the highest request indicating the highest quality among the required quality information included in the delivery destination information 422, that is, each required quality information shown in all the received request information. The quality information is calculated (step S105). The highest required quality information includes information on multiple types of quality. The highest required quality information is generated for each image transmission device ID of the delivery destination information 422. For example, when three image transmission device IDs are registered in the delivery destination information 422, three maximum required quality information is generated for each image transmission device ID.

Here, the calculation of the highest required quality information in the information updating unit 432 will be specifically described. For example, one delivery destination of an image packet transmitted from a specific image transmission device 10 has a required resolution type of 1920x1080, a required image quality type of medium image quality, a required frame rate type of 15 fps, and a required resolution type of another delivery destination. Is 1024x720, the required image quality type is low image quality, and the required frame rate type is 30 fps, the required resolution type related to the highest required quality information is 1920x1080, the required image quality type is medium image quality, and the required frame rate type is 30 fps. The highest required quality information is set so that each delivery destination can receive data having the required quality. Therefore, for example, if the required frame rate type of one delivery destination is 15 fps and the required frame rate type of the other delivery destination is 20 fps, the required frame rate type related to the highest required quality information is 15 fps and 20 fps instead of 20 fps. The frame rate is 60 fps, which is the least common multiple of. As described above, the highest quality indicated by the highest required quality information means the quality including all of the qualities indicated in each required quality information.

Next, the information update unit 432 of the distribution control device 40b generates the required information based on the calculated highest required quality information (step S106). The request information is generated for each generated highest required quality information, that is, for each image transmission device ID of the delivery destination information 422.

The data structure of the request information generated by the information update unit 432 is the same as the data structure of the request information shown in FIG. 6 (b). However, the distribution source device address is the address of the distribution control device 40a that distributes the extended packet to the distribution control device 40b, and the distribution destination device address is the address of the distribution control device 40b that is the distribution destination of the extended packet. Become. The information updating unit 432 specifies the address of the distribution source from the distribution source information 421 stored in the storage unit 42. The required resolution type, the required image quality type, and the required frame rate type, which are the required quality information, are the required resolution type, the required image quality type, and the required frame rate type of the highest required quality information calculated, respectively. As described above, the required quality information of the required information generated by the distribution control device 40 is equal to the highest required quality information calculated by the information updating unit 432.

Next, the information transmission unit 433 transmits the request information generated by the information update unit 432 to the distribution control device 40a of the packet distribution source via the communication unit 41 (step S107). Next, the information receiving unit 431 of the distribution control device 40a receives the request information from the distribution control device 40b via the communication unit 41 (step S108). Next, the information updating unit 432 updates the delivery destination information 422 based on the request information received by the information receiving unit 431 (step S109). The data structure of the delivery destination information 422 updated by the delivery control device 40a is the same as the data structure of the delivery destination information 422 shown in FIG. 6 (b). This ends the sequence related to the update processing of the delivery destination information.

When the distribution control device 40b is in communication connection with the distribution control device 40a via another distribution control device 40, the distribution control device 40b does not send the request information to the distribution control device 40a, but another distribution control device. Send to 40. In that case, the distribution source device address of the transmitted request information becomes the address of the other distribution control device 40. On the other hand, when the other distribution control device 40 receives the request information, the distribution destination information 422 is updated based on the received request information. In that case, the delivery destination information 422 of the other delivery control device 40 includes each item (image transmission device ID, required quality information) in which the delivery destination device address is the address of the delivery control device 40b. The other distribution control device 40 calculated and calculated the highest required quality information so as to include the required quality information (the highest required quality calculated by the distribution control device 40b) shown in the required information received from the distribution control device 40b. The request information based on the highest required quality information is transmitted to the delivery control device 40a, which is the delivery source of the packet.

Further, the sequence related to the update process of the delivery destination information is executed every time the terminal device 50 transmits the request information. That is, each time the information update unit 432 of each distribution control device 40 receives request information from the device (terminal device 50 or other distribution control device 40) to which the packet is delivered, the required quality included in the delivery destination information 422. Update the information and recalculate the highest required quality information. Then, the information transmission unit 433 retransmits the request information based on the recalculated highest required quality information to the distribution control device 40 of the distribution source of the extended packet.

Each terminal device 50 and each distribution control device 40 stores in advance the address of the distribution source distribution control device 40 that distributes the packet to each device, and transmits the request information to the stored address. However, when each distribution control device is mounted on a mobile body and communication is performed between the distribution control devices via a network such as a wireless ad hoc network where the communication path is unstable, each device is connected to the distribution control device. Each time the wireless communication is established, the request information may be transmitted by using the distribution control device for which the wireless communication has been established as the device of the packet distribution source.

FIG. 9 is an overall operation sequence diagram of the data distribution system 1 according to the present embodiment. The operation sequence described below is executed in cooperation with each element of each device mainly by the control unit of each device based on the program stored in advance in the storage unit of each device of the data distribution system 1. Will be done.

First, each image transmission device 10 encodes an image sequence acquired by a network camera into a plurality of data having different qualities, and generates an image packet related to each encoded data (step S201).

Next, each image transmission device 10 transmits the image packet generated in step S201 to the data transmission device 20 via the communication unit (step S202). Next, the extended packet generation unit 241 of the data transmission device 20 receives an image packet from each image transmission device 10 via the first communication unit 21 (step S203).

Next, the extended packet generation unit 241 executes an extended packet generation process for generating an extended packet based on the extended packet generation information 231 for each received image packet (step S204).

FIG. 10 is a flowchart showing an example of the operation of the extended packet generation process. The extended packet generation process shown in FIG. 10 is executed every time an image packet is received. Further, the extended packet generation unit 241 does not execute the extended packet generation process for the image packet received from each image transmission device 10 until the image packet including the image string format information is first received. That is, the extended packet generation process shown in FIG. 10 is executed from the image packet including the image string format information, and is executed from the head data of the reference frame as the partial image data.

First, the extended packet generation unit 241 determines whether or not the received image packet contains the image string format information (step S301). When the received image packet contains the image string format information (Yes in step S301), the extended packet generation unit 241 determines the quality of the image data included in the image packet (the image data is regenerated) based on the image string format information. The quality of the reconstructed data achieved by being included in the configuration data) is specified, and the extended packet generation information 231 is stored (initialized) (step S302).

That is, in step S302, the extended packet generation information 231 includes an image transmission device ID included in the image packet in which the image string format information is described, a resolution type included in the image string format information, and an image quality included in the image string format information. It is initialized with the frame rate value calculated from the frame rate information included in the type and image string format information. Further, the frame number of the extended packet generation information 231 is set to the initial value (0), and the frame type is also set to the initial value (information indicating the reference frame). Further, as a set of the frame number interval and the frame rate type, a special value (for example, (0,0)) including all frame rates is set as an initial value.

In step S301, when the received image packet does not include the image string format information (No in step S301), the extended packet generation unit 241 determines whether or not the received image packet contains the image format information (No). Step S304). When the received image packet contains the image format information (Yes in step S304), the extended packet generation unit 241 updates the extended packet generation information 231 based on the image format information (step S305).

In step S305, the extended packet generation unit 241 identifies the frame interval (GOP) at which the reference frame occurs from the frame type included in the image format information, obtains a set of the frame number interval and the frame rate type, and responds. It is set in the extended packet generation information 231 of the image string to be used. Here, the corresponding image string is the image string specified in S302 (that is, the value initialized in the image string format information of the image packet in the extended packet generation information 231, that is, the image transmission device ID, the resolution. An image string specified by type, image quality type, and frame rate type). Further, since the image format information is stored in the image packet at the head of each frame, the extended packet generation unit 241 increments (increments) the frame number of the corresponding image string of the extended packet generation information 231. Further, the extended packet generation unit 241 sets the frame type included in the image format information in the frame type of the corresponding image string of the extended packet generation information 231.

When the extended packet generation information 231 is initialized in step S302, when the extended packet generation information 231 is updated in step S305, or when the image packet received in step S304 does not include image format information (in step S304). No), the extended packet generation unit 241 generates an extended packet based on the extended packet generation information 231 (step S303). At this time, the quality information of the extended packet is set by using the value set in the extended packet generation information 231.

However, regarding the frame rate type, the setting method differs depending on whether the received image packet contains the image string format information or not. When the received image packet contains image string format information, a special value (for example, (0,0)) set in the extended packet generation information 231 is set as the frame rate type set in the extended packet. To.

On the other hand, when the received image packet does not include the image string format information, the frame rate type set in the extended packet is obtained as follows. First, the extended packet generation unit 241 divides the frame number interval stored in the extended packet generation information 231 by subtracting 1 from the frame number stored in the extended packet generation information 231. Extract the frame number interval that becomes 0. Then, the extended packet generation unit 241 sets the frame rate type corresponding to each extracted frame number interval as the frame rate type of the quality information. For example, in the extended packet generation information 231 as a set of frame number interval and frame rate type {(1, 60), (2, 30), (4, 15), (6, 10), (10, 6), When (12, 5), (20, 3), (30, 2), (60, 1)} is set, the frame number 2 is set as the frame number interval for the extended packet whose frame number is 2. Only 1 whose remainder is 0 when divided by 1, which is the value obtained by subtracting 1 from, is extracted, and 60 is set as the frame rate type of the quality information. By the process described later, this extended packet is delivered to the terminal device 50 that requests the image data having a frame rate of 60, but is not delivered to the terminal device 50 that requests the image data having a frame rate of less than 60.

Next, the extended packet generation unit 241 generates an extended packet based on the extended packet generation information 231 in step S303 of FIG. 10, and then ends a series of steps of the extended packet generation process.

As described above, the quality of the data contained in the image packet (the quality of the reconstructed data achieved by including the data contained in the image packet in the reconstructed data) is described in the payload portion of the image packet. The extended packet generation unit 241 specifies the quality of the data included in the image packet by reading the quality described in the payload unit of the image packet. Then, the extended packet generation unit 241 generates an extended packet in which the specified quality is added to the header unit as quality information.

The extended packet generation unit 241 may specify the quality of the data included in the image packet by analyzing the payload unit of the image packet. For example, the image data included in the payload section is RAW data in which each pixel with RGB values is arranged so that the vertical resolution is 4: 3 with respect to the horizontal resolution, and the start data is at the beginning of each frame. The case where it is included and the stop data is included at the end of each frame will be described. The extended packet generation unit 241 detects start data and stop data from one or more image packets, and specifies the resolution of the image from the number of pixels between the start data and the stop data. Further, the extended packet generation unit 241 specifies the frame rate of the image from the interval in which the start data exists.

As a result, the extended packet generation unit 241 can specify the quality of the data included in the image packet even for the image packet in which the quality of the data included in the image packet is not described in the payload unit.

Next, the extended packet transmission unit 242 transmits the extended packet generated in the extended packet generation process of step S204 of FIG. 9 to the distribution control device 40a via the second communication unit 22 (step S205). Next, the packet receiving unit 434 of the distribution control device 40a receives the extended packet from the data transmitting device 20 via the communication unit 41 (step S206). Next, the control unit 435 executes the delivery destination specifying process in order to specify the delivery destination of the packet based on the received extended packet (step S207).

FIG. 11 is a flowchart showing an example of the operation of the delivery destination specifying process. The delivery destination specifying process shown in FIG. 11 is executed for each extended packet received from the data transmission device 20.

First, the control unit 435 identifies the image transmission device ID included in the received extended packet, and the specified image transmission device ID corresponds to any of the image transmission device IDs stored in the delivery destination information 422. It is determined whether or not to do so (step S401).

When the specified image transmission device ID does not correspond to any of the image transmission device IDs stored in the delivery destination information 422 (No in step S401), the control unit 435 performs a series without executing any particular processing. End the step of. In this case, the extended packet or the delivery packet is not delivered to any delivery destination (other delivery control device 40 or terminal device 50).

On the other hand, when the specified image transmission device ID corresponds to any of the image transmission device IDs stored in the distribution destination information 422 (Yes in step S401), the control unit 435 receives the distribution destination information 422 from the distribution destination information 422. Items corresponding to the specified image transmission device ID (image transmission device ID, delivery destination device address, required resolution type, required image quality type, required frame rate type, which are required quality information) are extracted.

Next, the control unit 435 compares the required quality information of the extracted item with the quality information included in the header unit of the received extended packet for each delivery destination (delivery destination device address) extracted from the delivery destination information 422. Is performed, and it is determined whether or not the quality indicated in the quality information included in the header portion of the extended packet is higher than the quality indicated in the required quality information (step S402). The control unit 435 includes the quality shown in the quality information included in the received extended packet, that is, the partial image data included in the reconstructed data when the partial image data included in the extended packet is reconstructed at the delivery destination. Identify (estimate) the quality of the reconstructed data achieved by this. In the present embodiment, the resolution type, the image quality type, and the frame rate type included in the quality information indicate the quality of the reconstructed data. The control unit 435 indicates that the quality shown in the quality information of the extended packet is shown in the required quality information corresponding to the image transmission device ID of the source of the extended packet and the delivery destination device address of the delivery destination in the delivery destination information 422. Determine if it is higher than the quality.

When the quality of any kind shown in the quality information of the extended packet is higher than the quality shown in the required quality information of the delivery destination information 422, the control unit 435 determines that the quality of the reconstructed data related to the extended packet is the required quality. Judged as higher than the quality shown in the information. On the other hand, when the quality of all kinds shown in the quality information of the extended packet is lower than all the qualities shown in the required quality information of the delivery destination information 422, the control unit 435 changes the quality of the reconstructed data to the required quality information. Determined to be lower than the indicated quality.

When the quality of the reconstructed data is not higher than the quality shown in the required quality information (No in step S402), the control unit 435 generates various packets to be delivered to the delivery destination based on the received extended packet (step S403). ). That is, when the delivery destination is the delivery control device 40, an extended packet is generated, and when the delivery destination is the terminal device 50, a delivery packet is generated.

The extended packet to be delivered to the delivery control device 40 has a header unit and a payload unit, similarly to the extended packet received from the data transmission device 20. However, in the header unit, instead of the distribution control device address of the own distribution control device 40, the distribution destination device address of another distribution control device to be the distribution destination is shown. For example, the distribution control device address of the extended packet generated in step S207 of FIG. 9 is the distribution control device address of the distribution control device 40b.

On the other hand, when the quality of the reconstructed data related to the received extended packet is higher than the quality shown in the required quality information of the delivery destination information 422 (Yes in step S402), the control unit 435 may use another delivery control device 40 or a terminal device. Does not generate a packet to be delivered to 50.

When the processes of steps S402 and S403 are completed for all the corresponding delivery destinations, the delivery destination identification process ends.

Next, in step S208 of FIG. 9, the packet distribution unit 436 transmits the extended packet generated by the control unit 435 to the distribution control device 40b of the distribution destination device address included in the extended packet via the communication unit 41. (Step S208). On the other hand, if the quality of the reconstructed data related to the received image packet is higher than the highest required quality of the delivery destination, the extended packet is not generated for the delivery destination. As a result, the control unit 435 controls so that the quality of the reconstructed data related to the received image packet does not transmit the extended packet higher than the quality indicated in the required quality information of the delivery destination to the delivery destination.

Next, the packet receiving unit 434 of the distribution control device 40b receives the extended packet from the distribution control device 40a via the communication unit 41 (step S209). Next, the delivery control device 40b executes a delivery destination specifying process in order to specify the delivery destination of the packet based on the received extended packet (step S210). The distribution destination specifying process executed by the distribution control device 40b is the same process as the distribution destination specifying process executed by the distribution control device 40a in step S207. However, the distribution destination device in the distribution control device 40b is the terminal device 50, and the control unit 435 of the distribution control device 40b determines the quality of the quality information included in the header unit of the received extended packet in step S209 and each distribution destination. The quality shown in the required quality information is compared with the quality shown in the required quality information, and it is determined whether or not the quality shown in the quality information is higher than the quality shown in the required quality information of each delivery destination. Then, the control unit 435 of the distribution control device 40b generates a distribution packet to be distributed to the terminal device 50 in step S403.

When the distribution packet is generated in step S210 of FIG. 9, the packet distribution unit 436 transmits the distribution packet generated by the control unit 435 to each device (delivery destination device address) included in each packet via the communication unit 41. For example, it is transmitted to the terminal devices 50a and 50b) (step S211). Next, each terminal device 50 receives a distribution packet from the distribution control device 40b via the communication unit (step S212). Next, each terminal device 50 generates reconstruction data from the payload unit of the received distribution packet, decodes the reconstruction data, restores the image string, and displays it on the display unit (step S213). As a result, the entire operation sequence of the data distribution system 1 ends.

As described above, in the data distribution system 1 according to the embodiment of the present invention, the distribution control device 40 controls the distribution of the packet related to the image string, so that the data related to the image string of the quality required by the terminal device 50 can be obtained. Send. Therefore, even if a plurality of terminal devices 50 require image strings of different qualities, the image transmitting device 10 does not need to transmit data encoded with different qualities for each terminal device 50. Alternatively, it is not necessary to temporarily record the data encoded with a specific quality on a predetermined server such as an image recording device, convert the data into the data of the quality required by each terminal device 50 on the server, and transmit the data. Alternatively, the terminal device 50 that has received the data encoded with a specific quality does not need to convert the data into the data of the quality required by itself. Therefore, the data distribution system 1 can efficiently distribute data of the quality required by the distribution destination while suppressing an increase in the amount of communication.

Further, the distribution control device 40b requests the distribution control device 40a, which is the distribution source of the packet, as the request information the highest quality (highest required quality) of the quality required by each terminal device 50 of the packet distribution destination. The delivery control device 40a controls the packet to be transmitted to the delivery control device 40b so that the quality of the reconstructed data reconstructed by each terminal device 50, which is the final delivery destination of the packet, does not become higher than the highest required quality. ..

For example, the distribution control device 40b includes a terminal device 50a having a required resolution type of 1920x1080, a required image quality type of medium image quality, and a required frame rate type of 15 fps, and a required resolution type of 1024x720, a required image quality type of low image quality, and a required frame rate. The packet is delivered to the terminal device 50b whose type is 30 fps, and the delivery control device 40c has the required resolution type 3840x2160, the required image quality type is medium image quality, and the required frame rate type is 30 fps. Will be described in the case of delivering a packet to a terminal device 50d having 3840x2160, a required image quality type of low image quality, and a required frame rate type of 15 fps.

In this case, the required resolution type related to the highest required quality information of the distribution control device 40b is 1920x1080, the required image quality type is medium image quality, and the required frame rate type is 30 fps. Further, the required resolution type related to the highest required quality information of the distribution control device 40c is 3840x2160, the required image quality type is medium image quality, and the required frame rate type is 30 fps. Therefore, the distribution control device 40a distributes only the extended packet including the data whose resolution type is 1920x1080 or less, the image quality type is medium image quality or less, and the frame rate type is 30 fps or less to the distribution control device 40b. Further, the distribution control device 40a distributes and controls only extended packets containing data having a resolution type of 3840x2160 or less, an image quality type of medium image quality or less, and a frame rate type of 30 fps or less with respect to the distribution control device 40c. Deliver to device 40c. That is, the distribution control device 40a has an extended packet for quality information whose resolution type is higher than 1920x1080, an extended packet for quality information whose image quality type is higher than medium image quality, or an extended packet for quality information whose frame rate type is higher than 30 fps. Is not delivered to the delivery control device 40b. As described above, the distribution control device 40a transmits only the extended packet having the quality shown in the highest required quality information to the distribution control device 40b and the distribution control device 40c, which are the distribution destinations of the distribution control device 40a. Therefore, it is possible to efficiently distribute the quality data required by the distribution destination while suppressing the increase in the amount of communication between the distribution control devices 40.

Further, every time the quality required by the terminal device 50 is changed or added, the distribution control devices 40b and 40c recalculate the highest required quality information and transmit it to the distribution control device 40a as the required information. Then, the delivery control device 40a controls the delivery of the packet based on the received highest required quality information. Therefore, it is possible to suppress an increase in the amount of communication between the distribution control devices 40 in response to the dynamic change / addition of the quality required by the terminal device 50.

The present invention is not limited to the above-described embodiment, and may be further implemented in various different embodiments within the scope of the technical idea described in the claims. Moreover, the effect described in the embodiment is not limited to this.

In the above embodiment, the case where the distribution control device 40b communicates with each terminal device 50 is described. However, the present invention is not limited to this, and the distribution control device 40b may communicate with the terminal device 50 via another distribution control device 40. In this case, the delivery destination device address extracted in the delivery destination identification process executed in step S210 includes the address of another delivery control device 40. In that case, the distribution control device 40b generates an extended packet to be distributed to the other distribution control device 40 in step S303, and distributes the generated extended packet to the other distribution control device 40 in step S212.

Further, as in the above embodiment, when the resolution required by the terminal device 50a is higher than the resolution required by the terminal device 50b and the frame rate required by the terminal device 50a is lower than the frame rate required by the terminal device 50b, distribution is performed. The control device 40b does not transmit the required information including the highest required quality information in these qualities, but describes the required quality information related to the terminal device 50a and the required quality information related to the terminal device 50b. The request information may be distributed separately to the distribution control device 40a.

In that case, the information transmission unit 433 of the distribution control device 40b is the extended packet (that is, the required quality information of the terminal devices 50a and 50b) required to achieve the highest quality of the quality required by each of the terminal devices 50a and 50b. Data amount (data rate) per unit time between the distribution control device 40a and the distribution control device 40b when the distribution control device 40a transmits an extended packet (extended packet that achieves the quality of the highest required quality information obtained from) (data rate). First data amount) is calculated. Further, the information transmission unit 433 of the distribution control device 40b includes an extended packet required to achieve the quality required by the terminal device 50a and an extended packet required to achieve the quality required by the terminal device 50b. Is transmitted separately by the distribution control device 40a, the total amount of data per unit time between the distribution control device 40a and the distribution control device 40b (second data amount) is calculated. The amount of data per unit time corresponding to each quality is measured or calculated and set in advance.

The information transmission unit 433 of the distribution control device 40b compares the first data amount and the second data amount, and (a) when the first data amount is less than the second data amount, the terminal device 50a, The required information describing the highest required quality information indicating the highest quality of the quality shown in each required quality information of 50b is transmitted to the delivery control device 40a. On the other hand, in the information transmission unit 433 of the distribution control device 40b, (b) when the first data amount is equal to or larger than the second data amount, the request information describing the required quality information of the terminal device 50a and the request information of the terminal device 50b The required information describing the required quality information is separately transmitted to the distribution control device 40a.

When the information receiving unit 431 of the distribution control device 40a receives the request information describing the required quality information of the terminal devices 50a and 50b from the distribution control device 40b (that is, in the case of (b)), the control of the distribution control device 40a. The unit 435 controls to separately deliver the extended packet to the terminal devices 50a and 50b based on the received required quality information to the delivery control device 40b. Then, the control unit 435 of the distribution control device 40a distributes the extended packet group whose quality shown in the quality information of the received extended packet is equal to or lower than the quality of the required quality information of the terminal device 50a to the distribution control device 40b. Controlled so that the extended packet group whose quality shown in the quality information of the received extended packet is equal to or lower than the quality of the required quality information of the terminal device 50b is delivered to the delivery control device 40b separately from the extended packet group. do. This makes it possible to further suppress the amount of communication in the communication path between the distribution control device 40a and the distribution control device 40b.

Further, the data distribution system 1 may have only one image transmission device 10 and execute each of the above processes only for the image packet related to the image sequence transmitted by the image transmission device 10.

In the above embodiment, the image transmission device 10 transmits an image packet encoded by a plurality of predetermined different qualities, and the data transmission device 20 transmits the quality shown in the payload of the image packet (reconstruction related to the image packet). The extended packet is generated and transmitted based on the quality of the data), and the delivery control device 40 controls the delivery of the extended packet or the delivery packet by using the quality information of the extended packet and the delivery destination information 422.

However, the present invention is not limited to this, and the image transmission device 10 may transmit an image packet including image data that is hierarchically coded (scalable coded) so that a plurality of different qualities can be reconstructed by the terminal device 50. In this case, each image sequence (multiple frame image) generated by the image transmission device 10 is represented by the data of each layer (basic layer data or one or more extended layer data) in the layer coding. The basic hierarchy data is data that can independently reconstruct the data of the image string having a predetermined quality (lowest quality), and the extended hierarchy data is the basic hierarchy data (and the extended hierarchy data lower than the own data). By being added, the data of the image string having higher quality than the data before the addition can be reconstructed. The quality information shown in the extended packet indicates the quality of the reconstructed data obtained by adding the data of the extended packet to the data lower than the own data. Then, in order to show the hierarchical structure, the reference image string ID is described in the image format information shown in FIG. 2 (e) instead of the frame type. The reference image string ID indicates the image string ID of the hierarchical data one level lower than the own data, and is referred to when the data is reconstructed at the delivery destination. When the image string is basic hierarchical data, a blank is set in the reference image string ID. Then, the delivery control device 40 in the present other embodiment sends an extended packet containing extended hierarchical data higher than the required quality information shown in the delivery destination information 422 to the delivery destination (another delivery control device 40 or a terminal device 50). Do not deliver.

Further, the data distributed by the data distribution system 1 is not limited to image data, but may be other data such as audio data. When the data distribution system 1 distributes audio data, the audio sampling rate and the like are used as the quality, and the distribution control device 40b includes all the qualities required by the distribution destination terminal device of the distribution control device 40b. (Highest required quality) is distributed to the distribution control device 40a as required information. For example, when the distribution control device 40b distributes audio data to a terminal device that requests 8 kHz audio data and a terminal device that requests 16 kHz audio data, the maximum required quality is 16 kHz, and the distribution control device 40a is 16 kHz. It is controlled so that the voice data that is not necessary for reconstructing the voice data is not delivered to the delivery control device 40b.

As described above, in the data distribution system 1 according to the embodiment of the present invention, the quality of the reconstructed data of the distribution control device 40a is higher than the highest required quality of the required information transmitted from the distribution control device 40b. The image data is controlled so as not to be transmitted to the distribution control device 40b. As a result, the distribution system 1 is a terminal that is the final distribution destination while suppressing an increase in communication volume due to distribution of unnecessary quality data between the distribution control device 40a and the distribution control device 40b. Image data having the quality required by the device 50 can be delivered to the delivery destination terminal device 50.

Further, in the data distribution system 1, the image transmission device 10 has a plurality of images having different qualities for each terminal device 50 even when a plurality of terminal devices 50 need to receive image data of different qualities. It is not necessary to transmit data, and it is possible to suppress the amount of communication on the image transmission device 10 side.

Further, in the data distribution system 1, the quality data required by each terminal device 50 is stored in each terminal without individually storing the quality data required by each terminal device 50 in the predetermined image recording device. It can be delivered to the device 50, and the management cost can be reduced.

Further, since each distribution control device 40 immediately identifies and distributes the terminal device 50 to which the packet should be distributed, it is not necessary to store the data related to the received packet. Therefore, it is possible to reduce the storage capacity of each distribution control device 40.

Further, since the terminal device 50 can receive the data of desired quality only by transmitting the required information describing the required quality information to each distribution control device 40, the data distribution system 1 is the terminal device of the distribution destination. It is possible to easily and flexibly respond to the dynamic change of the quality required by the 50 or the terminal device 50.

Further, since each distribution control device 40 distributes the packet related to the predetermined data to each terminal device 50 by selecting the packet for each packet, the terminal device 50 needs to reconstruct the data from the packet. There is no need to convert data to quality (for example, quality conversion of image data), and the processing load related to data conversion in the server that distributes the data (for example, the distribution control device 40) or the terminal device 50 that receives the data. Can be reduced.

Further, since the distribution control device 40 only needs to access the header portion of the data and does not need to access the payload portion (data itself), it is possible to make it difficult for the administrator of the distribution control device 40 to view the data. Therefore, it is possible to improve security and protect privacy. In particular, even when the data in the payload section is encrypted and transmitted, the data can be sorted in packet units based only on the header section, which makes it even more difficult to view the data in the distribution control device 40.

1 data distribution system, 20 data transmission device, 40 distribution control device, 42 storage unit, 431 information reception unit, 432 information update unit, 433 information transmission unit, 434 packet reception unit, 435 control unit, 436 packet distribution unit

Claims (4)

A data transmission device that transmits a plurality of packets including data and quality information indicating a plurality of types of quality, and a packet that receives the plurality of packets and distributes a packet based on at least a part of the plurality of packets. A first distribution control device and a second distribution control device that selects and distributes a packet based on at least a part of the packets received from the first distribution control device for each of a plurality of distribution destinations. It is a data distribution system that has
The second delivery control device is
A storage unit that stores a plurality of required quality information indicating the quality required by each of the plurality of delivery destinations for each type of quality.
An information transmission unit that transmits the highest required quality information indicating the highest quality among the qualities shown in the plurality of required quality information for each type of quality to the first distribution control device.
It has a second control unit that controls delivery to the plurality of delivery destinations based on the quality information included in each packet received from the first delivery control device and the required quality information.
The first distribution control device compares the quality information contained in the packet with the highest required quality information received from the second distribution control device for each packet received from the data transmission device. A first control unit that controls so that any one of the plurality of types of qualities shown in the quality information does not deliver a packet having a quality higher than the quality shown in the highest required quality information to the second delivery control device. A data distribution system characterized by having. A data transmission device that transmits a plurality of packets including data and quality information, and a first distribution control device that receives the plurality of packets and distributes a packet based on at least a part of the plurality of packets. A data distribution system including a second distribution control device that selects and distributes a packet based on at least a part of the packets received from the first distribution control device for each of a plurality of distribution destinations. ,
The second delivery control device is
A storage unit that stores a plurality of required quality information indicating the quality required by each of the plurality of delivery destinations, and a storage unit.
An information transmission unit that transmits the highest required quality information indicating the highest quality among the quality shown in the plurality of required quality information to the first distribution control device, and
It has a second control unit that controls delivery to the plurality of delivery destinations based on the quality information included in each packet received from the first delivery control device and the required quality information.
The first distribution control device compares the quality information contained in the packet with the highest required quality information received from the second distribution control device for each packet received from the data transmission device. It has a first control unit that controls a packet whose quality shown in the quality information is higher than the quality shown in the highest required quality information so as not to be delivered to the second delivery control device.
The information transmission unit indicates that the quality of the first type shown in the first required quality information, which is the required quality information of the first delivery destination, is shown in the second required quality information, which is the required quality information of the second delivery destination. When the quality of the first kind is higher than the quality of the first kind and the quality of the second kind shown in the first required quality information is lower than the quality of the second kind shown in the second required quality information. The first required quality information and the second required quality information are transmitted to the first distribution control device, and the first required quality information and the second required quality information are transmitted to the first distribution control device.
When the first control unit receives the required quality information, the quality information included in the packet, the first required quality information, and the second required quality information are included in each packet received from the data transmission device. A packet based on a packet whose quality shown in the quality information is equal to or lower than the quality shown in the first required quality information is generated and delivered to the second delivery control device, and the quality information is shown. A data distribution system characterized in that a packet based on a packet whose quality shown in is equal to or lower than the quality shown in the second required quality information is generated and delivered to the second delivery control device . A plurality of packets including data and quality information indicating the quality of a plurality of types are received, and the quality information contained in each received packet and the required quality information indicating the quality required by each of the plurality of delivery destinations. To another delivery control device that selects and delivers a packet based on at least a part of each packet for each of the plurality of delivery destinations, a packet based on at least a part of the plurality of packets. It is a delivery control device that delivers
Receives the highest required quality information indicating the highest quality among the plurality of required quality information indicating the quality required by each of the plurality of delivery destinations for each of the quality types from the other delivery control device. Information receiver and
For each of the plurality of received packets, the quality information contained in the packet is compared with the highest required quality information received from the other delivery control device, and the quality of the plurality of types shown in the quality information is compared. A control unit that controls so that one of them does not deliver a packet having a quality higher than that shown in the highest required quality information to the other delivery control device.
A delivery control device characterized by having. A plurality of packets including data and quality information are received, and each packet is based on the quality information contained in each received packet and the required quality information indicating the quality required by each of the plurality of delivery destinations. A delivery control device that delivers a packet based on at least a part of the plurality of packets to another delivery control device that selects and delivers a packet based on at least a part of the packet for each of the plurality of delivery destinations. There,
An information receiving unit that receives the highest required quality information indicating the highest quality among the plurality of required quality information indicating the quality required by each of the plurality of delivery destinations from the other delivery control device.
For each of the plurality of received packets, the quality information contained in the packet is compared with the highest required quality information received from the other delivery control device, and the quality shown in the quality information is the highest required quality. It has a control unit that controls not to deliver a packet having a quality higher than that shown in the information to the other delivery control device .
The information receiving unit indicates that the quality of the first type shown in the first required quality information, which is the required quality information of the first delivery destination, is shown in the second required quality information, which is the required quality information of the second delivery destination. When the quality of the first kind is higher than the quality of the first kind and the quality of the second kind shown in the first required quality information is lower than the quality of the second kind shown in the second required quality information. The first required quality information and the second required quality information are received from the other distribution control device, and the first required quality information and the second required quality information are received.
The control unit compares the quality information contained in the packet with the first required quality information and the second required quality information for each received packet, and the quality shown in the quality information is the first. 1 Generates a packet based on a packet of quality or less indicated in the required quality information and distributes it to the other delivery control device, and the quality indicated in the quality information is equal to or less than the quality indicated in the second required quality information. A delivery control device characterized in that a packet based on a packet is generated and controlled so as to be delivered to the other delivery control device.

Images