JP4268511B2 - Image transmission device - Google Patents

Description

  The present invention relates to an image transmission apparatus, and more particularly to an image transmission apparatus that encodes and transmits an image.

  For example, in the remote image monitoring system, as represented by the Internet, the need for image transmission using an IP network as a conduction path is rapidly expanding. In this case, the image is digitized and transmitted. However, since the information amount of the digitized image becomes enormous, encoding processing (including compression processing) is required. Currently, MPEG-4 / MPEG-2 (moving image compression coding method) and JPEG (still image compression coding method), which are world standard methods, are widely used as the image coding method.

  Here, an example of a network type image monitoring system is shown in FIG. When network-type surveillance systems are put into practical use in the future, as shown in FIG. 2, there is a system in which video transmission using MPEG-4 encoding method and still image transmission using JPEG encoding method are mixed in one network. Become common. FIG. 2 illustrates transmission of an image 2-2 of one monitoring camera 2-1 by a moving image transmission device (WEB encoder (MPEG-4)) 2-3 and a still image transmission device (WEB encoder (JPEG)) 2. This is an example in which transmissions according to -3 are mixed. Generally, video transmission devices are used for applications that emphasize video quality (frame rate (fps: frame per second)) rather than definition (number of pixels) due to transmission capacity limitations. It is used for applications in which fineness is more important than that. The data 2-5 and 2-6 of the video transmission device 2-3 and the still image transmission device 2-4 are transferred as data 2-8 to a network 2-9 such as a LAN or the Internet by a hub (repeater) 2-7. Is output. On the other hand, on the receiving side, data 2-10 from the network 2-9 is converted into data 2-12 and 2-13 by the hub 2-11 as a moving picture receiving device (WEB decoder (MPEG-4)) 2-14 and a still picture, respectively. The data is input to a receiving device (WEB decoder (JPEG)) 2-17. The monitor 2-15 displays the image 2-16 output from the moving image reception device 2-14, and the monitor 2-19 displays the image 2-18 output from the still image reception device 2-17.

  Conventionally, in a network that normally performs moving image transmission, the transmission rate of compressed moving image data is lowered to a transmission rate of a predetermined ratio of the transmission capacity. There has been proposed a method for transmitting compressed moving image data and high-definition image data simultaneously by transmitting fine image data (for example, Patent Documents 1 and 2).

  2, the encoded data rate (bps: bit per second) of the moving picture transmission apparatus 2-3 is 800 kbps (320 × 480 pixels, 30 fps), and the encoded data rate of the still picture transmission apparatus 2-4 is 1. Assuming 5 Mbps (640 × 480 pixels, 3 fps), the transmission capacity of the network is required to be 2.3 Mbps or more. The transmission capacity of this example is possible with general ADSL of the Internet. However, since the transmission capacity fluctuates dynamically on the Internet, there is no guarantee that 2.3 Mbps or more is always secured.

  FIG. 3 shows a case where the transmission capacity of the network 2-9 is temporarily reduced from 2.3 Mbps or more to 1.5 Mbps (down to about 65%). This reduction in network transmission capacity inevitably affects the encoded data rate that can be output by the moving picture transmission apparatus 2-3 and the still picture transmission apparatus 2-4. Usually, since the output decreases at the same rate, the output of the moving image transmission apparatus is changed from 800 kbps to 520 kbps (800 kbps × (1.5 Mbps / 2.3 Mbps)), and the output of the still image transmission apparatus 2-4 is started from 1.5 Mbps. It decreases to 980 kbps (1.5 Mbps × (1.5 Mbps / 2.3 Mbps)). On the other hand, since the encoding processing unit in each transmission device continues to encode data of 800 kbps and 1.5 Mbps, data that cannot be transmitted inside each transmission device is discarded. That is, the encoded data 2-5 (520 kbps) and the encoded data 2-6 (980 kbps) in FIG. 3 are encoded data partially missing.

  The partially missing encoded data is input to the moving image receiving device 2-14 and the still image receiving device 2-17, respectively, and outputs images 2-16 and 2-18, but monitors 2-15 and 2-19. Causes block noise or frame skipping due to data loss. However, the effect of the omission is less in the output image of the still image receiving device 2-17 than in the output image of the moving image receiving device 2-14. This is because the JPEG encoding method for still images is based on intra-frame encoding, and the MPEG-4 encoding method for moving images is based on inter-frame encoding. Therefore, the JPEG encoding method does not cause error propagation due to partial data loss.

  The reason for the lack of data will be specifically described with reference to FIG. FIG. 4A shows the configuration of MPEG-4 encoded data in the moving picture transmission apparatus. MPEG-4 encoded data is a P (predictive) frame (inter-frame encoding: only the difference from the previous frame in time is encoded, and an image cannot be reproduced without the previous frame data. In this configuration, a periodic I (intra) frame (intra-frame encoding: encoded using data of only one's own frame) is inserted. Here, when missing of encoded data occurs in a part of the P frame, the encoded data up to the next I frame cannot be normally decoded. This is because the P frame performs a prediction process between the frames, and an error propagates to the frame after the loss. Note that interframe coding is also called predictive coding, for example. In addition, an I frame that is intra-frame coding is a frame that is coded without prediction, and is referred to as a reference frame that is a base point of predictive coding.

On the other hand, FIG. 4B shows the structure of JPEG encoded data. Since all JPEG encoded data is composed of I frames (intraframe encoding), there is no error propagation due to missing encoded data unlike MPEG-4 encoded data. Therefore, only the portion of the frame in which data loss has occurred only causes block noise or frame skipping, and therefore does not become a major obstacle.
JP 2003-219402 A JP 2003-284044 A

  In the conventional technique, due to a decrease in the transmission capacity of the network, partial omission occurs due to not being able to transmit to each encoded data of moving image transmission and still image transmission. In particular, moving picture transmission that is greatly affected by missing encoded data has a problem that block noise propagates to several frames, and the actual damage is large and it is difficult to realize a reliable system.

  The present invention has been made in view of such a conventional situation, and an object thereof is to provide an image transmission apparatus that preferentially transmits specific encoded data, for example.

  An image transmission apparatus according to the present invention includes an encoding unit that encodes an image with different code amounts, at least first and second buffer units that store encoded data from the encoding unit, and the buffer unit, respectively. A buffer control unit that monitors the amount of encoded data to be accumulated, and the buffer control unit has a predetermined threshold value that determines at least the amount of encoded data to be accumulated in the first buffer unit, and When the amount of encoded data in the first buffer unit exceeds the predetermined threshold value, the encoded data is read from the first buffer unit. If the predetermined threshold value is not exceeded, the encoded data is read from the second buffer unit.

  In the image transmission apparatus of the present invention, the data stored in the first buffer unit is moving image encoded data encoded by intraframe encoding and interframe encoding, and the second buffer The data stored in the unit is configured to be still image encoded data encoded by intraframe encoding.

  Further, in the image transmission apparatus of the present invention, the second buffer unit is configured to have a capacity sufficient to buffer the still image encoded data without discarding even when the transmission capacity is reduced. Is done.

  According to the image transmission apparatus of the present invention, specific encoded data is preferentially transmitted. For example, in a case where the transmission capacity temporarily decreases in a best effort network such as the Internet. However, it is possible to realize optimal image transmission with little influence such as loss of encoded data due to insufficient transmission capacity.

Embodiments according to the present invention will be described with reference to the drawings.
FIG. 1 shows an embodiment of a system configuration using a transmission apparatus according to the present invention. The image transmission device 1-1 receives the image data 2-2 and outputs the encoded data 1-7.

  The image data 2-2 is input to the moving image encoding unit 1-2 and the still image encoding unit 1-3 in the image transmission apparatus 1-1. The encoded data 1-4 and 1-5, which are the outputs of the respective encoding units, are input to the moving picture priority protocol processing unit (buffer control unit) 1-6, and the network is obtained as encoded data 1-7 by the respective protocol processing. Is transmitted to. The other configuration in FIG. 1 is the same as the configuration shown in FIG. 2 described above. For the sake of simplicity, FIG. 1 shows a case where one moving image receiving device and one still image receiving device are installed corresponding to one camera, but the moving image receiving device and the still image receiving device are shown. The number of units can be arbitrarily selected. In FIG. 1, the camera 2-1 and the image transmission device 1-1 are provided separately, but the camera 2-1 and the image transmission device 1-1 can be provided integrally.

  Here, the operation of the transmission apparatus according to an embodiment of the present invention will be described. For example, when the transmission capacity of the network 2-9 decreases from 2.3 Mbps or more to 1.5 Mbps, the moving image priority protocol processing unit 1-6 gives priority to the encoded data 1-4 output from the moving image encoding unit. To transmit. Therefore, since loss of encoded data does not occur, the display of the output of the moving picture reception device 2-14 has no effect even if the network transmission capacity fluctuates. It is also possible to ensure real-time monitoring of the monitor 2-15.

  On the other hand, since 800 kbps of the transmission capacity of 1.5 Mbps is used to transmit the encoded data 1-4 of the moving image encoding unit 1-2, the encoded data that is the output of the still image encoding unit 1-3. 1-5 is transmitted at 700 kbps, and 800 kbps of the encoded data of 1.5 Mbps is originally lost. However, since the encoded data 1-5 of the still image encoding unit 1-3 is intra-frame encoding, 3fps transmission of 640 × 480 pixels is temporarily skipped to about half of 1.5fps temporarily. Since there is an effect to the extent of occurrence, there is almost no practical trouble.

  Here, as described above, as an application of the system in which the image 2-2 from the same camera 2-1 is transmitted as a moving image and a still image, for example, the status of the monitoring area is usually grasped in real time by the moving image. However, when a specific situation change occurs in the monitoring area, a more detailed situation at that time may be confirmed with a high-definition still image. In such a case, for example, since a high-definition still image only needs to be confirmed on the monitor 2-19, the influence is small even if the delay time until reception by the receiving device is longer than that of a moving image.

  Next, FIG. 5 illustrates a specific configuration example of the moving image encoding unit 1-2, the still image encoding unit 1-3, and the moving image priority protocol processing unit 1-6 of the image transmission apparatus 1-1. The moving picture encoding unit 1-2 in FIG. 5 includes an MPEG-4 encoding unit 5-1 and an encoded data buffer 5-3. The still image encoding unit 1-3 includes a JPEG encoding unit 5-4 and an encoded data buffer 5-6. On the other hand, the moving picture priority protocol processing unit 1-6 includes a polling processing unit 5-7 and a packet processing unit 5-8.

  In the moving image encoding unit 1-2 in FIG. 5, the encoded data 5-2 compressed by the MPEG-4 encoding unit 5-1 is encoded before being output to the moving image priority protocol processing unit 1-6. The buffer 5-3 is temporarily buffered. Similarly, in the still image encoding unit 1-3, the encoded data 5-5 compressed by the JPEG encoding unit 5-4 is also temporarily buffered by the encoded data buffer 5-6, and then the moving image priority protocol is used. The data is output to the processing unit 1-6. On the other hand, in the moving picture priority protocol processing unit 1-6, the internal polling processing unit 5-7 polls the buffering state of each of the encoded data buffers 5-3 and 5-6 by signals 5-9 and 5-10. . Depending on the buffering state, the IP packet processing unit 5-8 receives the encoded data from the encoded data buffer 5-3 or the encoded data buffer 5-6, packetizes it, and transmits it to the network by protocol processing. . Various polling intervals of the polling processing unit 5-7 can be used, and may be arbitrarily set according to the actual system implementation requirements.

  Next, a basic processing flow of the moving image priority protocol processing unit 1-6 is shown in FIG. A portion surrounded by a broken line is a processing flow portion regarding the polling processing unit 5-7 and the IP packet processing unit 5-8. Processing 6-1 of the polling processing unit 5-7 is a step of determining whether or not the MPEG-4 encoded data buffer 5-3 is empty. If it is determined that the MPEG-4 encoded data buffer 5-3 is not empty, the process branches to the process 6-3 of the IP packet processing unit 5-8. In processing 6-3 of the IP packet processing unit 5-8, data (for example, MPEG encoded data for one frame) is fetched from the MPEG-4 encoded data buffer 5-3, and the data is converted into an IP packet. Is output to the network. Here, after performing the processing 6-3, the processing 6-2 of the polling processing section is skipped and the process branches to the processing 6-1, so that the MPEG-4 encoded data output is always prioritized. .

  On the other hand, in the process 6-1 of the polling processing unit 5-7, if it is determined that the MPEG-4 encoded data buffer 5-3 is empty, the process branches to the process 6-2 of the polling processing unit. Processing 6-2 of the polling processing unit is a step of determining whether or not the JPEG encoded data buffer 5-6 is empty. If it is determined that the JPEG encoded data buffer 5-6 is not empty, the process branches to the process 6-4 of the IP packet processing unit 5-8. In the process 6-4 of the IP packet processing unit 5-8, data (for example, JPEG encoded data for one frame) is fetched from the JPEG encoded data buffer 5-6, the data is converted into an IP packet, and the IP packet is converted into a network. Output to. After performing the process 6-4, the process branches to the process 6-1 of the polling processing unit. If it is determined in the process 6-2 of the polling processing unit that the JPEG encoded data buffer 5-6 is empty, the process branches to the process 6-1 of the polling processing unit.

  Therefore, when the network transmission capacity is reduced, transmission of MPEG-4 encoded data is prioritized, and therefore data in the JPEG encoded data buffer 5-6 in FIG. 5 cannot be transmitted. If the encoded data that cannot be transmitted is accumulated and a buffer overflow occurs, the loss of JPEG encoded data occurs at this point, but as described above, the influence is small and does not become a problem. On the other hand, MPEG-4 encoded data having a great influence can be transmitted preferentially without omission.

  That is, in the image transmission apparatus that transmits the output of the moving image encoding unit and the still image encoding unit to the network, the moving image encoding protocol processing unit preferentially outputs the transmission of the moving image encoding unit. As a result, in a best-effort network such as the Internet, even if the transmission capacity is temporarily reduced, the portion where the encoded data is lost due to insufficient transmission capacity is made a still image encoding unit with little actual damage. Therefore, it is possible to transmit the moving image encoded data that is seriously damaged by data loss as it is.

  In FIG. 6, the state of the MPEG-4 encoded data buffer 5-3 is determined using a threshold value indicating whether the buffer is empty, but the encoded data amount in the buffer is a predetermined amount (for example, an IP packet). It is possible to make a determination using other criteria such as determining whether or not it exceeds the payload length of one packet in the processing unit. For example, when the determination in the process 6-1 is performed based on a predetermined amount, the threshold value used as the predetermined amount may be arbitrarily set according to the actual system setting status. Similarly, the state of the JPEG encoded data buffer 5-6 can be determined using criteria other than empty.

  In the above-described embodiment, the polling processing unit 5-7 is configured to grasp (monitor) the buffering state of each encoded data buffer by polling. However, the embodiment of the present invention is not limited thereto. For example, various configurations such as detecting a change in the buffering state in each encoded data buffer and generating an interrupt in the moving picture priority protocol processing unit can be used.

  Further, in the above-described embodiment, the processing flow shown in FIG. 6 is performed even when the transmission capacity is not reduced. For example, the processing flow shown in FIG. The transmission rate ratio between MPEG encoded data and JPEG encoded data is transmitted at an arbitrary fixed ratio such as 10:90 or 50:50 while the transmission capacity is not reduced. It is good also as composition to do. As a method for detecting a decrease in transmission capacity, for example, various methods such as monitoring the amount of data transmission per unit time by the IP packet processing unit 5-8 can be used.

  Further, in FIG. 5 described above, the image transmission device 1-1 is configured by a plurality of circuits including a moving image encoding unit 1-2, a still image encoding unit 1-3, and a moving image priority protocol processing unit 1-6. Although an example is shown, the present invention is not limited to this, and any circuit or device may be used as long as it can realize such a function, and part or all of the function is implemented by software. For example, a configuration in which the software is executed by a CPU (Central Processing Unit) may be used. Furthermore, the function realizing means may be realized by a single circuit.

  In the embodiment described above, the delay time of the MPEG-4 encoded data output from the transmission device 1-1 and the moving image receiving device 2-14 by the transmission of the encoded data by the moving image priority protocol processing unit 1-6 is minimized. It is assumed that the transmission capacity is reduced to such a degree that the monitoring 2-15 does not affect the real-time monitoring (for example, the encoded data rate of the moving picture encoding unit 1-2 is 800 kbps, the transmission capacity Is assumed to have dropped to 1.5 Mbps). Therefore, for example, when the transmission capacity is reduced to 500 kbps, it is possible that the data in the MPEG-4 encoded data buffer 5-3 cannot be transmitted, and MPEG-4 encoded data is lost. In such a case, a method for efficiently transmitting MPEG-4 encoded data is known (for example, Japanese Patent Application No. 2002-344385). For example, following an I frame and an I frame according to a transmission delay time. Only the predetermined number of P frames are transmitted, the remaining P frames are deleted from the encoded data buffer 5-3 as appropriate, and the next I frame is transmitted, or the code of the moving image encoding unit 1-2 It is also possible to reduce the occurrence of a practical failure by performing processing for lowering the data rate in the moving image encoding unit 1-2 and the moving image priority protocol processing unit.

  Next, FIG. 7 shows a system example different from FIG. 1 as a system in which moving image transmission and still image transmission are mixed. FIG. 7 shows a system in which moving image transmission is recorded by real-time monitoring, and still image transmission is recorded by a still image recording device 7-1 at a remote location via a network. Even in an actual system, there is an increasing need for recording such a high-definition still image transmission at a remote location.

  When the present invention is applied to a system such as that shown in FIG. 7, the encoded data buffer 5-6 in FIG. Are temporarily buffered. Next, the buffered encoded data is transmitted after the temporary capacity reduction of the transmission path is restored, so that the encoded image data is not lost in the still image recording device 7-1. Recording can be realized. In such a system that uses still image transmission for the purpose of remote recording, the real-time property of still image transmission is not important. Therefore, the encoded data buffer 5-6 can be easily handled by using a large-capacity buffer. It is.

  Therefore, when the transmission capacity is temporarily reduced, the still image encoded data that cannot be transmitted is buffered inside the still image encoding unit, for example, the transmission of the data is resumed after recovering from the transmission capacity decrease. can do. In addition, by having an encoded data buffer that buffers data that cannot be transmitted when the transmission capacity temporarily drops in the still image encoding unit, monitoring and monitoring that always emphasizes real-time performance through stable video transmission and encoding Remote recording by still image transmission without data loss can be realized.

  The above-described embodiment is a system in which video transmission using MPEG-4 encoding method and still image transmission using JPEG encoding method are mixed in one network, that is, images are encoded with different code amounts, respectively. Although a transmission system is described as an example, various systems may be used as encoding methods for moving images and still images, and various systems such as a system in which audio data transmission using a voice encoding method is mixed. A system for transmitting various data may be used.

  In addition, the above-described embodiment is an example of a system that prioritizes moving image transmission over still image transmission, but the present invention is not limited to this, and data that prioritizes transmission is arbitrary according to practical needs. Should be set. For example, priority may be given to still image transmission over moving image transmission. In addition, data other than moving images and still images, for example, audio data may be preferentially transmitted.

  The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.

It is a figure which shows the system configuration example by the transmission apparatus which concerns on one Example of this invention. It is a figure which shows the system configuration example by a transmission apparatus. It is a figure which shows the case where transmission capacity falls temporarily in FIG. It is a figure which shows the influence of the missing coding data in MPEG-4 and JPEG coding data. It is a figure which shows the specific structural example of the transmission apparatus which concerns on one Example of this invention. It is a figure which shows the processing flow of the moving image transmission protocol process part of the image transmission apparatus which concerns on one Example of this invention. It is a figure which shows the system configuration example by the transmission apparatus which concerns on one Example of this invention.

Explanation of symbols

1-1: transmission apparatus, 1-2: moving image encoding unit, 1-3: still image encoding unit, 1-6: moving image priority protocol processing unit

Claims (2)

A moving image encoding unit that encodes input image data as a moving image by using inter-frame coding and intra-frame encoding together, and a code different from the moving image encoding unit using the input image data as a still image A still image encoding unit that encodes by amount, a first buffer unit that stores encoded data from the moving image encoding unit, and at least one frame of encoded data from the still image encoding unit. A second buffer unit and a predetermined threshold value that determines at least the encoded data amount stored in the first buffer unit, and the encoded data amount stored in the first and second buffer units; A buffer control unit that monitors at a set interval ,
The predetermined threshold is set based on the payload length of the IP packet;
When the amount of encoded data in the first buffer unit exceeds the predetermined threshold, the buffer control unit reads the encoded data from the first buffer unit for transmission to a network, and If the amount of encoded data in the buffer unit does not exceed the predetermined threshold value, one frame of encoded data from the second buffer unit is sent to the network according to the buffering status of the second buffer unit. do Shi read out in order to transmit,
The encoded data read from the first and second buffer units is IP packetized and output to the network,
Further, regarding transmission of encoded data in the first buffer unit, when a decrease in transmission capacity that affects real-time characteristics is detected, a part of the encoded data following the I frame and the I frame is included. An image transmission apparatus, wherein only P frames are transmitted, and the remaining P frames are deleted from the first buffer unit to perform transmission of an I frame subsequent to the I frame . A moving image encoding step for encoding the input image data as a moving image by using inter-frame coding and intra-frame encoding together, and a code different from the moving image encoding unit using the input image data as a still image A still image encoding step for encoding by the amount, a step of storing encoded data from the moving image encoding unit in a first buffer unit, and at least one frame of encoded data from the still image encoding unit. Storing in the second buffer unit; and having first and second threshold values for respectively determining at least the encoded data amounts stored in the first and second buffer units, the encoded data amount accumulated in the second buffer unit, the buffer control step of monitoring at set intervals, the read encoded data from the first and second buffer portions, IP Comprising a step of outputting to the network by packet of a,
The buffer control step is a sub-step of reading the encoded data from the first buffer unit for transmission to a network when the encoded data amount of the first buffer unit exceeds the first threshold value; When the encoded data amount of the first buffer unit does not exceed the first threshold value and the encoded data amount of the second buffer unit exceeds the second threshold value, the second buffer unit one frame of encoded data from the buffer unit of and a sub-step of reading for transmission to the network,
Further, regarding transmission of encoded data in the first buffer unit, when a decrease in transmission capacity that affects real-time characteristics is detected, a part of the encoded data following the I frame and the I frame is included. transmitting only the P-frame, the image transmission method characterized by comprising the processing steps to perform the transmission of the next I frame of the I frame to remove the remaining P frames from the first buffer portion.

Images