JP4593209B2 - Image receiving apparatus, image receiving method, and image transmission system - Google Patents

Description

  The present invention relates to an image receiving apparatus, an image receiving method, and an image transmission system that receive image data via a network.

  Conventionally, in an image transmission system, when a video signal is encoded and the image data is transmitted as a packet to the network, a delay of the video signal may occur at the receiving terminal due to fluctuations in the arrival time of the image data in the network. When an operation such as panning or tilting is performed from the receiving terminal to the camera on the transmitting terminal side, the greater the delay of this video transmission, the later the operation result is reflected in the received video, and the operability decreases.

  A conventional image receiving apparatus used as a receiving terminal includes an image expansion unit that expands compressed and encoded image data, an instruction frame generation unit that generates an instruction frame necessary for operation, and an image that combines an image with the instruction frame It has a composition unit and an operation information analysis unit for analyzing operation information and the like (see, for example, Patent Document 1). In this image receiving apparatus, the operation information and the display delay of the image are analyzed, and the expected movement position is presented on the delayed screen, thereby suppressing the decrease in operability.

  As another conventional example, in order to avoid a decrease in operability due to a delay in video transmission, an image compression unit and a control change detection unit are provided in the image transmission device, and the encoding mode is changed to a low delay one. Thus, there is known a method for reducing a delay in encoding time during camera operation (see, for example, Patent Document 2).

JP 2002-165212 A JP 2000-295601 A

  However, in the conventional image receiving apparatus, it is expected that the accuracy of the operation is improved by presenting the predicted position of the camera operation, but the time until the image display that reflects the operation result still cannot be reduced. .

  Further, in the conventional video transmission delay reduction method, only the time required for encoding and decoding is shortened, and the delay time of the reception buffer cannot be reduced.

  The present invention has been made in view of the above circumstances. When there is no camera operation, a stable image can be displayed even when there is fluctuation in the arrival time of image data. When there is a camera operation, the operation is displayed on the display image. An object of the present invention is to provide an image receiving apparatus, an image receiving method, and an image transmission system that can reduce the reflected delay time.

An image receiving apparatus according to the present invention is an image receiving apparatus that receives image data captured by a camera via a network, and a receiving unit that receives image data transmitted from the camera via the network; Image data storage means for storing the received image data, operation means for operating the camera via the network, and buffer control for controlling the storage amount of the image data storage means based on camera operation by the operation means The buffer control means comprises threshold value storage means for storing a threshold value corresponding to the operation content of the camera operation by the operation means, and when the camera operation is performed by the operation means, the image The accumulation amount of the data accumulation means is set as a threshold value corresponding to a predetermined number of frames of image data, It is to set to a value corresponding to the.

Thus, when outputting image data to a display device or the like, if there is no camera operation, a stable image is displayed even if the arrival time of the image data fluctuates, and if there is a camera operation, the operation is displayed as a display image. It is possible to reduce the delay time reflected in. Thus, by controlling the accumulation amount of the image data accumulation means based on the camera operation, it is possible to set the delay time for image display according to the state of the camera operation. For example, when there is no camera operation, it is possible to set a large storage amount of the image data storage means and perform stable display of an image in which the influence of fluctuations in the arrival time of the image data is suppressed. Further, when there is a camera operation, it is possible to set a small storage amount of the image data storage means and reduce a time until the operation is reflected on the screen to suppress an operation error. Furthermore, the responsiveness of the image data suitable for the camera operation content such as camera movement can be ensured, and the operability can be improved .

Further, as one aspect of the present invention, in the image receiving apparatus described above, the camera operation by the operation unit has a captured image position changing operation including any one of pan, tilt, and zoom. .
This makes it possible to improve the operability of camera operation while controlling the amount of image data storage means according to the camera operation content such as pan, tilt, zoom, etc., and suppressing the influence of fluctuations in image transmission. is there.

According to another aspect of the present invention, the image receiving apparatus includes an image expansion unit that expands image data stored in the image data storage unit, and the image data storage unit The image data is stored.
As a result, the storage amount of the image data before decompression can be controlled, and time-consuming useless image decompression processing can be omitted.

An image receiving method according to the present invention is an image receiving method for receiving image data captured by a camera via a network, and when storing the received image data, the operation of the camera is performed via the network. In this case, the image data storage amount is set to a value corresponding to the operation content of the camera as a threshold value corresponding to a predetermined number of frames of the image data.
Accordingly, it is possible to improve the operability of the camera operation while controlling the accumulation amount of the image data accumulating means according to the camera operation and suppressing the influence of the fluctuation of the image transmission.

An image transmission system according to the present invention is an image transmission system in which a camera and an image receiving apparatus are connected via a network, and image data captured by the camera is received by the image receiving apparatus via the network. The image receiving apparatus receives image data transmitted from the camera via the network, image data accumulation means for accumulating the received image data, and operates the camera via the network. An operation unit; and a buffer control unit that controls a storage amount of the image data storage unit based on a camera operation by the operation unit, wherein the camera changes a shooting operation according to an operation from the image receiving device. comprising an operation changing means, and transmission means for transmitting the captured image data, the buffer control means Threshold storage means for storing a threshold value corresponding to the operation content of the camera operation by the operation means, and when there is a camera operation by the operation means, the storage amount of the image data storage means as a threshold corresponding to a predetermined number of frames, is shall be set to a value corresponding to the operation content of the camera operation.
Accordingly, it is possible to improve the operability of the camera operation while controlling the accumulation amount of the image data accumulating means according to the camera operation and suppressing the influence of the fluctuation of the image transmission.

An image receiving program according to the present invention includes a reception step of receiving image data transmitted from a camera via a network in a computer, a storage step of storing the received image data in an image data storage unit, An operation step for operating the camera via a network, a buffer control step for controlling an accumulation amount of the image data accumulation means for accumulating the received image data based on a camera operation by the operation step, and a camera by the operation step Setting the amount of image data stored when an operation is performed as a threshold value corresponding to a predetermined number of frames of the image data to a value corresponding to the operation content of the camera operation. Is .

  According to the present invention, when there is no camera operation, a stable image can be displayed even if the arrival time of the image data fluctuates, and when there is a camera operation, the delay time in which the operation is reflected in the display image is reduced. An image receiving apparatus, an image receiving method, and an image transmission system can be provided.

  In this embodiment, the present invention is applied to an image transmission system that transmits image data captured by a remotely operated camera, such as a video surveillance system that receives and displays an image while operating a remote camera via a network. The example of a structure of the image receiving apparatus used is shown.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration of an image transmission system according to the first embodiment. This image transmission system has a configuration in which an image reception device 1 and an image transmission device 3 are connected to a network 2 that performs data transmission.

  The image transmission device (camera) 3 was photographed by an imaging unit (camera body) 31, a shooting position control mechanism (head) 32 (corresponding to an example of a shooting operation changing unit), and an imaging unit of the camera body 31. A transmission unit 33 (which corresponds to an example of a transmission unit) that transmits image data is provided. The image transmission device 3 performs a shooting operation including a camera shooting range changing operation (ie, a shot image position changing operation) such as pan, tilt, and zoom in accordance with an operation from the image receiving device 1 notified via the network 2. The image data of the captured image is transmitted to the image receiving apparatus 1 via the network 2.

  The image receiving apparatus 1 is connected to a display unit 101 for displaying an image, an operation unit 102 (which corresponds to an example of an operation unit) for operating camera movement such as panning and tilting, and a network 2 to control a pan head control message, A network interface (I / F) unit 103 (corresponding to an example of a receiving unit) that transmits and receives image data as a packet, and a buffer memory 104a (corresponding to an example of an image data storing unit) that temporarily stores received image data are provided. Then, a buffer control unit 104 (corresponding to an example of a buffer control unit) that controls the amount of image data stored in the buffer memory 104a, and an image expansion unit 105 (image expansion) that expands the compressed / encoded image data Corresponding to an example of the means). In the present embodiment, an example in which the buffer memory is built in the buffer control unit is shown, but the buffer memory may be provided independently as an external memory.

  The buffer control unit 104 includes a known processor, a memory such as a ROM and a RAM in which a control program described later is stored. The operation unit 102 includes means for measuring a mechanical position operated by a user, such as a joystick, a mouse, and a keyboard, as a user interface. As this measuring means, for example, it is possible to detect the amount of movement of the stick, the pressing or movement of the button, or the pressing or movement of a virtual operation button or the like displayed on a Web browser or a monitor dedicated to control. May be interpreted and output.

  The operation of the image transmission system having the above configuration will be described. First, when an image is transmitted from the image transmission device 3 to the network 2, the image transmission device 3 compresses and encodes the captured image, and transmits the generated image data to the network 2 as a packet. The network IF unit 103 receives a packet including image data transmitted from the image transmission device 3 and transfers the packet to the buffer control unit 104. The buffer control unit 104 temporarily stores the image data in the internal buffer memory 104a, and does not output the image data to the image decompression unit 105 until the storage amount reaches a predetermined threshold value (specified amount). When the storage amount of the image data reaches a specified amount, the buffer control unit 104 outputs the image data to the image expansion unit 105. The image decompression unit 105 decompresses the compressed and encoded image data and sends the decompressed image data to the display unit 101. The display unit 101 displays the image data sent from the image expansion unit 105.

  The operation unit 102 detects a camera operation in response to an instruction from the user, and transmits the camera operation as a control packet to the image transmission device 3 via the network IF unit 103. The operation unit 102 notifies the buffer control unit 104 of the camera operation information. Based on the operation information notified from the operation unit 102, the buffer control unit 104 changes the threshold value (specified amount) of the buffer memory 104a. That is, the image data accumulation amount is set to a threshold value suitable for the camera operation state.

  FIG. 2 is a diagram for explaining buffer control in the camera operation of the first embodiment. Normally, when a packet including image data transmitted from the image transmission device 3 is received by the image reception device 1 via the network 2, fluctuation occurs in the packet reception interval. Since the image display is performed at regular intervals, if the fluctuation of the reception interval is large, the image display may be disturbed such as frame dropping. In order to suppress such disturbance of image display, a certain amount of image data is accumulated in the buffer memory 104a, and the image data is supplied from the buffer memory 104a to the image expansion unit 105 and the display unit 101 at regular intervals. This suppresses frame dropping and image display disturbance when fluctuations occur in image transmission.

  On the other hand, in the accumulation of image data, there is a difference between the imaging time and the actual display time, and the difference increases in proportion to the accumulation amount. When the camera is remotely operated while referring to the display image, there is a possibility that an error occurs in the operation of moving the camera to a desired position due to a time difference. For this reason, in the present embodiment, the buffer control unit 104 changes the accumulation amount of the image data according to the state of the camera operation. Specifically, when detecting “movement” of the operation of the camera, the buffer control unit 104 reduces the temporary accumulation amount of the image data so as to become a predetermined accumulation amount (threshold value) when the camera moves. The old data is deleted, and the next deleted data is transferred to the image decompression unit 105.

  FIG. 2 shows a case where image data is supplied from the buffer memory 104a to the image expansion unit 105. Here, one square cell represents an image encoding unit (one frame), and is input to the image receiving apparatus 1 in numerical order. In the case of no camera operation, the amount stored in the buffer memory 104a is set to a threshold value A (second threshold value), and after reaching the threshold value A, image data is supplied to the image decompression unit 105 by a first-in first-out method. To do. On the other hand, when the camera operation state is entered, the accumulation amount in the buffer memory 104a is changed to the threshold value B (first threshold value). At the time of changing to the threshold value B, all the surplus image data already accumulated is discarded from the oldest one or supplied to the image decompression unit 105 by dropping frames. In this embodiment, the threshold A is an accumulation amount corresponding to 8 frames, and the threshold B is an accumulation amount corresponding to 2 frames, but each accumulation amount is particularly limited. Of course, it may be set arbitrarily.

  As described above, the camera operation state includes, for example, a state in which operations such as panning, tilting, and zooming of the camera are accompanied by movements of the screen (a state in which the camera is operated) and operations involving movements of the screen. There are two states, that is, no state (state without camera operation). When there is no camera operation, the amount of accumulation (threshold value) that can suppress the disturbance of the screen display due to network fluctuations, for example, more than the average value of fluctuations in packet arrival interval, the accumulation amount by statistical reference amount Secure. On the other hand, in the state with camera operation, in order to suppress the delay time reflected in the screen display from the start of the operation, the accumulation amount is reduced compared to the state without camera operation.

  For example, the threshold value (accumulation amount) may be set to 0 and the received image data may be sent immediately to the image decompression unit 105. In this case, the difference between the imaging time and the display time can be automatically reduced, the response of the image display to the operation can be shortened, and an operation error such as misalignment or overshoot when the operation is stopped is suppressed. On the other hand, when the operation is stopped and the state is shifted to a state where there is no camera operation, the buffer control unit 104 restores the accumulated amount of the buffer memory 104a. Note that the state without camera operation includes a preset operation state in which a predetermined camera operation is executed, in addition to a state in which the camera is fixed without being operated by the user.

  FIG. 3 is a flowchart showing a buffer control processing procedure in the first embodiment. This processing program is stored in the ROM in the buffer control unit 104, and is also executed at predetermined time intervals by the CPU in the buffer control unit 104. First, it is determined whether or not the camera operation state notified from the operation unit 102 is a camera operation state (step S1). If the camera is not operated, the amount stored in the buffer memory 104a is set to the threshold value A (step S2), and this process is terminated. On the other hand, if the camera is being operated in step S1, the amount stored in the buffer memory 104a is set to the threshold value B (step S3), and this process ends.

  As described above, according to the image receiving apparatus of the first embodiment, the buffer control unit changes the accumulation amount of received image data according to the operation state of the camera. Stable display of images that suppress the effect of fluctuations in arrival time can be performed. On the other hand, when there is a camera operation, the time until the operation is reflected on the screen is automatically reduced to suppress operation errors. Can do.

(Second Embodiment)
In the first embodiment described above, the threshold value is set in two stages depending on whether or not the camera is operated. However, in the second embodiment, in addition to the presence or absence of the camera operation, depending on the operation content of the camera operation, The case where the threshold value of the buffer memory is changed is shown. The configuration of the image transmission system is the same as that of the first embodiment shown in FIG. In the second embodiment, the threshold value table is stored in the ROM in the buffer control unit 104.

  FIG. 4 is a diagram illustrating a threshold value table stored in the ROM in the buffer control unit 104 according to the second embodiment. This threshold value table can be set to threshold value C when the camera operation is pan, set to threshold value D when tilted, and set to threshold value E when zoomed. It is shown. As an example of the threshold values C, D, and E, for example, the accumulation amount corresponding to 1, 2, and 3 frames may be set.

  FIG. 5 is a flowchart showing a buffer control processing procedure in the second embodiment. The same step processes as those in the first embodiment shown in FIG. This processing program is stored in the ROM in the buffer control unit 104, and is also executed at predetermined time intervals by the CPU in the buffer control unit 104. First, it is determined whether or not the camera operation state notified from the operation unit 102 is a camera operation state (step S1). If the camera is not operated, the amount stored in the buffer memory 104a is set to the threshold value A (step S2), and this process is terminated. On the other hand, when the camera operation is present in step S1, the operation content notified from the operation unit 102 is determined in order to change the threshold value according to the camera operation content (step S3A). Then, referring to the threshold value change table, the threshold value of the buffer memory 104a corresponding to the notified operation content is selected and set (step S3B). Then, this process is complete | finished.

  As described above, according to the image receiving apparatus of the second embodiment, it is possible to set the optimum storage amount of the buffer memory to the threshold value suitable for the camera operation content such as pan, tilt, and zoom. Thereby, it is possible to appropriately suppress the influence of fluctuations in image transmission according to the contents of camera operation.

  As described above, in the present embodiment, an image in which the influence of fluctuations in the arrival time of image data is suppressed when there is no camera operation is provided by providing a buffer control unit that changes the accumulation amount of image data received depending on the operation state of the camera. On the other hand, when the camera is operated, it is possible to automatically reduce the time until the operation is reflected on the screen, thereby suppressing an operation error.

  The above is the description of the present embodiment, but the present invention is not limited to the configuration of these embodiments, and the configuration that can achieve the functions shown in the claims or the functions of the configuration of the embodiments. Any other means such as other alternative means can be applied.

  According to the present invention, when there is no camera operation, a stable image can be displayed even if there is fluctuation in the arrival time of the image data, and when there is a camera operation, the delay time that the operation is reflected in the display image can be reduced. The image receiving apparatus and the image receiving method that receive the image data via the network, the image transmission system, and the like are effective.

The figure which shows the structure of the image transmission system in the 1st Embodiment of this invention. The figure explaining buffer control in camera operation of a 1st embodiment The flowchart which shows the buffer control processing procedure in 1st Embodiment. The figure which shows the threshold value table stored in the buffer control part in 2nd Embodiment. The flowchart which shows the buffer control processing procedure in 2nd Embodiment

Explanation of symbols

1 Image receiving device 2 Network 3 Image transmitting device (camera)
31 Imaging unit (camera body)
32 Shooting position control mechanism (head)
33 Transmission Unit 101 Display Unit 102 Operation Unit 103 Network Interface (I / F) Unit 104 Buffer Control Unit 104a Buffer Memory 105 Image Expansion Unit

Claims (6)

An image receiving device that receives image data captured by a camera via a network,
Receiving means for receiving image data transmitted from the camera via the network;
Image data storage means for storing the received image data;
Operating means for operating the camera via the network;
Buffer control means for controlling the storage amount of the image data storage means based on the camera operation by the operation means,
The buffer control means includes a threshold value storage means for storing a threshold value corresponding to the operation content of the camera operation by the operation means, and when there is a camera operation by the operation means, the image data storage means stores An image receiving apparatus that sets the amount as a threshold corresponding to a predetermined number of frames of image data to a value corresponding to the operation content of the camera operation. The image receiving device according to claim 1,
An image receiving apparatus having a photographed image position changing operation in which the camera operation by the operation means includes any one of pan, tilt, and zoom. The image receiving device according to claim 1,
Image expansion means for expanding the image data stored in the image data storage means;
The image data storage means stores the image data before being decompressed. An image receiving method for receiving image data captured by a camera via a network,
When accumulating received image data, the accumulation amount of the image data when the camera is operated via the network is set as a threshold corresponding to a predetermined number of frames of the image data, An image receiving method that is set to a value corresponding to the operation content of the camera. An image transmission system in which a camera and an image receiving device are connected via a network, and image data captured by the camera is received by the image receiving device via the network,
The image receiving apparatus receives image data transmitted from the camera via the network, image data accumulation means for accumulating the received image data, and operates the camera via the network. An operation means, and a buffer control means for controlling an accumulation amount of the image data accumulation means based on a camera operation by the operation means,
The camera includes a photographing operation changing unit that changes a photographing operation according to an operation from the image receiving device, and a transmission unit that transmits photographed image data.
The buffer control means includes a threshold value storage means for storing a threshold value corresponding to an operation content of the camera operation by the operation means, and when the camera operation is performed by the operation means, the image data storage means stores An image transmission system in which the amount is set to a value corresponding to the operation content of the camera operation as a threshold corresponding to a predetermined number of frames of image data. On the computer,
A receiving step of receiving image data transmitted from the camera via the network;
A storage step of storing the received image data in an image data storage means;
An operation step of operating the camera via the network;
A buffer control step for controlling an accumulation amount of image data accumulation means for accumulating the received image data based on a camera operation in the operation step;
A step of setting the accumulated amount of the image data when the camera operation is performed in the operation step as a threshold corresponding to a predetermined number of frames of the image data to a value corresponding to the operation content of the camera operation And an image receiving program for executing the program.

Images