JP3357559B2 - Image data receiving device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image data receiving apparatus, and more particularly to an image data receiving apparatus capable of remotely controlling a photographing apparatus such as a video camera.

[0002]

2. Description of the Related Art Image data captured by a video camera located at a remote location is received and displayed on a display device, and the image capturing range of the video camera or the like can be remotely controlled on the receiving side. A receiving device is used in various situations. For example, a technique described in Japanese Patent Application Laid-Open No. 6-121229 is disclosed as an image data receiving apparatus for freely selecting a region desired by a user from a display screen and enlarging the region. Has been proposed. This image data receiving device specifies an arbitrary area in the image displayed on the image display unit by the display screen control unit, enlarges the image information of the specified area to the display area of the image display unit, and displays the image. The output is enlarged to a part. Further, Japanese Patent Application Laid-Open No. Hei 6-205407 proposes a technique for operating a selection of a received image and a setting of an area on an image receiving side in a videophone. In this technique, the receiving side can switch between the camera input unit and the document camera input unit on the transmitting side, or control the swing function and zoom function thereof. In this case, control information for controlling the camera input unit and the like is input from the partner image input operation unit on the receiving side. Upon receiving this control information, the transmitting side controls the camera input unit and the document camera input unit on the transmitting side based on the received control information.

[0003]

However, in the conventional image data receiving apparatus, in any case, the user cannot confirm the image other than the current image currently being picked up by the transmitting camera. That is, the embodiment around the current image could not be confirmed. For this reason, even if the user attempts to display an image in another area by remotely operating the camera or the like on the transmission side, it is difficult to immediately specify the area that the user really wants to check. For this reason, when remote control is performed by the conventional image data receiving apparatus, it is necessary to analogize the surroundings with reference to the current display screen and remotely control the screen to move up, down, left and right, and to obtain a desired image. It took time to get. In addition, in the conventional image data receiving apparatus, since the user cannot recognize the correspondence between the range that can be photographed by the transmitting camera and the current display screen, even if the camera moves between the movement limit positions, Since it cannot be recognized, there is a possibility that the user needs to move to a position beyond the limit position. In this case, since the displayed image on the screen does not change despite the remote operation, it is only possible to indirectly recognize that the user has reached the limit position, meaning that an unnecessary operation was performed during this time. In the case of a television broadcast or a wired videophone as in the above-mentioned publication technology, even if an image in a desired range is searched while appropriately shifting the shooting range by remote control, the communication speed of the image is fast and the time lag is relatively small. Therefore, the desired image can be found relatively quickly. However, if the time lag is relatively large, such as when transmitting and receiving compressed image data by a wireless telephone such as a mobile phone, even if the remote control is instructed to shift the shooting range to the right, the screen on the right side is actually displayed. You need to wait for some time before it appears. If there is no target image at a position where the photographing range is shifted, useless video is displayed. In order to obtain a desired image, it is necessary to shift the photographing range again, which requires a long time as a whole. Such a problem is conspicuous when the communication speed is low. However, even if the communication speed is high, useless video may be displayed in order to immediately acquire the target video, which takes time. Then it was a common problem.

In order to solve such a problem, a plurality of images captured at different positions are sequentially transmitted by controlling an imaging position of a camera on a transmitting side, and a plurality of images received by an image data receiving apparatus are combined. There is a method of displaying. However, when transmitting a plurality of images by controlling the position of the camera, the captured images may overlap with each other due to the relationship between the angle of view of the camera and the control range of the position control mechanism of the camera. In particular, the above problem occurs when the transmitting side camera and its position control mechanism (drive device) are not unified, or when there are many types of transmitting side cameras capable of capturing images in the position control mechanism. As described above, when transmitting a plurality of images to be combined from the transmitting side, transmitting a duplicate image has a problem in that the transmission and reception time of the image becomes longer. In particular, when image data is compressed by a predetermined method and transmitted, redundant processing is performed in compression on the transmission side and decompression on the reception side, and there is a problem that the processing time becomes longer. . Further, on the receiving side, it is necessary to perform synthesis without the overlapped image portion, and there is a problem that time is required for the synthesis.

Accordingly, the present invention has been made to solve such problems of the prior art, and has as its object to provide an image data receiving apparatus capable of avoiding the transmission and reception of duplicate captured images. .

[0006]

According to the first aspect of the present invention, there is provided a control mechanism for controlling the position of the image pickup device in accordance with the received data of the image pickup position. An image pickup device that transmits a captured image and transmits an image of a designated region in the captured image when region designation data is received, and receives image data from an image pickup device and displays the image on an image display device. A device,
When the control mechanism of the imaging device sequentially captures an entire range that can be imaged by the imaging element, an overlap determination unit that determines whether or not there is an area that overlaps with another image, and an overlap determination unit. When it is determined that there is an overlapping area, a calculation unit that calculates the position of the imaging device and an area excluding the overlapping range so that the images do not overlap, and the position of the imaging device calculated by the calculation unit is an imaging position. Transmitting means for transmitting the area excluding the overlapping range to the image capturing apparatus as area designation data, and sequentially receiving each image transmitted from the image capturing apparatus corresponding to each data transmitted by the transmitting means. An image data receiving apparatus comprising: an image receiving means for performing image processing; and an image synthesizing means for synthesizing each image received by the image receiving means into one image and displaying the image on an image display device. To achieve the purpose. According to a second aspect of the present invention, in the image data receiving apparatus according to the first aspect, when the control mechanism of the imaging device can change an imaging angle of view of an imaging device at the same imaging position, And an angle-of-view transmitting unit that transmits data of the image-taking angle of view, wherein the overlap determining unit transmits the entire range that can be imaged by the image-capturing element by the control mechanism of the image-capturing device, by the angle-of-view transmitting unit It is determined whether or not there is an image that overlaps with another image when images are sequentially captured at the imaging angle of view.

[0007]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of an image data receiving apparatus according to the present invention; FIG. (1) Overview of Embodiment In this embodiment, the image capturing position of a video camera is sequentially changed by remote operation, and images captured at different positions are combined, so that an image of the entire photographable range is set as a background image. Combine and display on the window. When requesting a change in the imaging position, the image data receiving apparatus specifies a position that avoids overlap with another image, but if the overlap cannot be avoided (for example, the entire imageable range is not an integral multiple of the imaging field angle). In the case), the area excluding the imaging position and the overlapping range is transmitted to the transmitting station as area designation data. The transmitting station compresses and transmits the image only for the designated area, thereby avoiding unnecessary data compression and data transmission. By looking at the background image synthesized and displayed on the window, the user can know the range that can be confirmed in detail as the main image. And
Regarding the display of the main image, a desired position and range can be easily specified. When the position and range specified by the image data receiving device are transmitted, the transmitting station changes the position and zoom of the video camera, captures image data for the specified range, and transmits the image data.
At this time, the image data receiving apparatus sets the current designated range and the previously designated range (the range currently displayed on the screen) to the new camera frame and the last designated range until the image corresponding to the current designated range is received. It is displayed as a camera frame in the background image. Thus, the user can immediately recognize whether the video currently displayed on the screen is the image switched to the current designated range or the image before the switching. Further, in the present embodiment, the background image at the corresponding position is updated using the bitmap data of the main image newly captured in the range of the new camera frame. As a result, it is possible to provide the user with a partial but up-to-date background image.

(2) Details of Embodiment FIG. 1 shows a system configuration of an image data communication system using the image data receiving apparatus of the embodiment. As shown in this figure, an image data communication system includes a transmitting station 1 for capturing and transmitting an image, and a receiving side for remotely controlling the capturing of an image by the transmitting station 1 and receiving and displaying image data. And a station 2. This image data communication system, for example, monitoring and reporting the progress of building sites, etc.,
It can be widely used as remote monitoring of places where humans cannot always be present, investigation of volcanoes and dams, transmission of news images taken in remote places, production tools for television programs and events, and the like.

The transmitting station 1 mainly includes a video camera 4, a control unit 5, an image transmitting device 6, a mobile phone 7, and a modem 8. The video camera 4 can be connected to and disconnected from the image transmission device 6 with an NTSC cable. The video camera 4 constantly captures images and converts image data based on NTSC signals into, for example, 3 to 5
The data is supplied to the image transmission device 6 periodically at intervals of seconds. The control unit 5 can be connected to and disconnected from the image transmission device 6 by, for example, an RS232C cable. The control unit 5 controls the imaging angle of the video camera (upward / downward angle = pan, left / right direction angle = in accordance with the camera position change command supplied from the receiving station 2 via the image transmission device 6).
(Tilt), an image pickup range (image pickup angle of view) by zoom, focus, and the like.

Although not shown, the image transmission device 6 includes a CPU for performing various controls related to transmission and reception of image data,
R storing programs for various controls by PU
OM, a memory for storing image data and the like (flash memory, S-RAM, etc.), a conversion unit for converting an NTSC signal to an RGB signal, a compression unit for compressing image data, and a power supply cable and the like. It has various connection terminals and other devices. The image transmission device 6 receives the camera position change command from the receiving station 2 and supplies it to the control unit 5, and converts the image data supplied from the video camera 4 from an NTSC signal to an RGB signal by a conversion unit. Then, the compression section compresses the image data in the capture range specified by the receiving station 2 by a predetermined method to obtain compressed image data. Image transmission device 6
Is configured to add the camera position information (pan, tilt, zoom values and image data capture range) to the header of the compressed image data, and transmit the compressed image data to the receiving station 2.

The transmission of the compressed image data is performed through the side of the cellular phone 7 and the modem 8 to which the communication line between the transmitting station 2 is connected. For compression of the image data in the image transmission device 6, various existing compression methods can be adopted. For example, Japanese Patent Application Laid-Open No. 5-227
The data is compressed by the FST compression method described in Japanese Patent No. 547. FST compression encodes an A / D converted video signal into a matrix block consisting of a plurality of pixels and a bitmap representing one of two types of reference values, and finds non-redundant information in the coded block. Identifying and coding and comparing each block to the corresponding block of the immediately preceding frame to eliminate inter-block and inter-frame redundancy and to further reduce the current color value relative to the immediately preceding color value. The amount of information is compressed by coding in the form of Other image compression methods include MPEG, JPEG, and JP-A-8-154.
Various methods such as TIM compression described in Japanese Patent No. 261 may be used.

TIM compression is a method of image compression in which decompression processing of compressed image data can be easily performed on the image data receiving device side. In TIM compression, (a) human eyes are sensitive to luminance changes but insensitive to color changes in image compression. (B) When the pixels constituting an image are appropriately blocked, The image compression is performed based on the assumption that there are usually only two colors in one block which is generally established in the natural image. That is, each pixel data is divided into image data composed of a luminance signal Y and color difference signals U and V, and block data composed of n × m (usually 4 × 4) is generated. Then, two colors of a first color component (y1, u1, v1) and a second color component (y2, u2, v2) having a luminance signal and a color difference signal, which represent the generated block, and a first color component and a second color component are represented. Data is compressed by generating a block component consisting of an n × m bitmap indicating the distribution of two color components. Then, the generated block component is coded from the block component, a similar block having the same or most similar block component, and the code data indicating the similar block and the difference data with respect to the block component of the similar block. By doing so, the data is further compressed and transmitted to the image data receiving apparatus.

The image data compression employed in the present embodiment is processed by an image compression board (printed board) arranged in the image transmission device 6.
Compression may be performed by software processing such as JPEG. The image transmission device 6 is provided with a function of encrypting compressed image data using a predetermined encryption key in order to maintain the secrecy of the communication contents, and the encrypted compressed image data is transmitted to the receiving station 2. You may make it transmit.

On the other hand, the receiving station 2 is mainly composed of a mobile phone 9 and a personal computer 10 to which the mobile phone 9 is connected / disconnected (hereinafter simply referred to as a personal computer 10). Mobile phone 9 and mobile phone 7
Is mainly 9600b via digital mobile communication network
The connection is made by high-speed communication of ps or 19200 bps, but another communication method may be used.
For example, ISDN (Integrated Services Digital Network
ork), ordinary telephone line network, Internet, FAX
Communication via various data communication networks such as a communication network, PHS, wireless satellite communication, and optical communication may be performed. Various types of telephones can also be used for the mobile phones 7 and 9 on the transmitting and receiving sides, and different types of telephones may be used. Further, the communication speed between the transmitting and receiving devices 1 and 2 may be automatically set according to the communication capabilities of the mobile phones 7 and 9 and the modem 8.

FIG. 2 shows the configuration of the personal computer 10. As shown in FIG. 2, the personal computer 10 includes a control unit 11 for controlling the entire apparatus. The control unit 11 includes a keyboard 13, a mouse 14, a display device 15, a storage device 16, a storage medium drive device 17, a communication control device 18,
Also, an input / output IF 19 is connected.

The control unit 11 includes a CPU 111 functioning as a so-called computer, a ROM 112, a RAM 11
3 is provided. The ROM 112 is a read-only memory in which various data and programs are stored in advance.
The RAM 113 is a random access memory used as a working memory by the CPU 111. This RA
When the personal computer 10 is used as the image data receiving device of the present embodiment, a background image file,
An area for storing data of a main image file, a camera position change command, and a camera type is secured.

The keyboard 13 has various keys such as a kana key and a numeric keypad for inputting a kana character, a function key for executing various functions, a cursor key, and the like. As the display device 15, for example, a CRT or a liquid crystal display is used. The display device 15 includes a main image window for displaying an image captured by the video camera 4,
A camera control window or the like for remotely controlling the imaging position of the video camera 4, the zoom value, and the like is displayed. The camera control window displays camera control keys and a background image. The main image window and the camera control window can be simultaneously displayed, or only one of them can be displayed. In the case of simultaneous display, both windows can be displayed side by side or overlaid. The mouse 14 is a pointing device, and is an input device for designating a corresponding function by left-clicking a key, an icon, or the like displayed on the display device 15. In the present embodiment, the mouse 14 is used to instruct the capture and synthesis of the background image, to specify the imaging position of the video camera 4 by dragging the camera frame displayed in the background image, and to set the size of the camera frame. Or change.

The storage device 16 is composed of a readable and writable recording medium and a drive device for reading and writing various information such as programs and data on the storage medium.
As the storage medium used for the storage device 16, a hard disk is mainly used, but a readable / writable storage medium among the storage media used in the storage medium driving device 17 described later may be used. . The storage device 16 stores the types of the video cameras 4 that can be connected to the image transmission device 6 and information (zoom, pan, tilt range, and the like) on items that can be controlled by each video camera. Also, a dial number for connecting to the transmitting station 1 and a background image once captured and synthesized are stored for each transmitting station. The program storage area of the storage device 16 stores various application programs such as an image data reception-related program according to the present embodiment, in addition to a basic program such as an OS for operating the personal computer 10. In the receiving station 2 of the present embodiment, a program for receiving the compressed image data from the transmitting station 1 and expanding the compressed image data by a method corresponding to the image compression method by the transmitting station 1 is stored in the storage device 16. Is stored in For example, the image transmission device 6 of the transmitting station 1
When performing image compression by G, the storage device 16 stores the MPE
A program for performing expansion by G is stored. As described above, in the image data receiving apparatus according to the present embodiment, it is not necessary to set special hardware for image expansion in the receiving station 2. When the compressed image data transmitted from the transmitting station 1 is encrypted, a program for performing decryption using the corresponding key is also stored in the storage device 16.

The recording medium driving device 17 is a driving device for the CPU 111 to read a computer program from an external recording medium. The computer program recorded on the recording medium includes the personal computer 1 of the present embodiment.
0, an application program related to image data reception, and dictionaries and data used therein. Here, the recording medium refers to a recording medium on which a computer program is recorded, and specifically,
Magnetic storage media such as floppy disks, hard disks, and magnetic tapes; semiconductor storage media such as memory chips and IC cards; storage media such as CD-ROMs, MOs, and PDs (phase change rewritable optical disks); A computer program is recorded by a recording medium using paper (and a medium having a function equivalent to paper) such as a printed matter for reading a program using a paper card, a paper tape, and a character recognition device, and other various methods. A recording medium is included. As a recording medium used in the personal computer 10 of the present embodiment, a CD-ROM or a floppy disk is mainly used. The recording medium driving device 17 can read data from the RAM 113 and the storage device 16 in the case of a writable recording medium such as a floppy disk, in addition to reading a computer program from these various recording media. .

In the personal computer 10, the CPU 111 reads a computer program from an external recording medium set in the recording medium driving device 17 and stores (installs) it in the storage device 16. Then, when the image data reception related program according to the present embodiment is executed, the corresponding program is read from the storage device 16 into the RAM 113 and executed. However, it is also possible to execute reading and reading directly from the external recording medium to the RAM 113 by the recording medium driving device 17 instead of the storage device 16. Also, depending on the personal computer 10, the program related to the image data reception of the present embodiment may be stored in the ROM 112 in advance.
And the CPU 111 may execute this.

The communication control device 18 has a connection terminal for the mobile phone 9, and transmits a camera position change command for remotely operating the video camera 4 and receives compressed image data from the transmitting station 1. It is supposed to do. Further, the communication control device 18 can transmit and receive various data such as text data and bitmap data to and from another personal computer or the like. The input / output IF 19 is an interface for connecting / disconnecting a printing device for performing various printings such as images and characters displayed on the display device 15 and a speaker for outputting voices and music. As the printing device, various printing devices such as a laser printer, a dot printer, an ink jet printer, a page printer, a thermal printer, and a thermal transfer printer are used.

Next, the operation of transmitting and receiving image data by the image data communication system configured as described above will be described. FIG. 3 shows a display screen of the display device 15 of the present embodiment. As shown in FIG.
On the display screen 5, a main image window 151 in which an image captured by the video camera 4 is displayed, and a camera control window 152 for remotely controlling an imaging position, a zoom value, and the like of the video camera 4 are displayed. On the left side of the drawing of the camera control window 152, “Full screen capture” key 153 and “Detailed setting” key 1
54, a “cancel setting” key 155, and a “send setting” key 156 are arranged. Also, in the camera control window 152, a background image screen 157 in which the entire range that can be imaged by synthesizing images captured at each position of the video camera 4 is displayed as a background image is arranged. In the background image screen 157, a camera frame 158 that displays the image capturing position and range of the video camera 4 is displayed. Background image screen 157
Cannot be imaged at once by the video camera 4, so the background image is imaged for each divided screen including a plurality of small areas, and each divided screen is synthesized and displayed.

A “full screen capture” key 153 of the camera control window 152 is a key for instructing capture of a background image displayed on the background image screen 157. A “detailed setting” key 154 is a key for setting a luminance level, a hue level, manual operation of focus, and other detailed contents of the video camera 4. When this key is left-clicked, the camera control window 152 is set. Is extended to the lower side, and each setting item is displayed. The “cancel setting” key 155 is a key for canceling the change operation when left-clicked after changing the position or range of the camera frame 158 in the background image screen 157. Canceling the change operation using this key is effective only before the “setting transmission screen” key is left-clicked.
The “setting transmission” key 156 is a key for requesting the image transmission device 6 to change the imaging position and range of the video camera 4.

FIG. 4 explains each divided screen when a background image is captured. As shown in FIG. 4, the background image screen 157 is divided into divided screens 157a to 157l, and images captured for each divided screen are combined and sequentially displayed on the background image screen 157. At this time, the exposure and the hue of each screen may be adjusted to make it easier to see. The movable angle of the pan of the video camera 4 is θp,
The tilt movable angle is θt, the position of the video camera 4 at the upper leftmost position is the origin (θp = θt = 0 °), the horizontal angle of view of the video camera 4 when the zoom value is the widest angle is αp, and the vertical angle is αp. The angle of view is αt, the number in the horizontal direction of the divided screen is x from the left, and the number in the vertical direction is y from the upper side (divided screen 1
If 57i is x = 3, y = 2), the control unit 11 calculates the coordinates of pan and tilt for each divided screen according to the following equations (1) and (2). Pan coordinates = (y−1) αp (1) Tilt coordinates = (x−1) αt (2) The control unit 11 transmits the calculated data of the pan and tilt coordinates of the divided screen to the video camera 4 sequentially. Is transmitted to the image transmission device 6 together with the position change command, the video camera 4 is remotely operated, and the image at the corresponding division position is received and synthesized as a background image. The number of divisions of the background image in the horizontal direction is (θp / αp) +1, and the number of divisions in the vertical direction is (θt /
αt) +1. The image transmission device 6 receives the bread,
When the tilt coordinates are supplied to the control unit 5, the control unit 5 moves the video camera 4 to the corresponding coordinate position.

As described above, the personal computer 10 specifies the position of each divided screen so as not to overlap with the other divided screens. For example, when the entire range in which the image can be captured is not an integral multiple of the imaging field angle, In addition, overlapping of captured images is inevitable. Further, in order to capture a wide background image, the zoom value is set to the widest angle, but the captured images do not always overlap at that time. FIG. 5 illustrates the overlap of captured images that occurs due to the relationship between the entire range that can be captured and the captured angle of view of the divided screen. As shown in FIG. 5, for example, the total angle of view that can be captured by the video camera 4 under the position control of the control unit 5 is 2220 pixels in the horizontal direction and 480 pixels in the vertical direction (FIG. 5A). It is assumed that the angle of view captured by the video camera 4 at one time corresponding to the screen is 640 pixels in the horizontal direction and 240 pixels in the vertical direction at the maximum angle of view (FIG. 9B). In such a case, in the horizontal direction, since the entire viewable angle of view (2220) is not an integral multiple of the single viewable angle of view (640), when the divided screen is sequentially moved from the left side of the drawing to take an image, The two outermost divided screens (the divided images C and D and the divided images G and H) are overlapped with each other in the hatched portions.

Therefore, in this embodiment, with respect to the divided screens D and H on one side, the angle of view of the divided screen (0, 0-64)
0,240), excluding the overlapping range (non-overlapping area)
(340, 0-640, 240) and the image capturing positions of the divided screens D and H are transmitted to the image transmission device 6 as area designation data, and a request is made for image data acquisition, compression, and transmission for the designated area. The imaging position of each of the divided screens D and H is the center position of each divided screen, and the entire imageable range (0, 0-
2220, 480), the split screen D is (tilt = 1
750, pan = 120), on the split screen H (tilt = 1
740, pan = 360). Although it is possible to specify the capture range (area) for the divided screens A, B, C, E, F, and G, the capture range of the divided screen that does not overlap is the entire area (0, 0-640, 240). So,
In the present embodiment, the region is not specified from the personal computer 10 to the image transmission device 6. Thus, the personal computer 10
In this method, the image capturing position and the non-overlapping area are transmitted to the transmitting station as area designation data, and the image is compressed and transmitted only in the designated area. And data transmission and reception are avoided. Further, since the image is captured at the maximum angle of view (wide angle), the maximum background image without waste can be captured.

In the embodiment, the non-overlapping area is specified for the divided screens D and H.
For H, the entire area may be captured, and for the divided screens C and G, a non-overlapping area may be specified. In this case, the non-overlapping area of the divided screens C and G is (0, 0-3) in the angle of view (0, 0-640, 240) of the divided screen.
00, 240). In addition, the outer 2
It is not necessary to overlap the divided screens. For example, the divided screens B and C, the divided screens F and G are overlapped, and one of the divided screens (B and F, C and G, B and G, or C and Regarding F), a non-overlapping area may be specified. Also,
In the example shown in FIG. 5, in the vertical direction, the entire viewable angle of view (480) is an integral multiple of one image capture angle of view (240), so that no overlap occurs. Can be processed in the same manner as in the horizontal direction. If both the vertical and horizontal directions are not integral multiples, a maximum of four divided screens will overlap, but in this case, the entire area is captured only for one divided screen, and the other three divided screens overlap each other. Specify an area that avoids.

FIG. 6 shows a change in the background image screen 157 when the desired imaging position and range (size) of the video camera 4 are remotely controlled while confirming the entire imageable range with the background image. is there. 14A shows a screen before the camera frame 158 is changed, FIG. 14B shows a screen in which a new camera frame 159 is set, and FIG.
5 shows the screen after receiving the main screen corresponding to. In the state before the change shown in FIG. 6A, the current imaging position and range of the video camera 4 are indicated by a camera frame 158, and the image captured at this camera position is displayed in the main image window 151 (see FIG. 3). Is displayed. By moving and dragging the mouse cursor on the frame of the camera frame 158, the size of the camera frame 158 can be changed.
Further, the position of the camera frame 158 can be changed by moving the mouse cursor into the frame and dragging it. Since the user can confirm the entire range in which the image can be captured by remotely operating the video camera 4 by the background image, the user can easily change the camera position and range desired by the user.

When the position and size of the camera frame 158 are changed by operating the mouse 14, a new camera frame 159 is created at the changed position as shown in FIG. The background image is displayed in reverse. Since the camera frame 158 indicates the current imaging position and range of the video camera 4 (the image is displayed in the main image window 151), the camera frame 158 is displayed on the background image screen 152 as the current camera frame together with the new camera frame 159. Is displayed. If the range of the main image displayed in the main image window 151 is within the range of the new camera frame 159, the user selects the “setting transmission” key in the camera control window 152 to fix the new camera frame. Alternatively, the new camera frame may be determined by moving and releasing the mouse button while pressing it. As a result, the position information (pan, tilt, and zoom values) of the video camera 4 corresponding to the new camera frame 159
Is transmitted from the personal computer 10 to the image transmission device 6.

On the other hand, the image transmission device 6 periodically takes in image data picked up by the video camera 4 at predetermined intervals, and reads the position information (pan, tilt, and zoom values) of the video camera 4 at that time. The position information is added to the header of the image data on which the predetermined image processing has been performed, and transmitted to the personal computer 10. Then, the image transmission device 6
Receives the camera position change command from the personal computer 10, changes the angle and zoom of the video camera 4 via the control unit 5 so as to correspond to the new camera frame 159. The image data after the change is captured by the video camera 4 and the personal computer 1
Sent to 0. If it is time to take an image of the video camera 4 while moving from the position of the camera frame 158 to the new camera frame 159, the image data and camera position information at that time are transmitted to the personal computer 10.

In the personal computer 10, the position of the received image data is confirmed from the header of the image data.
If it does not correspond to the image data 95, it is determined that the image data is in the middle of the movement of the video camera 4, and the position of the camera frame 158 and the main image of the main image window 151 are displayed while the new camera frame 159 is displayed in reverse. Update. If the received image data corresponds to the new camera frame 159, as shown in FIG.
Of the camera frame 15
8 is deleted. Then, the image data received corresponding to the camera frame 159 is displayed in the main image window 151 as a main image.

Further, since the image received corresponding to the camera frame 159 is the latest image data, the background image in the camera frame 159 is also updated to the latest image data. This allows
Although partial, the background image in the range captured by the remote operation of the user is sequentially updated to the latest image.
In FIG. 6C, in which the background image in the camera frame 159 is updated, a gull is flying at the upper left of the yacht, as compared with FIG. It is understood that it is a background image.

Next, the details of the operation of capturing the background image will be described with reference to the flowcharts of FIGS.
First, the control unit 11 of the personal computer 10 reads out the dial number of the mobile phone 7 connected to the image transmission device 11 from the storage device 16 via the mobile phone 9 and automatically dials to connect a communication line. When there are a plurality of transmitting-side stations 1 to be remotely controlled by the personal computer 10, the names, installation locations,
Information such as a dial number is read from the storage device 16 and displayed on the display device 15 to prompt the user to make a selection. Then, the control unit 11 automatically dials the mobile phone 7 of the selected transmitting station 1 and connects the communication line (step 11). Note that the connection of the communication line with the image transmission device 6 is established when a manual dial is selected by the user.
The user makes a call to the mobile phone 9 or a dial number input from the keyboard 13.

Next, the control section 11 checks with the image transmission apparatus 6 whether or not a camera (video camera 4 and control section 5) capable of remote control is connected to the image transmission apparatus 6 (step 12). If the connection is not established (N), the process is terminated and the process proceeds to another process. On the other hand, when the video camera 4 or the like is connected (step 12; Y), the control unit 1
1 inquires of the type of the video camera 4 to the image transmission device 6 according to the following procedure (step 13). First, the control unit 1
1 transmits to the image transmission device 6 a command for asking the type and version of the camera. Upon receiving this command, the image transmission device 6 checks the type and version of the camera via the control unit 5 and transmits the same to the personal computer 10. Upon receiving the answer of the type and version of the camera, the control unit 11
Similarly, the image transmission device 6 is inquired about the commands for asking the pan range, the tilt range, and the zoom range of the video camera 4 sequentially. When the inquiry about all of the camera type, version, pan range, tilt range, and zoom range is completed, the control unit 11 stores these in the camera type area of the RAM 113 as camera type information.

Next, the control unit 11 accesses the storage device 16 and, in step 11, the transmitting station 1 connected to the line.
It is confirmed whether or not a background image corresponding to the image has already existed as a background image file (step 14). If there is a corresponding background image (step 14; Y), the storage device 16
Then, the background image corresponding to the connected transmitting station 1 is read from the background image file and stored in the background image file in the RAM 113 (step 15). Further, the control unit 11
Displays the read background image on the background image screen 1 of the display device 15.
57 is displayed (step 16), the present process is terminated, and the process proceeds to another process.

On the other hand, when the background image does not exist in the storage device 16 (Step 14; N), the control unit 11 performs a process of sequentially fetching the background image from a plurality of divided screens. That is, the control unit 11 monitors the user's selection of the "take in all screens" key (step 17). When the “full screen capture” key is selected (Y), the control unit 11 controls the pan and tilt coordinates when capturing the background image from the angle of view of the camera, and
When the divided images overlap, an area where the overlap is avoided (non-overlapping area) is calculated and stored in the RAM 113 (step 1).
8). For example, in the example shown in FIG.
The coordinates of pan and tilt with respect to each of a to 157l are calculated by the above equations (1) and (2),
13 is stored. As shown in FIG. 5A, when the divided images overlap each other, a non-overlapping area is calculated for one of the overlapping screens and stored in the RAM 113 together with the imaging position (pan, tilt) as area designation data. I do.

Next, the control unit 11 transmits a command for moving the video camera 4 to the corresponding division position of the background image, a zoom value, and, if the images overlap, the area designation data to the image transmission device 6 (step). 19; FIG. 8). In the present embodiment, a zoom value when the wide-angle is set as the transmitted zoom value is transmitted. By these commands, in the transmitting station 1, the imaging position of the video camera 4 (pan, pan,
The tilt and zoom values are changed, and the captured image at that position is compressed and transmitted by a predetermined method. When the image area designation data is transmitted, only the image of the designated area among the images captured at the designated position is taken into the image transmission device 6, and only the image of the area is compressed. And sent. Then, the control unit 11 receives the compressed image data sequentially transmitted from the image transmission device 6 via the mobile phone 9 and the communication control device 18 (step 20), and receives the compressed image data of the video camera 4 added to its header. From the position information, it is confirmed whether or not the received compressed image data is data captured at the designated position (step 21). If the received compressed image data is not the data at the designated position, that is, the image data is captured before or during the movement of the video camera (Step 21; N), the process returns to Step 20 to receive the compressed image data again. I do.

On the other hand, if the received compressed image data is data at the designated position (step 21; Y), the control unit 11
Expands (decodes) the received compressed image data by a method corresponding to the compression method in the image transmission device 6 and converts it into bitmap data (step 22). The converted bitmap data of the divided image is stored at the corresponding division position of the background image file in the RAM 113 (step 2).
3). Then, the control unit 11 displays the contents of the background image file in the RAM 113 on the background image screen 157 (step 24). As a result, the background image in the range where the capture of the divided image is completed is displayed on the background image screen 157.

Next, the control section 11 confirms whether or not the capture of the background images corresponding to all the divided screens has been completed (step 25). If not finished (Step 2
5; N), the control unit 11 updates the corresponding division position of the division screen to be taken in next as the background image (Step 26),
Returning to step 19, the division image capturing process is continued. On the other hand, when all background images have been taken (step 2
5; Y), the bitmap data of the entire background image stored in the RAM 113 is stored in the background image file of the storage device 16 in association with the currently connected transmitting station 1 (step 27). This ends the processing,
Move to another process.

Next, the details of the camera frame updating operation will be described with reference to the flowcharts of FIGS. The control unit 11 receives the compressed image data sequentially transmitted from the image transmission device 6 of the transmitting station 1 (step 31), and controls the camera position (pan, tilt) and zoom added to the header of the compressed image data. The position information including the value is read (step 32). Then, the control unit 11 obtains the background image screen 1 of the camera control window 152 from the read position information as shown in FIG.
The camera frame 158 is drawn in the area 57 (step 3
2). Further, the control unit 11 expands (decodes) the received compressed image data by a method corresponding to the compression method in the image transmission device 6 and converts the data into bitmap data (step 34). The converted bitmap data is R
It is stored in the main image file of the AM 113 and displayed in the main image window 151 (step 35). Therefore, an image of a range corresponding to the camera frame 158 of the background image screen 157 is displayed in the main image window 151 as a main image.

Here, the control section 11 determines whether or not the camera position change command has been transmitted to the image transmission device 6 (processing in step 42) (step 36). If not transmitted (step 36; N), the control unit 11 determines whether the camera frame 158 has been clicked by the user,
If not clicked within the predetermined time (step 3
7; N), returning to step 31 to continue the processing. When the camera frame 158 is clicked (step 37;
Y), the control unit 11 acquires the position and size of the new camera frame 159 set by the user (step 3).
8) It is monitored which of the “setting abort” key and the “setting transmission” key has been clicked (steps 39 and 40). When the “cancel setting” key is clicked (Step 39; Y), the control unit 11 cancels the display of the new camera frame 159 (Step 41) and returns to Step 31.

If the "setting transmission" key is clicked (step 40; Y), the position and size of the new camera frame 159 acquired in step 38 are converted into pan, tilt, and zoom camera commands (step 40). 42), and transmits a camera position change command to the image transmission device 6 (step 43), and reversely displays the background image in the new camera frame 159 (step 44). In this way, by displaying the background image in the new camera frame 159 in reverse, the user can confirm that the video camera 4 is moving.

On the other hand, if the camera position change command has been transmitted in step 36 (Y), the control unit 11
Furthermore, the camera position of the video camera 4 is changed to the new camera frame 15.
It is determined from the camera position read in step 32 whether or not the camera has moved to the position corresponding to 9 (step 45).
If it is not the camera position after the movement (Step 45; N),
Since the image data converted to the bitmap data in step 34 is not the image data corresponding to the new camera frame 159, the process returns to step 31 to continue receiving the image data corresponding to the new camera frame 159.

If the camera position of the video camera 4 has moved to a position corresponding to the new camera frame 159 (step 4).
5; Y), the control unit 11 returns the display of the background image in the camera frame 159 from the reverse display to the original display (step 4).
6). Then, for the background image file in the RAM 113, the bitmap data at the corresponding position is
The bitmap data updated in step 4 is updated (step 47). Since the background image is captured at the widest angle, the pixel density does not match and the background image file is thinned out at a predetermined interval unless the bitmap data converted in step 34 is imaged at the widest angle. Needs to be updated. In this case, the thinning rate is determined from the zoom value of the background image (the widest angle) and the zoom value of the new camera frame 159. After that, the control unit 11 updates the updated background image with
It is displayed on the background image screen 157 (step 48).

The control unit 11 further monitors whether or not an off process for terminating the process according to the present embodiment has been performed (step 49). If the off process has not been performed (N), the process returns to step 31 and returns to the image transmission device. 6 and the remote control of the video camera 4 is continued. On the other hand, if the off process has been performed (step 49; Y), the RAM 11
3 is stored in the storage device 1 based on the background image file stored in the storage device 1.
After the content of the background image file No. 6 is updated (step 50), the processing according to the present embodiment ends.

In the embodiment described above, after the compressed image data received from the image transmission device 6 is expanded and converted into bitmap data, the bitmap data is stored in the background image file in the RAM 113. Alternatively, the received compressed image data may be stored in a background image file in the RAM 113, and then expanded. In the above-described embodiment, the bitmap data of the background image is also stored in the background image file of the storage device 16. However, the compressed image data of the background image is also stored in the background image file of the storage device 16. You may make it.

As described above, according to the present embodiment, by sequentially capturing images for each divided screen, the entire range that can be captured by the video camera 4 is displayed on the background image screen as a background image. Can accurately determine an area to be confirmed in detail as a main image. Therefore, when image data transmission is relatively slow, unnecessary image transmission / reception due to an error in position designation is eliminated, and image data for an area that the user wants to check can be displayed in the main image window 151 quickly. Further, according to the present embodiment, since the video camera 4 is remotely operated by moving and enlarging / reducing the camera frame displayed on the background image screen 157, the user can easily operate the video camera 4.

Further, according to the present embodiment, the background image file at the corresponding position is updated using the bitmap data of the newly captured main image within the range of the new camera frame. However, the latest background image can be provided to the user. In addition, the first background image (step 27 in FIG. 8) and the new background image (FIG. 9)
Is stored in the storage device 16,
There is no need to capture a background image every time connection is made with the transmitting station 1, and the remote control of the video camera 4 can be performed immediately.

In the present embodiment, since the functions for remotely controlling the video camera 4 of the transmitting station 1 are processed by software, the programs for the respective functions must be stored in a recording medium such as a CDROM or a floppy disk. And can be widely provided. Then, by installing the program recorded on the recording medium into a personal computer, each function of the present embodiment can be realized. That is, the image data receiving apparatus of the present embodiment does not require special hardware, and can be easily configured by a personal computer and a recording medium.

For example, there is provided a control mechanism for controlling the position of the image pickup device in accordance with the received data on the image pickup position, and transmits an image picked up at a position controlled by this control mechanism and receives area designation data. In this case, an image capturing apparatus that transmits an image of a designated area in the captured image, and a recording medium that stores a program for realizing a function of causing a computer to receive an image from the captured image and display the image on an image display apparatus. And a duplication determination function for determining whether or not there is an area that overlaps with another image when the entire range that can be imaged by the imaging device is sequentially imaged by the control mechanism of the imaging device. A calculating function for calculating an area excluding the position of the imaging device and the overlapping range so that overlapping of images does not occur when it is determined that an overlapping area exists in the function;
A transmission function of transmitting the position of the imaging device calculated by the calculation function as data of the imaging position and an area excluding the overlapping range as area designation data to the imaging apparatus, and corresponds to each data transmitted by the transmission function. An image receiving function for sequentially receiving each image transmitted from the imaging device,
An image synthesizing function of synthesizing each image received by the image receiving function into one image and displaying the image on an image display device can be provided as a recording medium storing a program for causing a computer to realize the image synthesizing function. Further, when the control mechanism of the imaging device can change the imaging angle of view of the imaging device at the same imaging position, the image capturing device has an angle of view transmission function of transmitting data of the imaging angle of view to the imaging device, The overlap determination function is configured such that when an entire range that can be imaged by the image sensor by the control mechanism of the imaging device is sequentially imaged at the imaging angle of view transmitted by the angle of view transmission function, an image that overlaps with another image is generated. It may be recorded on the recording medium as a program for judging whether or not it exists.

Further, a divided image requesting function for dividing the entire range that can be imaged by the remotely operable imaging device into a plurality of ranges, and sequentially requesting the imaging of an image corresponding to each divided range, and a divided image requesting function. A divided image receiving function for sequentially receiving each of the divided images taken in response to the request, a combining function for combining the divided images received by the divided image receiving function, and a background image for combining the images combined by the combining function. A background image display function to display as, a position transmission function to transmit an imaging position by the imaging device, and an image captured at the imaging position transmitted by the position transmission function,
The present invention can be provided as a recording medium recording a program for causing a computer to realize a main image display function of displaying a main image. Further, an image range display function for displaying, in the background image displayed by the background image display function, an image range corresponding to the main image displayed by the main image display function, the computer realizes an image range display function. The program may be recorded on the recording medium. Further, the image range display function, when a new imaging position is transmitted by the position transmission function, until an image captured at the new imaging position is received, the image range at the new imaging position, The program may be recorded on the recording medium as a program for distinguishing and displaying an image range corresponding to the main image displayed by the main image display function.

In the embodiment described above, step 19
In the above description, the case where the zoom value at the widest angle is transmitted as the zoom value transmitted from the personal computer 10 to the transmitting station 1 has been described. Alternatively, the designated zoom value may be transmitted. in this case,
In step 18, the range captured by the designated zoom value is set as a divided screen, and the video camera 4
Of the image pickup position (pan, tilt), the presence / absence of screen overlap, and the non-overlap area. As a result, although the number of divided screens increases and transmission / reception takes some time, a clearer background image can be obtained. This is effective when there is enough time, such as when a background image is newly acquired and updated immediately before the end of business hours.

[0053]

According to the image data receiving apparatus of the present invention,
Since the entire range that can be imaged by the imaging device can be displayed on one screen, it is possible to accurately specify a desired image range and remotely control the image. Furthermore, since the captured images are transmitted without duplication, a composite image can be created without wasting time.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of an image data communication system using an image data receiving device according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a personal computer in a receiving station.

FIG. 3 is an explanatory diagram of a display screen of the display device of the embodiment.

FIG. 4 is an explanatory diagram of each divided screen when a background image is captured.

FIG. 5 is an explanatory diagram showing overlap of captured images which occurs due to the relationship between the entire range in which images can be captured and the angle of view of a divided screen.

6A and 6B show changes in a background image screen when a video camera is remotely controlled while confirming the entire imageable range with a background image. FIG. 6A shows a screen before a camera frame is changed. , (B) shows a screen in a state where a new camera frame is set,
(C) shows the screen after receiving the main screen corresponding to the new camera frame.

FIG. 7 is a part of a flowchart illustrating details of a background image capturing operation according to the embodiment;

FIG. 8 is another part of the flowchart showing details of the background image capturing operation according to the embodiment.

FIG. 9 is a part of a flowchart showing details of a camera frame update operation according to the embodiment;

FIG. 10 is another part of the flowchart showing details of the camera frame update operation according to the embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 transmitting station 2 receiving station 4 video camera 5 control unit 6 image transmission device 7, 9 mobile phone 8 modem 10 personal computer 11 control unit 111 CPU 112 ROM 113 RAM 13 keyboard 14 mouse 15 display device 151 main image window 152 camera Control window 153 Full screen capture key 154 Detailed setting key 155 Setting cancel key 156 Setting transmission key 157 Background image screen 158 Camera frame 159 New camera frame 16 Storage device 17 Storage medium drive device 18 Communication control device 19 Input / output IF

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Toshio Takaichi 2-19-12 Sotokanda, Chiyoda-ku, Tokyo Inside Equos Research Inc. (72) Inventor Yukio Yamamoto 2-19 Sotokanda, Chiyoda-ku, Tokyo 12 Equos Research Co., Ltd. (72) Inventor Toshihiro Mori 2-19-1, Sotokanda, Chiyoda-ku, Tokyo (72) Inventor Yuki Ishikawa 2-19, Sotokanda, Chiyoda-ku, Tokyo No. 12 Equos Research Co., Ltd. (56) References JP-A-8-275045 (JP, A) JP-A 8-88787 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) ) H04N 5/232 H04N 7/18

Claims (2)

(57) [Claims] A control mechanism for controlling the position of the image sensor in accordance with the received data on the imaging position; transmitting an image captured at a position controlled by the control mechanism; and receiving area designation data. An image data receiving device that receives an image from the image capturing device that transmits an image of a designated area in the captured image, and displays the image on an image display device. When sequentially capturing the entire range that can be imaged by the image sensor, overlap determining means for determining whether or not there is an area that overlaps with another image, and the overlapping determining means determines that an overlapping area exists. In the case of, the calculating means for calculating the area excluding the position of the imaging device and the overlapping range so that the image does not overlap, the position of the imaging device calculated by the calculating means as data of the imaging position, Transmitting means for transmitting the area excluding the overlapping range as the area designation data to the image capturing apparatus; and image receiving means for sequentially receiving each image transmitted from the image capturing apparatus corresponding to each data transmitted by the transmitting means. An image data receiving apparatus comprising: an image synthesizing unit that synthesizes each image received by the image receiving unit into one image and displays the image on an image display device. 2. An image capturing apparatus according to claim 1, further comprising an angle-of-view transmitting unit configured to transmit data of an angle of view of the imaging device to the imaging device when a control mechanism of the imaging device can change an angle of view of the imaging device at the same imaging position. The overlap determining unit, when sequentially capturing the entire range that can be imaged by the image sensor by the control mechanism of the imaging device at the imaging angle of view transmitted by the angle of view transmission unit, overlaps with another image. 2. The image data receiving apparatus according to claim 1, wherein it is determined whether or not the resulting image exists.