JP3577127B2 - Image handling apparatus and image composition method - Google Patents

Description

[0001]
[Industrial applications]
The present invention relates to an image handling apparatus that is used in a business camera (hereinafter, referred to as an OA camera) for forming a high-definition image and that performs a process of bonding each partial image by a plurality of image sensors, and the like.And image composition methodIt is about.
[0002]
[Prior art]
Conventionally, the following OA camera has been proposed by the present applicant as an image handling apparatus using a predetermined method for obtaining an image with a large number of pixel configurations with a small number of pixel configurations.
[0003]
That is, the predetermined method is to scan a target image by a mirror for each partial image, and fetch each of the captured partial images for each of the plurality of image sensors corresponding to each of the plurality of image sensors. This is a method in which a multi-pixel image to be captured in a large area can be input by exposing and sequentially bonding the partial images obtained in this manner again. FIG. 19 is a block diagram for explaining the configuration of an OA camera as an example of its use.
[0004]
Hereinafter, the configuration and operation of the OA camera as the image handling device will be described with reference to FIGS.
[0005]
In FIG. 19, the total reflection mirror 101 reflects the incident light 102 in a predetermined direction and changes the reflection angle with respect to the image to be captured (hereinafter, changing the angle of the total reflection mirror 101 is referred to as mirror scan). On the optical axis of the total reflection mirror 101, a lens unit 103 including a stop, a focusing lens, and the like, and a half mirror 104 on the optical axis of the total reflection mirror 101. Each is provided in order, and the first CCD 105 and the second CCD 106 are arranged as two imagers. The first CCD 105 is configured to receive the passing light 102a of the incident light 102 that has passed through the half mirror 104, and the second CCD 106 is configured to receive the reflected light 102b of the incident light 102 reflected by the half mirror 104. I have. As shown in FIG. 20, at each position of the total reflection mirror 101, the first CCD 105 has a substantially upper half area of the partial images A, B, C, and D, which is a part of the imaging target image 200. Are exposed, and the second CCD 106 exposes lower half images A2, B2, and C2 corresponding to substantially lower half regions of the same partial images A, B, C, and D. In order to expose C2 and D2 at the same time, the positional relationship between the two imagers is configured to be shifted by a predetermined amount with respect to each optical axis. Here, FIG. 20 is an explanatory diagram for explaining an overlapping area and the like of each partial image.
[0006]
Further, the incident light 102 captured by, for example, the total reflection mirror 101 at the position of the initial angle is output from each imager, and passed through the corresponding first process circuit 107 or second process circuit 108 to output a video signal. Is converted to When the two CCD pixels as imagers are both 752 pixels × 580 pixels, an upper half image A1 composed of 752 pixels × 580 pixels (in FIG. 20, a region indicated by solid hatching). A lower half image A2 of the same pixel size (the area shaded by the dotted line in FIG. 20) is used as an overlap margin A12 of 100 pixels × 580 pixels (the area where the solid line and the area shaded by the dotted line are overlapped in FIG. 20). ) Is input to the vertical bonding unit 109 for uniformly bonding. That is, in each video signal input to the vertical combining unit 109, the upper half image A1 and the lower half image A2 are uniformly combined and fetched by a single imager having 1404 pixels × 580 pixels. It is converted into a video signal as if it were (corresponding to the partial image A of the imaging target image 200 shown in FIG. 20).
[0007]
Such an operation is repeated every time the total reflection mirror 101 is scanned at a predetermined position.
[0008]
Next, after the upper half image A1 corresponding to the partial image A and the lower half image A2 are captured, the mirror driving unit 110 operates based on an instruction from the CPU 116 to move the total reflection mirror 101 by a predetermined angle from the initial position. Set to the moved position. At the position where the mirror scan is performed in this manner, the upper half image B1 corresponding to the substantially upper half area of the partial image B of the imaging target image 200 is again obtained by the same processing as described above. A video signal as if the lower half image B2 corresponding to the lower half area was uniformly attached and captured by one imager (corresponding to the partial image B of the image 200 to be captured shown in FIG. 20) Is converted to
[0009]
Furthermore, in order to capture the partial images C and D, the area to be captured changes each time the total reflection mirror 101 is sequentially moved by a predetermined angle (mirror scan), and the same processing as described above is repeated to obtain a large picture as a whole. can get.
[0010]
However, what is important here is that, as shown in FIG. 20, for example, as shown in FIG. 20, in order to further combine the captured and uniformly combined partial images (partial images A, B, C, and D), At the position, the partial image A captured by the total reflection mirror 101 for the first time and vertically laminated and the partial image B captured in the second time and formed in the same manner, such as the partial image B, are each overlapped. The image is configured to have a predetermined overlapping portion, that is, an overlapping region composed of 1404 pixels × 72 pixels as shown in FIG.
[0011]
That is, in order to perform such superposition, as shown in FIG. 19, an overlapping area of a previously captured partial image (for example, an overlapping area Ab belonging to the partial image A as shown in FIG. Buffer 111 for temporarily storing, the overlapping area of the contents of the boundary buffer 111 and the previously captured partial image of the next captured partial image (for example, the overlapping area Ab belonging to the partial image A) Is compared with the contents of the overlapping area (for example, the overlapping area Ba belonging to the partial image B) to find a characteristic part of the image, that is, a part of the same picture, and to detect a shift between the pictures. A vector detecting unit 112, a shift amount calculating unit 113 for calculating a shift amount using the detection result, and a partial image (for example, a unit In accordance with the image A), the position of the next captured partial image (for example, the partial image B) is corrected, and the image data joined together is output by the coordinate conversion unit 110 and the like for correctly overlapping the overlapping images. This is output from the unit 114. Note that the coordinate conversion referred to here is a parallel movement or a rotation movement, and includes a conversion process such as enlargement / reduction when the sizes of target images are different. In this way, each time the mirror scan is performed, the partial images that are sequentially overlapped are connected.
[0012]
Here, the following may be considered as factors that cause a shift in the mutual positional relationship between the captured overlapping partial images.
[0013]
For example, it is possible that the overlapping partial images captured by the mirror scan are configured to have a predetermined overlapping portion that is predetermined in advance, that is, an overlapping region composed of 1404 pixels × 72 pixels as shown in FIG. As described above, it is possible to determine whether there is an error or variation in the scan amount (predetermined angular movement amount) of the total reflection mirror 101, or while capturing the partial images sequentially at predetermined timing. Is changed, or conversely, a position shift on the side of the capturing device such as a camera shake by the operator.
[0014]
On the other hand, as described above, the lens unit 103 includes an aperture, a focus lens, and the like. In order to control these at the time of imaging, the output signals of the first CCD 105 and the second CCD 106 as imagers are accumulated. Sent to
[0015]
The data subjected to the so-called cumulative addition processing in the cumulative addition unit 115 is sent to the CPU 116, and based on the data, the control signal from the CPU 116 drives the AE (automatic exposure adjustment) and the AF (automatic focusing). Is performed.
[0016]
Here, for example, the cumulative addition process in AF means that a set of pixels constituting a CCD as an imager is divided into predetermined regions to form one area, and each area is obtained by each pixel. Focusing on the high frequency components of the image data, it is intended to obtain contrast data (the larger the cumulatively added data is, the greater the contrast is determined) by cumulatively adding those waveforms.
[0017]
Each of the obtained contrast data is stored in a predetermined memory (not shown), and each of the contrast data is read out from the memory as necessary, and the AF is controlled based on the data. The control signal is transmitted to the focus lens of the lens unit 103.
[0018]
[Problems to be solved by the invention]
However, in such a configuration, if there is no image feature to be used for superimposition joining in the overlapping area of the partial images, how much misalignment of the partial images to be joined has occurred with respect to each other However, there is a problem that there is no way to cope with this. Further, since there is no criterion or means for determining the presence or absence of a characteristic portion of the image, there is also a disadvantage that it takes a long time to search for a characteristic portion which may or may not be present for a long time.
[0019]
The present invention takes into account such problems of a conventional image handling apparatus, and when using a DCT (Discrete Cosine Transfer) used for a JPEG (Joint Photographic Expert Group) or the like, a part without adding a special circuit. It is possible to determine whether or not an image feature to be used for superimposition joining exists in the overlapping area of the image.Enables efficient image composition processingImage handling equipmentAnd image composition methodThe purpose is to provide.
[0020]
[Means for Solving the Problems]
To solve the aforementioned issues,The present inventionImage handling equipmentIs an image andThePartial overlap with one imageHaving an areaFor other images,In an image handling apparatus having an image synthesizing function of performing a bonding process by aligning both at a characteristic portion of the overlapping region,The one imageThe overlapping area on the sideAn image signal representing and / or the other imageThe overlapping area on the sideImage signal representingFrom eachOrthogonal transform means for obtaining orthogonal transform data;On the basis of the obtained orthogonal transform data, a contrast data extraction unit for obtaining contrast data, and the overlapping area on the one image side and / or the overlapping area on the other image side,Whether there is a feature necessary to overlap the one image and the other image,By comparing the obtained contrast data with a predetermined contrast value,Determining means for determining, and the determination result by the determining meansBased on the image synthesis control means for controlling the combining processEquippedCharacterized by.Further, in the image handling apparatus according to the present invention, when the image combining control unit determines that the characteristic portion is present by the determination unit, the image combining control unit performs the bonding process, and determines that the characteristic portion does not exist. In this case, a warning that bonding is impossible is issued. Further, in the image handling apparatus according to the present invention, the orthogonal transform unit is a DCT processing unit that obtains a DCT coefficient as orthogonal transform data, and the contrast data extracting unit accumulates only a necessary part from the obtained DCT coefficient. It is characterized by further comprising a cumulative addition section for adding.
Image synthesis method for image handling device according to the present inventionIsAn image handling apparatus having an image combining function of performing alignment processing of one image and another image having an area that partially overlaps the one image by aligning the two at a characteristic portion of the overlapped area An image transformation method for obtaining each orthogonal transformation data from an image signal representing the overlapping region of the one image and / or an image signal representing the overlapping region of the other image, A contrast data extraction step of obtaining contrast data based on the obtained orthogonal transformation data; and, in the overlapping area on the one image side and / or the overlapping area on the other image side,Whether there is a feature necessary to overlap the one image and the other image,By comparing the obtained contrast data with a predetermined contrast value,A determining step for determining;
The judgment result of the judgment stepAnd an image combining control step of controlling the combining process based on
Also, an image synthesizing method for an image handling apparatus according to the present invention.IsThe image combining control step performs the bonding process when the characteristic portion is determined to be present by the determining step, and when the characteristic portion is determined not to be present, bonding is not possible. A warning to the effect is issued. Further, an image synthesizing method of the image handling apparatus according to the present inventionIsThe orthogonal transformation step is a DCT processing step of obtaining DCT coefficients as orthogonal transformation data, and the contrast data extraction step further includes a cumulative addition step of cumulatively adding only a necessary part from the obtained DCT coefficients. It is characterized by.
[0021]
[Action]
In the present invention, the orthogonal transform meansEach orthogonal transform data is obtained from an image signal representing an overlapping area on the one image side and / or an image signal representing an overlapping area on the other image, and contrast data extracting means converts the orthogonal transform data into the obtained orthogonal transform data. And determining the contrast data based on the determined contrast data and comparing the determined contrast data with a predetermined contrast value to determine whether or not the overlap area on the one image side and / or the overlap area on the other image side. Determining whether there is a characteristic part necessary for superimposing the one image and the other image, and determining that the characteristic part is present by the determining means. In such a case, the bonding process is performed, and if it is determined that the characteristic portion does not exist, a warning that bonding is impossible is issued.
[0022]
【Example】
Hereinafter, embodiments according to the present invention will be described with reference to the drawings.
[0023]
FIG. 1 is a block diagram illustrating a configuration of an OA camera as an example of the image handling apparatus according to the present embodiment. FIG. 2 is a block diagram illustrating a configuration of the accumulative adding unit 115 according to the present embodiment. The configuration of this example will be described with reference to FIG.
[0024]
Note that the same components as those in FIG. 19 are denoted by the same reference numerals, and description thereof will be omitted. The main difference from the configuration of FIG. 19 is that, in addition to the conventional method of using the contrast data obtained by the accumulating unit 115 for AF control, the difference between the overlapping areas of the partial images to be joined is It is determined whether or not a characteristic portion exists in the CPU 2 and the result can be output from the joint information output terminal 1 of the CPU 2.
[0025]
Here, the determination means of the present invention is included in the CPU 2, and the predetermined characteristic evaluation criterion of the present invention is a criterion defined for the contrast data. The coordinate conversion unit 110 includes a boundary buffer 111, a motion vector detection unit 112, and a shift amount calculation unit 113.
[0026]
In FIG. 2, a band-pass filter (BPF) 115a is for extracting only a predetermined high-frequency component from input image data, and an absolute value calculator 115b is configured to convert a negative waveform of the input signal into a positive waveform. The accumulator 115c converts each input waveform obtained by the absolute value calculator 115b into a predetermined accumulative addition area 31 (a first CCD 105 shown in FIG. 3A). And each of the second CCDs 106 is divided into 64 areas, and one area of the divided 94 pixels × 72 pixels is calculated within the cumulative addition area 31). In the contrast data memory 115d, 64 pieces of contrast data obtained by accumulative addition are created for one CCD. A storage means for storing the 128 contrast data in parts. Here, FIG. 3A is an explanatory diagram for explaining the configuration of the partial image, and FIG. 3B is an explanatory diagram for explaining the cumulative addition area.
[0027]
In the above configuration, the operation of this example will be described mainly with reference to FIG.
[0028]
In the figure, as for the input image data 21, only the high-frequency component waveform 22 is taken out by the BPF 115a, and the waveform 23 is converted into an absolute value by passing through the absolute value calculator 115b. The shaded portion 24 of the converted waveform is cumulatively added in each of the cumulative addition areas 31, and a total of 128 pieces of contrast data are stored in the contrast data memory 115d as one partial image.
[0029]
Then, the CPU 2 reads each contrast data from the contrast data memory 115d as necessary, sends a control signal for controlling AF to the AE / AF driving unit 117 based on the data, and Control the focus lens.
[0030]
Further, the CPU 2 determines whether or not a characteristic portion exists in the overlapping area between the partial images to be joined, and outputs the result from the joining information output terminal unit 1.
[0031]
That is, the CPU 2 determines, as the partial image to be joined, for example, an overlapping area Ab belonging to the partial image A in the overlapping area of the partial image A and the partial image B (the first rightmost part of the partial image A shown in FIG. Paying attention to 16 contrast data (corresponding to 16 cumulative addition areas, which are denoted by reference numerals 16 to 16), a predetermined contrast value (a predetermined characteristic evaluation criterion as a predetermined characteristic evaluation criterion) If these 16 values are all smaller than the predetermined contrast value, it is determined that there is no contrast in the overlapping area Ab, that is, there is no feature.
[0032]
As described above, the CPU 2 does not perform the bonding because there is no contrast data exceeding the predetermined threshold value in any of the cumulative addition areas in the overlapping area Ab (the same applies to other overlapping areas). When it is determined that the connection is possible, information indicating that is output from the joining information output terminal unit 1.
[0033]
According to the output signal from the joining information output terminal unit 1, information indicating that bonding is impossible is output to the operator as an alarm. The output signal from the joint information output terminal 1 is also used as a control signal for a shutter operation for automatically guarding the shutter of the OA camera of the present embodiment so that the shutter does not operate. Here, the alarm includes various methods such as, for example, emitting an LED in a viewfinder or emitting a sound for calling attention.
[0034]
On the other hand, for example, when the CPU 2 determines that contrast data exceeding the threshold value exists in the cumulative addition area in the overlapping area Ab and it is possible to perform the bonding, the coordinate conversion unit 110, the shift amount calculation unit 113, and the like. In this method, the partial images are bonded and joined in the same manner as in FIG.
[0035]
In this way, the one image and the other image are superimposed on the one image side and / or the other image side of the area where one image and another image partially overlap. A determination unit for determining whether there is a characteristic part necessary for the determination based on a predetermined characteristic evaluation criterion determined in advance, and using the determination result by the determination unit, the one image It is possible to realize an image handling apparatus including a joining image forming unit for performing a bonding process by superimposing the other image and the other image.
[0036]
Therefore, according to the present example, it is possible to determine whether or not there is an image feature to be used for superimposition joining in the overlapping area of the partial images.
[0037]
More specifically, if there is no characteristic portion in the overlapping area, the operator determines that the feature cannot be attached without adding a special circuit or the like, and determines that bonding is not possible. The information such as an alarm to be transmitted to is output.
[0038]
In the present embodiment, the determination unit focuses on the contrast data as AF control data, and a case has been described in which the conventional cumulative addition unit 115 and the CPU 2 are used without using a special dedicated circuit. The present invention is not limited to this, and may be realized by configuring a dedicated circuit for the above determination. The predetermined feature evaluation criterion may be realized not only for the contrast data but also for the overlapping area of the partial images. As long as it can be determined whether or not there is a characteristic part of an image to be used for lamination joining in, the contents of the realized circuit and the contents of the predetermined characteristic evaluation criterion do not matter.
[0039]
Further, in the above-described example, the case where the presence of the characteristic portion is determined with respect to the overlapping area belonging to the previously captured partial image of the two overlapping partial images has been described. However, the present invention is not limited to this. For example, an overlap area belonging to a partial image captured later may be targeted, or both may be targeted.
[0040]
Further, in the above example, the output of the information indicating that the bonding is not possible is output from the joining information output terminal unit 1 by the purpose of locking the operation of the shutter or calling attention to the operator. However, the present invention is not limited to this, and the use of the output from the joint information output terminal unit 1 is not limited to the condition, the scene, the purpose, or the like.
[0041]
In addition, the present invention may be realized by hardware using each component, or by not only this, but also by software.
[0042]
The content to be described next is that, when performing the bonding and joining of the partial images using the OA camera described above, if there is no contrast data exceeding the predetermined threshold, the OA camera side performs This is related to an example in which contrast is actively created.
[0043]
In the above embodiment, if there is no contrast data exceeding a predetermined threshold value in the overlapping area, since the sticking cannot be performed, the shutter operation of the OA camera is automatically locked, and for example, the operator may mistakenly shoot an image. Is to be prevented beforehand. Accordingly, although there is a great advantage that it is possible to prevent the displaced images from being stuck together while being displaced, this is the same as the conventional case in that the displaced images cannot be correctly stuck.
[0044]
Therefore, in this example, it is possible to determine whether or not there is a characteristic portion of an image to be used for superimposing and joining in the overlapping region of the partial images in consideration of the above-described points and the like, and furthermore, the characteristic It is an object of the present invention to provide an image handling apparatus capable of correcting a positional shift between partial images and pasting them correctly even when a part does not exist.
[0045]
FIG. 4 is a block diagram showing a configuration of an OA camera as an example of the image handling apparatus. The configuration of this example will be described with reference to FIG.
[0046]
Note that the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. The main difference from the configuration of FIG. 19 is that even when it is determined by the CPU 2 or the like that the characteristic portion does not exist in the overlapping area, the image pickup target In contrast, the LED 11 (which is a part of the marker projecting means) as an auxiliary light element and the timing of light emission of the LED 11 are controlled with the intention of actively creating a contrast as a characteristic portion. And an auxiliary light driving unit 12 (which is a part of the marker projecting means).
[0047]
The LED 11 projects light having a wavelength that can be identified by the first CCD 105 (or the second CCD 106) as an imager. The LED 11 is provided inside the OA camera so that light projected by the LED 11 illuminates a portion corresponding to an overlapping area of the partial images with respect to the imaging target. The auxiliary light driving unit 12 controls the light emitting operation of the LED 11 as the auxiliary light element based on an instruction from the CPU 2 that there is no feature in the overlapping area.
[0048]
The operation of this example in the above configuration will be described with reference to FIGS.
[0049]
Here, FIG. 5 is a time chart for explaining the operation of this example.
[0050]
First, when the power of the OA camera is turned on (the ON / OFF state of the power of the OA camera is indicated by the POWER signal 51 in FIG. 5), the mirror scan starts regardless of whether or not an image is taken. Here, in FIG. 5, the operation status of the mirror scan is represented by A, B, and A to indicate the correspondence with each partial image (partial images A, B, C, D,...) Captured at a predetermined timing. Scanning is repeated in the order of A, B, C, D, A, B, C, D,... As shown in the mirror scan time chart 52 indicated as C and D. In the mirror scan time chart 52, a “bore line” connecting between the alphabetic characters A to D indicates that the total reflection mirror 101 is moving, and furthermore, each alphabetic character is surrounded by a “square”. This indicates that, for example, in the portion A, the picture of the area of the partial image A is displayed on the imager (the first CCD 105 and the second CCD 106). Light is accumulated in the imager at the timing A in the mirror scan time chart 52.
[0051]
This is sequentially repeated, and scanning is performed from A to B, and then from B to C, and a predetermined partial image is projected on an imager every time the total reflection mirror 101 moves sequentially. By performing the mirror scan in this manner, the outputs 53 of the CCDs (the first CCD 105 and the second CCD 106) are partially output such that, for example, after the partial image A is exposed, the output of the output A is delayed by a predetermined timing. The output is delayed by a predetermined timing with respect to the image exposure timing, but the order of A, B, C, and D is basically unchanged.
[0052]
Here, if it is determined by the CPU 2 that contrast data exceeding a predetermined threshold, for example, as a characteristic portion of an image necessary for superimposition, does not exist in the overlapping area of the partial images, the joining information output is performed. A high signal is output as the alarm signal 54 from the terminal 1 (see FIG. 4). Here, the alarm includes various methods such as, for example, emitting an LED in a viewfinder or emitting a sound for calling attention.
[0053]
A High signal is output as the alarm signal 54, and a search for a feature is repeated several times as necessary. If the search is still not found, it is determined that there is no feature, that is, no contrast. In this case, when the release button is pressed (corresponding to the case where the release signal 55 indicates the High level in FIG. 5), the CPU 2 determines that there is no contrast in the bonding area at this time. Therefore, the LED 11 is emitted as auxiliary light to create a pseudo-contrast, and this is used to bond partial images.
[0054]
As shown in the mirror scan time chart 52, the LED 11 emits light twice at a timing 51a at which the partial image A is exposed and at a timing 51b at which the partial image B is exposed (the LED 11 emits light). The light emission timing corresponds to the case where the auxiliary light signal 56 in FIG. 5 indicates the High level).
[0055]
Since the place where the auxiliary light (LED 11) is projected must be in the overlapping area of the partial images, for example, as shown in FIG. Is the left end 62 of the image plane.
[0056]
In this way, the auxiliary light is projected at each position of the partial image.
[0057]
Originally, the overlapping area between the partial images was a “solid image” with no feature for bonding and no contrast, but the pseudo light was projected by this auxiliary light. There are various characteristic parts, and it is possible to detect the positional shift, and the partial images can be correctly pasted together without causing the positional shift. Therefore, for example, even when the camera is shifted due to camera shake during operation or the like and the partial image A and the partial image B are shifted from the original position by Y in the vertical direction, based on the information of the auxiliary light, If they are stuck together, they can be stuck correctly.
[0058]
In this way, the one image and the other image are superimposed on the one image side and / or the other image side of the area where one image and another image partially overlap. Determining means for determining whether there is a characteristic part necessary for the determination based on a predetermined characteristic evaluation criterion, and superimposing the image capturing target corresponding to the overlapping area A marker projecting unit for projecting a marker to be used for the first image and the other image by using the determination result by the determining unit to form a joined image for performing a bonding process And an image handling apparatus having the means.
[0059]
Therefore, according to the present example, it is possible to determine whether or not there is a feature of an image to be used for superimposition joining in the overlapping region of the partial images. Even if there is, it is possible to correct the positional deviation between the partial images and to paste them correctly.
[0060]
The marker in the present invention has been described as LED light in the above embodiment, but the present invention is not limited to this, and any marker may be used as long as the imager can be identified.
[0061]
When light is used as a marker, light having any wavelength may be used as long as the imager can be identified, and the type of light is not limited.
[0062]
In the above example, when there is no characteristic portion in the overlapping area of the partial images, the device side projects the light of the LED to the imaging target, thereby positively creating the contrast, and the position of the partial images relative to each other. The displacement can be detected and correct bonding processing without the displacement can be performed.
[0063]
However, in such a configuration, there is a case where a contrast that does not originally exist in the imaging target is output to the output image.
[0064]
Therefore, although it is necessary for the bonding process of bonding, as an output image, unnecessary contrast, that is, an objectionable image that is not intended to exist should be prevented from appearing in the output image. Next, an example in which can be erased in advance will be described.
[0065]
If the overlapping region of the partial images is determined to be a region having no characteristic portion, that is, a region having no contrast, the region is determined to be a “blurry image” with little change, and the above-mentioned disturbing image is determined. Is replaced with a peripheral image, the shining part is deleted, the image is returned to an image very close to the original image, and then an interpolation unit 71 (see FIG. 7) for outputting the image is provided. Here, FIG. 7 is a block diagram showing the configuration of the interpolation unit 71. In FIG. 7, the parts marked with (1) are marked with (1) as the output side of the output unit 114 in FIG. The corresponding portions correspond to those in a circuit-like state.
[0066]
As another example, a case of an OA camera that handles a binarized signal will be described below with reference to FIG. Here, FIG. 8 is a block diagram showing the configuration of the binarizing unit 75. In FIG. 8, the portions marked with (1) and (3) correspond to the output side of the output unit 114 in FIG. (1) and (3) as the output side of the joint information output terminal unit 1 correspond to each other, indicating that they are in a mutually circuit-connected state.
[0067]
In this case, the output signal is binarized, and when the light of the LED 11 is superimposed on the binarized output image signal, only the light-irradiated portion becomes “black” even though the surroundings are “black”. The image becomes “white” and becomes different from the original image.
[0068]
In this case, as described in the above example, there is no contrast in the overlap region. For example, only the overlap region between the right end of the partial image A and the left end of the partial image B is forcibly set to “black”. By performing the process of performing the above, even in the case of binarization, it can be handled in the same manner as in the above example.
[0069]
By the way, in the embodiment described below, in order to record an image which has been subjected to a joining process by pasting partial images at a high density, for example, a device which performs band compression based on the JPEG format and outputs the resultant image has overlapping image portions. With reference to the drawings, a specific description will be given of an image handling apparatus capable of correcting a positional shift between partial images and pasting them correctly even when a characteristic portion does not exist in an area.
[0070]
FIG. 9 is a block diagram showing the configuration of an OA camera as an embodiment of the image handling apparatus according to the first aspect of the present invention. The configuration of this embodiment will be described with reference to FIG. Note that the same components as those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted. The main difference from the configuration shown in FIG. 4 is that, in the present embodiment, when contrast data is obtained, DCT processing based on the JPEG format (here, DCT processing mainly refers to the arrangement of images) This is a point of utilizing a DCT coefficient obtained in a process of performing band conversion of image data.
[0071]
In FIG. 9, a DCT processing section 81 (corresponding to the orthogonal transformation means of the present invention) performs DCT processing on the block image data, and a zigzag scanning section 82 performs a series of transformations in the frequency domain. Are serially rearranged in order from low-frequency components to high-frequency components, and the accumulator 83 accumulates only a necessary portion of the signal after the zigzag scan, thereby obtaining an AE signal. Data and AF contrast data are obtained. The Huffman encoding unit 84 encodes the data after the zigzag scan.
[0072]
The operation of the present embodiment in the above configuration will be described with reference to FIG.
[0073]
First, DCT processing is performed on block data corresponding to a block 91 of 8 pixels × 8 pixels of 64 pixels based on the JPEG format, and a number 1 assigned to each of the block data is given. Are decomposed for each frequency in the order of No. 64 to No. 64. Here, FIG. 10A is an explanatory diagram showing a block 91 of 64 pixels of 8 × 8 pixels based on the JPEG format, and FIG. 10B shows a DCT process for the blocked data. It is an explanatory view for explaining.
[0074]
The data with the number 1 has a lower frequency, and the data with the larger number has a higher frequency component. When this is zigzag scanned by the zigzag scanning unit 82, the data of numbers 1 to 64 are serially rearranged in order (see FIG. 10B). Here, the number 1 is a DC component in the block 91 of 8 × 8 pixels, and the frequency component increases as the number increases in the order of numbers 2, 3,....
[0075]
Therefore, for example, in order to obtain necessary data for performing AE control using a low frequency component of a signal, the cumulative addition unit 83 uses only relatively low frequency components such as numbers 1 to 4 to obtain data. By performing the cumulative addition, data for AE control is obtained.
[0076]
On the other hand, in the case of AF control, a relatively large contrast is required. Therefore, cumulative addition is performed by the cumulative addition unit 83 using only a relatively high frequency region, and data for AF control is obtained.
[0077]
In this manner, as a result of the zigzag scanning by the zigzag scanning unit 82, only the parts necessary for the predetermined control are cumulatively added by the accumulative adding unit 83, and the information is sent to the CPU 2 to drive the AE and the AF. Is
[0078]
As is apparent from the above, when the CPU 2 determines that there is no contrast in the overlapping area of the partial images, the CPU 2 outputs alarm information or further projects auxiliary light toward the imaging target, By positively forming the pseudo-contrast, it is possible to form a correct joint image without positional displacement.
[0079]
Also, in a system using DCT used in JPEG or the like, it is determined whether or not there is a feature necessary to superimpose one image on another image without adding a special circuit. An evaluation criterion that can be used for making a determination can be easily obtained. Therefore, according to the present embodiment, unlike the conventional apparatus without the above-described evaluation criterion or the like, the process of searching for a feature point takes a long time and the photographing time does not become long.
[0080]
Further, from the above, according to the above-described embodiment, for example, an image to be used for superimposition and joining in an overlapping region of a partial image based on a criterion defined for orthogonal transform data such as the DCT coefficient is used. It is possible to determine whether or not there is a characteristic portion, and even if the characteristic portion does not exist, it is possible to correct the positional displacement between the partial images and bond the partial images correctly.
[0081]
Further, the present invention is not limited to a camera but can be applied to other image handling apparatuses such as a copying machine, a printer, a facsimile, and a film scanner.
[0082]
In the above embodiment, the case where the DCT coefficient or the like is used as a reference defined for the orthogonal transform data of the present embodiment has been described. However, the present invention is not limited to this. Of course, it may be realized by using a Fourier transform coefficient or the like.
[0083]
By the way, in the above-described OA camera and the like, when performing a joining process by bonding partial images, the following process is proposed by the present applicant.
[0084]
Although the configuration of this OA camera is shown in FIG. 21, it is almost the same as the content of FIG. 19 described above except for the binarization unit 119, and a description thereof will be omitted.
[0085]
That is, the main difference from FIG. 19 is that in FIG. 21, the image data subjected to the coordinate conversion by the coordinate conversion unit 110 is binarized by the binarization unit 119 and output via the output unit 114. At the same time, the motion vector is stored in the boundary buffer 111 and a motion vector is detected based on the stored binary data.
[0086]
Usually, it is conceivable to calculate the mutual positional shift amount of the partial images in the overlapping area by the shift amount calculating unit 113 and to perform the operation at a position where the mutual image switching is uniformly fixed.
[0087]
For example, as shown in FIG. 23A, if a character “1” is supposed to be located at a predetermined switching location on the left end of the partial image A, the partial image A is moved from the predetermined switching location to the partial image B. , The images are joined together in the middle of the character "1". Here, FIG. 23A is a diagram for explaining a case where no deviation has occurred in the process of bonding the partial image A and the partial image B. FIG. 23A is a diagram for explaining a case where there is no shift in the process of bonding the partial images A and B, and FIG. FIG. 7 is a diagram for explaining a case where a displacement has occurred in the bonding process.
[0088]
FIG. 22A is a time chart for explaining an outline of an output state in the joining process, and FIG. 22B is an explanatory diagram for showing an image area. Hereinafter, the processing operation and the like in the OA camera of FIG. 21 will be described with reference to FIGS.
[0089]
First, a mirror scan is performed at the time of imaging, and as shown in FIG. 22B, scanning is performed as partial images A, B, C, and D. Therefore, the data of each partial image is as shown in FIG. {Circle around (1)} As shown in the output 141, the data is output in the order of A, B, C, D.
[0090]
For each of the partial images output in this way, after adjusting the mutual positional deviation, the position for switching the partial images at the time of joining is basically set at any position within the predetermined overlapping area. Can be joined.
[0091]
Therefore, in this example, as shown in FIG. 22B, assuming that the start point of the partial image B, the start point of the partial image C, and the start point of the partial image D are the switching positions of the partial images, The areas are Ax, Bx, Cx, and D. Therefore, as a time chart of the X output 142, (1) Ax, Bx, Cx, and D are output from the data A, B, C, and D output from the output 141, as shown in FIG. And the bonding of all the partial images is completed.
[0092]
However, in such a configuration, since the switching position or timing of the partial image is always fixed, if a character exists at the switching position of the partial image and the positional deviation correction is slightly deviated, As shown in FIG. 23B, there is a disadvantage that a shifted image is output. In other words, in the process of detecting a motion vector, calculating the amount of displacement between partial images, performing coordinate transformation, and returning the position of the displaced image to its original position, some kind of bonding error occurs, For example, when a shift of about one pixel occurs, the above-described problem is unavoidable, and particularly when the image is a character.
[0093]
In this example, in consideration of such a problem of the image handling apparatus of FIG. 21, in a case where partial images are pasted together, an image handling apparatus which does not cause a shift in a characteristic portion existing in an overlapping area of the partial images is provided. The purpose is to do.
[0094]
FIG. 11 is a block diagram illustrating a configuration of an OA camera as an example of the image handling apparatus, and FIG. 13 is a diagram illustrating a circuit configuration of a switching control unit 5 described later. The configuration of the OA camera of the present example will be described. The same components as those in FIG. 4 described above are denoted by the same reference numerals, and description thereof will be omitted. The main difference from the configuration shown in FIG. 4 is that, in the configuration shown in FIG. 11, the switching of the partial image to be used is performed in the form of the characteristic portion in the overlapping area in the joining process by bonding the partial images. That is, a switching control means 5 is provided for performing control based on the above. In the configuration shown in FIG. 11, the image data subjected to the coordinate conversion by the coordinate conversion unit 110 is binarized by the binarization unit 119, output via the output unit 114, and held in the boundary buffer 111. In addition, a difference is that a motion vector is detected based on the held binary data.
[0095]
In FIG. 13, portions marked with (1) and (2) correspond to portions marked with (1) and (2) as input portions to the switching control means 5 in FIG. It indicates that it is in the connected state. Referring to FIG. 15, a signal switching SW 121 switches between signals input from the (1) and (2) portions, ie, (1) data 125 and (2) data 126 (see FIG. 15). The SW control circuit 122 controls switching of the connection state of the SW 121 based on a signal from the AND circuit 123 and an input signal of the horizontal synchronization signal (HSYNC) 128 (see FIG. 15). And outputs a SW control signal 129 (see FIG. 15). The AND circuit 123 has its input side outputting a select signal (SELECT) 127 (see FIG. 15) that is always output to switch the signal of the partial image at the same timing, and the output side of the output section 114 (▲). 1 対 応).
[0096]
With the above configuration, the operation of this example will be described mainly with reference to FIG. FIG. 12A is a time chart for explaining an outline of an output state in the joining processing of this example, and FIG. 12B is an explanatory diagram for explaining an image area. In FIG. 9A, the (1) output 125 and the (2) output 126 (corresponding to the output of the data in the boundary buffer 111) correspond to the (1) data 125 and the (2) data 126 ( 15 (see FIG. 15), the same reference numerals are given, and the image data combined using these is designated as a Y output 124. Here, the main difference from FIG. 22 is that (2) the output 126 is utilized.
[0097]
First, (2) the output 126, that is, the output in the boundary buffer 111, is output at the right end of the partial image A in order to detect a motion vector with the partial image B when the partial image A is scanned once, for example. It is necessary to store Ar as image information in the boundary buffer 111 as binarized data in case the partial image B is scanned. Next, when the partial image B is scanned and comes out, the positional deviation between the partial image B and the right end Ar of the partial image A stored in the boundary buffer 111 is determined using the left end of the partial image B and the right end Ar of the partial image A, respectively. Since the configuration is such that a motion vector is detected, when the partial image B is output from the output unit 114, Ar is contained in the boundary buffer 111, and therefore {circle around (1)} in FIG. As shown by the output 125 and (2) output 126, Ar is output from the boundary buffer 111 at the timing when the partial image B is output from the output unit 114.
[0098]
Thereafter, the right end Br of the partial image B is sequentially output at the timing when the partial image C is output, and Cr is output at the timing when the partial image D appears.
[0099]
Here, the important point is that the output of the output unit 114 and the output of the boundary buffer 111 are skillfully switched by the SW 121 so that the image of the joint portion does not shift when pasting the partial images as in the related art. I have.
[0100]
That is, as shown in FIG. 12B, in short, in the overlapping area of the partial images A and B, the partial images are not switched at a uniform timing.
[0101]
Further, the timing and the like will be described with reference to FIG. 13, FIG. 15, and FIG. Here, FIG. 15 is a time chart for explaining an output state in the joining processing of the present example, and FIG. 13 is an explanatory diagram for explaining a circuit configuration of the switching control means 5. FIG. 16 is an explanatory diagram for explaining the timing of switching between the partial image A and the partial image B.
[0102]
In FIG. 15, an area of one partial image is captured by one mirror scan. For example, a timing at which an area of the partial image B is scanned and output will be described.
[0103]
The image signal is scanned in the horizontal direction, and for example, data of the partial image B is output as (1) data 125 at the time indicated by the falling timing 154 of the horizontal synchronization signal (HSYNC) 128. Here, (1) the signal waveform of the data 125 indicates "High" for a character and "Low" for a white background. The data indicated by (2) data 126 is the data of the boundary of the partial image A held in the boundary buffer 111, and partially has the same signal waveform as the (1) data 125.
[0104]
Next, the operation of the select signal 127 and the like will be described with reference to FIG.
[0105]
As shown in the drawing, the partial image 151 (corresponding to the partial image A in FIG. 12B) and the partial image 152 (corresponding to the partial image B in FIG. 12B) overlap in a predetermined area. Here, if the partial image 151 is uniformly switched from the partial image 151 to the partial image 152 at the time indicated by the falling timing 154 of the horizontal synchronizing signal (HSYNC) 128, the result is the same as the conventional case.
[0106]
Therefore, if the partial image 151 or the partial image 152 is output in the overlapping area 153 of the partial image, the falling timing 154 of the horizontal synchronization signal (HSYNC) 128 is basically the same. If it is assumed that a switching signal is generated at a time sufficiently earlier than the time at which the output of the partial image 151 ends and at the time indicated by the rising timing 155 of the select signal (SELECT) 127, basically, a correct paste Matching is performed.
[0107]
However, if the low pulse width of the select signal (SELECT) 127 synchronized with the horizontal synchronizing signal (HSYNC) 128 is fixed temporally in accordance with each H, as a result, a boundary line is always formed. If only this is used as the switching signal and used, the joint of the partial images is linearized by the switching for the above-described reason, and thus there is a possibility that a character shift or a level difference may occur.
[0108]
In the present embodiment, in order to avoid this, the following control is further performed. Returning to FIG. 15, the description will be continued.
[0109]
At the first rising timing of the horizontal synchronization signal (HSYNC) 128, the select signal (SELECT) 127 switches from low to high, and at the timing 129a of switching from the partial image A to the partial image B, (1) the data 125 is Low, that is, Since there is no "white" part without characters, no problem can occur at this timing even if switching is performed unconditionally.
[0110]
At the next timing 129b when the select signal (SELECT) 127 switches from low to high, the {circle around (1)} data 125 indicates High, that is, the presence of a character. For some reasons, there is a possibility that the joined images may be shifted.
[0111]
Therefore, it is necessary to wait until (1) the data 125 changes from High to Low, that is, until the timing of the timing 129c, and then to switch to the next partial image B.
[0112]
The switching operation of the partial images described above is realized by the switching control unit 5 shown in FIG. 13, so that a shift or a step does not occur. According to the circuit of the switching control means 5, as shown in FIG. 17, the switching location (in FIG. 17, the switching location is indicated by a broken line 201. In addition, all the images of "1" are included in the image data of the partial image A. Is reproduced along the right side of the character. Here, FIG. 17 is an explanatory diagram showing a switching location of image data when the circuit of the switching control means 5 having the configuration shown in FIG. 13 is used.
[0113]
In the above example, a simple character having a shape such as “1” is used. However, in the case of a complex character such as “2”, it is difficult to judge properly using the above method. Next, a corresponding example will be described.
[0114]
That is, in such a case, for example, a so-called projection process for each character is performed. Here, the projection process is a process of projecting a predetermined character in the X direction and the Y direction, and detecting the width thereof.
[0115]
For example, when the character is “2”, as shown in FIG. 18, a rectangular area circumscribing “2” (in FIG. 18, this circumscribed rectangular area 202 is represented by a chain line) is recognized and obtained as data. Can be The circumscribed rectangle data 203 obtained as described above is used as input data of the AND circuit 123 shown in FIG. Except for this, the configuration is the same as that of FIG. 13 described in the above example, and by performing the processing by the same method, it is possible to more accurately prevent the image from shifting. Here, FIG. 14 is an explanatory diagram for explaining another circuit configuration of the switching control means 5 in FIG. FIG. 18 is an explanatory diagram showing a switching location of image data when the circuit of the switching control means 5 having the configuration shown in FIG. 14 is used.
[0116]
That is, in the above configuration, by determining the switching timing, as shown in FIG. 18, the character is always switched along the circumscribed rectangular area 202, so that the character never crosses the character. Even if an error occurs, the error does not affect the joining state of images such as characters, so that it is possible to more accurately prevent misalignment as compared with the configuration shown in FIG.
[0117]
In this manner, for one image and another image that partially overlaps with the one image, a joined image forming unit for superimposing the one image and the other image and performing a bonding process When forming a joined image as a result of the bonding process, whether to use the one image for the overlapping image portion or to use the other image, An image handling apparatus including a switching control unit for performing switching control based on the form of an image can be realized.
[0118]
Therefore, according to the present example, it is possible to provide an image handling apparatus that does not cause a shift in the characteristic portion existing in the overlapping region of the partial images when the partial images are pasted together.
[0119]
In the above example, the case where the output data is binary data has been described. However, the present invention is not limited to this case.
[0120]
In addition, the present example may be realized by using the respective components in hardware, or is not limited thereto, and may be realized by software.
[0121]
【The invention's effect】
As is clear from the above description, the present inventionFor one image and another image having an area that partially overlaps with the one image, when performing the bonding process by aligning the two in the characteristic portion of the overlapping area,Without adding any special circuits,eachPartial imageDuplicateIn the areaPasteMatchingprocessingCan determine whether or not there is an image feature to use forWhen it is determined that the characteristic portion exists, the pasting process without displacement can be performed. When it is determined that the characteristic portion does not exist, a warning can be issued. Remarkable effect that it can be prevented from matchingHaving.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an OA camera as an example according to an image handling apparatus of the present embodiment.
FIG. 2 is a block diagram showing a configuration and the like of a cumulative addition unit according to the embodiment;
FIG. 3A is an explanatory diagram for explaining a configuration of a partial image according to the embodiment;
FIG. 3B is an explanatory diagram for explaining the cumulative addition area of the embodiment.
FIG. 4 is a block diagram showing a configuration of an OA camera as an example according to the image handling apparatus of the present embodiment.
FIG. 5 is a time chart for explaining the operation of the present example.
FIG. 6 is an explanatory diagram for explaining a place where auxiliary light is projected in FIG. 4;
FIG. 7 is a block diagram showing a configuration of an example in which an interpolation unit is added in FIG. 4;
FIG. 8 is a block diagram illustrating a configuration of a binarizing unit according to another example;
FIG. 9 is a block diagram showing a configuration of an OA camera as one embodiment of an image handling apparatus according to the present invention.
FIG. 10 (a) is an explanatory view showing a block of 64 pixels of 8 × 8 pixels based on the JPEG format.
FIG. 10B is an explanatory diagram for explaining DCT processing on blocked data.
FIG. 11 is a block diagram showing a configuration of an OA camera as an example according to the image handling apparatus of the present embodiment.
FIG. 12A is a time chart for explaining an outline of an output state in the joining processing of the present example.
FIG. 12B is an explanatory diagram for explaining an image area.
FIG. 13 is an explanatory diagram for explaining a circuit configuration of a switching control unit in FIG. 11;
FIG. 14 is an explanatory diagram for explaining another circuit configuration of the switching control means in FIG. 11;
FIG. 15 is a time chart for explaining an output state in the joining processing by the configuration of FIG. 11;
FIG. 16 is an explanatory diagram for explaining the timing of switching between the partial image A and the partial image B according to the configuration of FIG. 11;
FIG. 17 is an explanatory diagram showing a switching position of image data when the circuit of the switching control means having the configuration shown in FIG. 13 is used
FIG. 18 is an explanatory diagram showing a switching position of image data when the circuit of the switching control unit having the configuration shown in FIG. 14 is used;
FIG. 19 is a block diagram for explaining a configuration of an OA camera according to a proposal.
FIG. 20 is an explanatory diagram for explaining an overlapping area and the like of each partial image of the conventional example shown in FIG. 19;
FIG. 21 is a block diagram for explaining a configuration of another OA camera proposed previously.
FIG. 22A is a time chart for explaining an outline of an output state in a joining process of another conventional OA camera shown in FIG. 21;
FIG. 22 (b) is a diagram showing an image area for explaining an outline of an output state in the joining processing of another conventional OA camera shown in FIG.
FIG. 23 (a) is a view for explaining a case where there is no shift in the process of bonding the partial images A and B in another conventional OA camera shown in FIG. 21;
FIG. 23 (b) is a view for explaining a case where a shift has occurred in the process of bonding the partial image A and the partial image B in another conventional OA camera shown in FIG.
[Explanation of symbols]
1 Joint information output terminal
2 CPU (determination means)
81 DCT processing unit
101 Total reflection mirror
102 Incident light
102a Light passing through
102b reflected light
103 lens section
104 half mirror
105 1st CCD
106 Second CCD
110 coordinate conversion unit (joining image forming means)
200 Image to be captured

Claims (6)

As an image, the other image having a region that overlaps the first image and the partial image handling apparatus having the image synthesizing function of performing combined and bonded processing position both in features of the area to the overlap At
Orthogonal transformation means for obtaining each orthogonal transformation data from an image signal representing the overlapping region on the one image side and / or an image signal representing the overlapping region on the other image side ;
A contrast data extraction unit for obtaining contrast data based on the obtained orthogonal transform data,
Whether a feature necessary for superimposing the one image and the other image exists in the overlapping area on the one image side and / or the overlapping area on the other image side . Determining means for determining by comparing the obtained contrast data with a predetermined contrast value ,
An image synthesis control unit that controls the combining process based on the determination result by the determination unit ;
Image handling apparatus comprising the. The image synthesis control unit performs the bonding process when the determination unit determines that the characteristic portion exists, and cannot perform the bonding process when it is determined that the characteristic unit does not exist. The image handling apparatus according to claim 1, wherein a warning to the effect is issued. The orthogonal transform means is a DCT processing unit that obtains DCT coefficients as orthogonal transform data,
The image handling apparatus according to claim 1, wherein the contrast data extraction unit further includes a cumulative addition unit that cumulatively adds only a necessary part from the obtained DCT coefficients. An image handling apparatus having an image combining function of performing alignment processing of one image and another image having an area partially overlapping with the one image by aligning the two with a characteristic portion of the overlapped area In the image synthesis method of
An orthogonal transformation step for obtaining each orthogonal transformation data from an image signal representing the overlapping region of the one image and / or an image signal representing the overlapping region of the other image;
A contrast data extraction step of obtaining contrast data based on the obtained orthogonal transformation data;
Whether a feature necessary for superimposing the one image and the other image exists in the overlapping area on the one image side and / or the overlapping area on the other image side . A determination step for determining by comparing the obtained contrast data with a predetermined contrast value ,
An image combining control method for controlling the combining process based on the result of the determination by the determining step . The image combining control step performs the bonding process when the characteristic portion is determined to be present by the determining step, and when the characteristic portion is determined not to be present, bonding is not possible. 5. The method according to claim 4, wherein a warning to the effect is issued. The orthogonal transformation step is a DCT processing step of obtaining DCT coefficients as orthogonal transformation data,
6. The image combining method according to claim 4, wherein the contrast data extracting step further includes a cumulative adding step of cumulatively adding only a necessary portion from the obtained DCT coefficients.

Images