JP5278010B2 - Image capturing apparatus, portable terminal, image composition method used therefor, and program thereof - Google Patents

Description

  The present invention relates to a photographing apparatus, a portable terminal, an image composition method used therefor, and a program thereof, and more particularly to an image composition method in a photographing method in which a shutter delay occurs.

  In recent years, various methods have been adopted for releasing a shutter in a mobile phone or a digital camera, such as releasing the shutter at the timing of smile, or releasing the shutter when the distance between persons is reduced (for example, Patent Documents 1 and 2). reference).

  In the shooting method related to the present invention, the size of the display unit is adjusted as a preview display image when shooting a large image, an output image having a size smaller than the acquired image is used, and the output image is switched to a larger output image only when the captured image is acquired. There are many.

  For this reason, at the time of shooting, since a shot image is acquired after the output size is switched after the shutter is pressed, a delay (shutter delay) occurs between when the shutter is pressed and when the image is actually acquired.

JP 2006-345254 A JP-A-2005-341016

  In the above-described photographing method, since a shutter delay occurs, an object that is not displayed in the preview image may appear in the photographed image at the moment when the shutter is pressed, which may be different from the image expected by the photographer.

  In other words, in actual shooting, there is a delay (shutter delay) from the determination of pressing the shutter to the actual image acquisition, so this is different from the expected shot image in shooting in a moving state. That image may be acquired.

  Accordingly, an object of the present invention is to solve the above-described problems, and in an imaging method in which a shutter delay occurs, an imaging device, a portable terminal, and those capable of synthesizing and acquiring an image of the moment that the photographer wants to acquire It is an object to provide an image composition method and a program therefor used in the above.

The photographing apparatus according to the present invention
Acquisition means for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
Creating means for creating a reduced image for comparison by performing a reduction process so that the size of the preview image that is smaller than the shooting size is output with respect to the image for synthesis acquired by the acquisition unit;
Said plurality of reduced images each for comparison created by the creation means compared to the preview image, and labeling means for labeling a large part of the difference in the comparison result a small portion of the NG region Toshikatsu the difference as OK region ,
And indexing means for indexing the OK region and the OK region and the NG region of the synthesis image of the NG region by mapping the obtained synthesized image by the acquisition unit,
And generating means for generating a storage image by combining the OK regions in the plurality of combining images based on the calculated result.

  The portable terminal by this invention is characterized by including said imaging device.

An image composition method according to the present invention includes:
The shooting device
An acquisition process for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
A creation process for creating a reduced image for comparison by performing a reduction process to the same size as the preview image output smaller than the shooting size with respect to the composition image acquired in the acquisition process;
A labeling process for comparing each of the plurality of comparative reduced images created in the creation process with the preview image, and labeling a portion having a large difference as an NG region and a portion having a small difference as an OK region in the comparison result ; ,
And indexing process to index the OK region and the OK region and the NG region of the synthesis image of the NG region by mapping the synthesized image obtained by the obtaining process,
Generation processing for generating a storage image by combining the OK regions in the plurality of combining images based on the calculated result is executed.

A program according to the present invention is a program to be executed by a central processing unit in a photographing apparatus,
An acquisition process for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
A creation process for creating a reduced image for comparison by performing a reduction process to the same size as the preview image output smaller than the shooting size with respect to the composition image acquired in the acquisition process;
A labeling process for comparing each of the plurality of comparative reduced images created in the creation process with the preview image, and labeling a portion having a large difference as an NG region and a portion having a small difference as an OK region in the comparison result ; ,
And indexing process to index the OK region and the OK region and the NG region of the synthesis image of the NG region by mapping the synthesized image obtained by the obtaining process,
And generating processing for generating a storage image by combining the OK regions in the plurality of combining images based on the calculated result.

  By adopting the configuration and operation as described above, the present invention can obtain an effect that a photographer can synthesize and acquire an image at a moment when the photographer wants to acquire even in a shooting method in which a shutter delay occurs.

It is a block diagram which shows the structural example of the imaging device by this invention. It is a block diagram which shows the structural example of the imaging device by the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the image process part of FIG. It is a figure which shows the flow of the image processing by the 1st Embodiment of this invention. It is a flowchart which shows the flow of the imaging | photography process by the 1st Embodiment of this invention. It is a flowchart which shows the flow of the imaging | photography process by the 1st Embodiment of this invention. It is a figure which shows the flow of the image processing by the 2nd Embodiment of this invention. It is a flowchart which shows the flow of the imaging | photography process by the 2nd Embodiment of this invention. It is a flowchart which shows the flow of the imaging | photography process by the 2nd Embodiment of this invention.

  Next, embodiments of the present invention will be described with reference to the drawings. First, the outline of the photographing apparatus of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a photographing apparatus according to the present invention. In FIG. 1, a photographing apparatus 1 according to the present invention uses an output image having a size smaller than an acquired image as an image for preview display when photographing a large size image, and an output image having a large size only when a captured image is obtained. This is a device that performs switching photography to switch to, and is used for a mobile phone, a digital camera, and the like.

  The image capturing apparatus 1 according to the present invention acquires a plurality of images by the composite image acquisition unit 111 of the image processing unit 11 at the time of image acquisition, and the shutter button is pressed by the storage image generation unit 112 in the switching shooting described above. By generating an image close to the preview image displayed at the moment when the image is displayed, a captured image with less influence of the shutter delay is obtained.

  FIG. 2 is a block diagram showing a configuration example of the photographing apparatus according to the first embodiment of the present invention. In FIG. 2, the photographing apparatus 1 includes an image processing unit 11, a CPU (central processing unit) 12, a camera sensor 13, a processing memory 14, a memory unit 15, a key operation unit 16, and a display unit 17. It has.

  The key operation unit 16 is a circuit through which a photographer inputs operations such as camera activation, shutter pressing, storage, and cancellation. The camera sensor 13 is a circuit that acquires an image and an image for synthesis, and transfers the acquired image to the image processing unit 11.

  The image processing unit 11 stores in the processing memory 14 the preview image when the shutter is pressed and the composition image acquired after switching the output image size of the camera. In addition, the image processing unit 11 reduces the composition image to generate a comparison reduced image, and compares the comparison reduced image with the preview image stored in the processing memory 14.

  In the comparison result, the image processing unit 11 labels a portion having a large difference as an NG region and a region having a small difference as an OK region, and maps the OK / NG region labeled with the comparison reduced image to the acquired image for synthesis. Then, the OK areas of the plurality of images for synthesis are synthesized to generate one image for storage.

  The memory unit 15 holds the storage image generated by the image processing unit 11. The memory unit 15 stores a program executed by the CPU 12 and realizing the processing of the image processing unit 11. The display unit 17 performs preview display at the time of shooting and confirmation display of an image for storage.

  The CPU 12 obtains an image and transfers data to the camera sensor 13 by an operation input from the key operation unit 16, an image processing command and a data transfer command to the image processing unit 11, and a data transfer and storage process to the memory unit 15. Alternatively, the display unit 17 performs control so that the image from the image processing unit 11 or the memory unit 15 is displayed.

  FIG. 3 is a block diagram illustrating a configuration example of the image processing unit 11 of FIG. In FIG. 3, the image processing unit 11 includes a compositing image acquisition unit 111, a storage image generation unit 112, a comparative reduced image creation unit 113, a labeling unit 114, and an OK / NG area indexing unit 115. ing.

  The composite image acquisition unit 111 acquires a plurality of composite images having the same size as the shooting size from the camera sensor 13 when the shutter is pressed in the switching shooting described above. The comparison reduced image creation unit 113 performs a reduction process so that the synthesis image acquired by the synthesis image acquisition unit 111 becomes the same size as the preview image output smaller than the shooting size, and creates a comparison reduced image. To do.

  The labeling unit 114 compares the comparison reduced image created by the comparison reduced image creation unit 113 with a preview image at the moment when a shutter button (not shown) is pressed, and sets a portion having a large difference as an NG region. Other portions are labeled as OK regions.

  The OK / NG area indexing unit 115 maps the OK / NG area calculated with the reduced image for comparison to the synthesis image, thereby determining the OK / NG area of the synthesis image. The storage image generation unit 112 generates one storage image by combining the OK regions in the plurality of composite images determined by the OK / NG region determination unit 115.

  FIG. 4 is a diagram showing the flow of image processing according to the first embodiment of the present invention. In FIG. 4, in the present embodiment, in switching shooting, a plurality of compositing images 202 to 204 having the same size as the shooting size are acquired when the shutter is pressed, and the size is the same as the preview image 201 output smaller than the shooting size. The reduction process is performed as described above, and comparison reduced images 205 to 207 are created.

  In the present embodiment, the preview image 201 at the moment when the shutter button is pressed is compared with the reduced images for comparison 205 to 207, and the locations with large differences are labeled as NG regions 215 to 217, and the other locations are labeled as OK regions. To do. In this embodiment, the OK / NG area calculated by the comparison reduced image is mapped to the synthesis images 202 to 204 to determine the OK / NG area of the synthesis image (208 to 210), and a plurality of synthesis images are used. By combining the OK areas in the image (211 to 213), a single storage image 214 is generated.

  5 and 6 are flowcharts showing the flow of the photographing process according to the first embodiment of the present invention. The photographing process according to the first embodiment of the present invention will be described with reference to FIGS. The processing shown in FIGS. 5 and 6 is realized by the CPU 12 executing the program stored in the memory unit 15.

  In this photographing process, the number of images for composition acquired when the shutter button is pressed is arbitrary, but in this embodiment, it is N (N is an integer of 2 or more).

  When the camera activation is input from the key operation unit 16 (step S1 in FIG. 5), the CPU 12 of the photographing apparatus 1 activates and initializes the camera sensor 13 and the image processing unit 11 (step S2 in FIG. 5). The CPU 12 acquires an image from the camera sensor 13 and starts preview display of the image on the display unit 17 (step S3 in FIG. 5).

  When the shutter pressing is input from the key operation unit 16 (step S4 in FIG. 5), the CPU 12 causes the image processing unit 11 to accumulate the preview image 201 displayed when the shutter is pressed in the processing memory 14 (FIG. 5). 5 step S6).

  Thereafter, the CPU 12 changes the output size of the image of the camera sensor 13 to the shooting size as the shooting mode switching process (step S7 in FIG. 5). The CPU 12 causes the image processing unit 11 (compositing image acquisition unit 111) to store the images output from the camera sensor 13 in the processing memory 14 as the combining images 202 to 204 (step S9 in FIG. 5).

  Further, the CPU 12 causes the image processing unit 11 (comparison reduced image creation unit 113) to perform a reduction process on the composite image (step S10 in FIG. 5), and the comparison reduced image having the same size as the preview image 201. 205 to 207 are acquired (step S11 in FIG. 5).

  The CPU 12 causes the image processing unit 11 (labeling unit 114) to compare the reduced images for comparison 205 to 207 with the preview image 101, and sets the regions having large differences from the preview image 201 as NG regions 215 to 217 and the preview image 201. An area having a small difference from the above is obtained as an OK area (step S12 in FIG. 5).

  The CPU 12 causes the image processing unit 11 (OK / NG area determination unit 115) to map the result of the labeling unit 114 onto the synthesis images 202 to 204 (step S13 in FIG. 6). The CPU 12 causes the image processing unit 11 to perform this process on N images for synthesis (the processes in steps S8 to S14).

  The CPU 12 causes the image processing unit 11 (storage image generation unit 112) to perform synthesis by interpolating the NG area with the OK area from the N synthesis images 202 to 204 (step S15 in FIG. 6). A work image 214 is generated (step S16 in FIG. 6).

  After acquiring the storage image 214, the CPU 12 causes the display unit 17 to confirm and display the storage image 214 (step S17 in FIG. 6), and executes a storage process or cancellation input from the key operation unit 16 (FIG. 6). Step S19). The CPU 12 continues the shooting input in the key operation unit 16 (step S20 in FIG. 6) or ends the camera (step S21 in FIG. 6).

  As an example of an implementation method of the comparison between the preview image 201 and the reduced images for comparison 205 to 207 (step S12), an area where the similarity is equal to or less than an arbitrary threshold value in block matching or template matching using the preview image 201 as a template. It can be determined as an NG area which is an area having a large difference from the preview image.

  As the interpolation method for the NG area (step S15), after the OK / NG area is determined on the first sheet, the OK / NG determination is performed only on the area that has become NG on the first sheet from the second sheet. The first image is merged, the third image is a composite image of the first and second images, and OK / NG is determined only for the area still remaining as NG. This is done by constructing the image.

  As described above, in the present embodiment, by combining a storage image from a plurality of acquired images using the final preview image as a template, only unnecessary objects are excluded when an unnecessary object is captured instantaneously. Since the obtained image can be acquired, it can be expected to reduce re-shooting in such a scene.

  In the present embodiment, since the captured image is synthesized in accordance with the final preview image, the gap between the photographer's expected image and the actual acquired image due to the shutter delay can be reduced.

  Furthermore, in this embodiment, since it can be realized by software processing in the image processing unit 11, it can be realized by an image processing chip provided in a general digital camera or a portable terminal device (such as a mobile phone). is there.

  FIG. 7 is a diagram showing the flow of image processing according to the second embodiment of the present invention. Since the imaging device according to the present embodiment has the same configuration as the imaging device 1 according to the first embodiment of the present invention shown in FIG. 2, the illustration and description of the configuration are omitted.

  In FIG. 7, in this embodiment, after mapping the OK / NG area to each image for reduction (301 to 303), the composite area is calculated with the image for reduction regarding the interpolation of the NG area (304 to 306). The synthesized image is applied to the processing image (307 to 309), and the storage image (310) is synthesized. The flow of image processing at that time is as shown in FIGS.

  That is, FIG. 8 and FIG. 9 are flowcharts showing the flow of the photographing process according to the second embodiment of the present invention. The photographing process according to the second embodiment of the present invention will be described with reference to FIGS. 2, 3, and 7 to 9. The processing shown in FIGS. 8 and 9 is realized by the CPU 12 executing a program stored in the memory unit 15.

  In this photographing process, the number of images for composition acquired when the shutter button is pressed is arbitrary, but in this embodiment, it is N (N is an integer of 2 or more).

  When the camera activation is input from the key operation unit 16 (step S31 in FIG. 8), the CPU 12 of the photographing apparatus 1 activates and initializes the camera sensor 13 and the image processing unit 11 (step S32 in FIG. 8). The CPU 12 acquires an image from the camera sensor 13 and starts preview display of the image on the display unit 17 (step S33 in FIG. 8).

  When the shutter pressing is input from the key operation unit 16 (step S34 in FIG. 8), the CPU 12 causes the image processing unit 11 to accumulate the preview image 201 displayed when the shutter is pressed in the processing memory 14 (FIG. 8). 8 step S36).

  Thereafter, the CPU 12 changes the output size of the image of the camera sensor 13 to the shooting size as the shooting mode switching process (step S37 in FIG. 8). The CPU 12 causes the image processing unit 11 (composition image acquisition unit 111) to accumulate the images output from the camera sensor 13 in the processing memory 14 as composition images 202 to 204 (step S39 in FIG. 8).

  Further, the CPU 12 causes the image processing unit 11 (comparison reduced image creation unit 113) to perform a reduction process on the compositing images 202 to 204 (step S40 in FIG. 8), and makes the comparison image the same size as the preview image. Reduced images 205 to 207 are acquired (step S41 in FIG. 8).

  The CPU 12 causes the image processing unit 11 (labeling unit 114) to compare the comparison reduced images 205 to 207 with the preview image 201, and select an area with a large difference from the preview image 201 as the NG areas 215 to 217 and the preview image 201. An area having a small difference from the above is obtained as an OK area (step S42 in FIG. 8). The CPU 12 causes the image processing unit 11 to perform this process on the N images for composition (the processes in steps S38 to S43).

  The CPU 12 causes the image processing unit 11 (OK / NG region determining unit 115) to calculate a combined region in the reduced image (step S44 in FIG. 9) (301 to 303, 304 to 306 in FIG. 7), and combines the images. The image is applied to the processing image (step S45 in FIG. 9) (307 to 309 in FIG. 7).

  The CPU 12 causes the image processing unit 11 (storage image generation unit 112) to generate the storage image 310 by applying the composite image to the processing image (step S46 in FIG. 9).

  After acquiring the storage image 310, the CPU 12 causes the display unit 17 to confirm and display the storage image 310 (step S47 in FIG. 9), and executes a storage process or cancellation input from the key operation unit 16 (FIG. 9). Steps S48 and S49). The CPU 12 performs the photographing operation input through the key operation unit 16 (step S50 in FIG. 9) or ends the camera (step S51 in FIG. 9).

In the first embodiment of the present invention described above, the matching process is performed, an area having a similarity equal to or higher than the threshold is set as an OK area, and the OK area is sequentially added from the first sheet. in the second embodiment of the invention, separate the comparison reduced image and the preview image into blocks (e.g., divided into regions of 8 × 8), compared with the processing image in each block, the highest similarity as Are extracted and synthesized in block units.

As described above, in the present embodiment , the preview image and the reduced image for comparison are divided into blocks, compared with the processing image in each block, and the one with the highest similarity is extracted and combined in units of blocks. As in the first embodiment of the present invention, when an unnecessary object is captured instantaneously, an image excluding only the unnecessary object can be acquired. It can be expected to reduce reshooting.

1 Shooting device
11 Image processing unit
12 CPU
13 Camera sensor
14 Processing memory
15 Memory part
16 Key operation section
17 Display
111 Image acquisition unit for composition
112 Image generation unit for storage
113 Reduced image creation unit for comparison
114 Labeling section
115 OK / NG area index
201 Preview image 202-204 Composite image 205-207 Reduced image for comparison 214, 310 Storage image 215-217 NG area

Claims (12)

Acquisition means for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
Creating means for creating a reduced image for comparison by performing a reduction process so that the size of the preview image that is smaller than the shooting size is output with respect to the image for synthesis acquired by the acquisition unit;
Labeling means for comparing each of the plurality of reduced images for comparison created by the creating means with the preview image, and labeling a portion having a large difference as an NG region and a portion having a small difference as an OK region in the comparison result; ,
Indexing means for determining the OK area and the NG area of the image for synthesis by mapping the OK area and the NG area to the image for synthesis acquired by the acquisition means;
Generation means for generating an image for storage by combining the OK regions in the plurality of images for combining based on the determined result;
A photographing apparatus comprising: The photographing apparatus according to claim 1, wherein the preview image in the labeling unit is an image at a moment when the shutter is pressed. 2. The labeling unit according to claim 1, wherein a part having a large difference in the comparison result between the reduced image for comparison and the preview image is set as the NG area, and the other part is set as the OK area. 2. The photographing apparatus according to 2. The generation unit sequentially adds, from the first sheet, an OK region that is a region having a similarity equal to or higher than a threshold in the result of the matching process used for comparing the preview image and the reduced image for comparison in the labeling unit. The photographing apparatus according to claim 1, wherein the image for storage is generated. The preview image and the reduced image for comparison are divided into blocks, compared with the processing image in each block, and the image having the highest similarity is extracted and synthesized in units of blocks to generate the storage image. The imaging device according to any one of claims 1 to 3, wherein the imaging device is characterized. A portable terminal comprising the photographing apparatus according to claim 1. The shooting device
An acquisition process for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
A creation process for creating a reduced image for comparison by performing a reduction process to the same size as the preview image output smaller than the shooting size with respect to the composition image acquired in the acquisition process;
A labeling process for comparing each of the plurality of comparative reduced images created in the creation process with the preview image, and labeling a portion having a large difference as an NG region and a portion having a small difference as an OK region in the comparison result; ,
An indexing process for determining the OK area and the NG area of the image for synthesis by mapping the OK area and the NG area to the image for synthesis acquired in the acquisition process;
A generating process for generating a storage image by combining the OK regions in the plurality of combining images based on the determined results;
A method of synthesizing images. 8. The image composition method according to claim 7, wherein the preview image in the labeling process is an image at a moment when the shutter is pressed. 8. The labeling process according to claim 7, wherein a part having a large difference in the comparison result between the reduced image for comparison and the preview image is set as the NG area, and the other part is set as the OK area. 9. The image composition method according to 8. In the generation process, by sequentially adding, from the first sheet, an OK area whose similarity is equal to or greater than a threshold in the result of the matching process used to compare the preview image and the reduced image for comparison in the labeling process. The image composition method according to claim 7, wherein the image for storage is generated. The preview image and the reduced image for comparison are divided into blocks, compared with the processing image in each block, and the image having the highest similarity is extracted and synthesized in units of blocks to generate the storage image. 10. The image composition method according to claim 7, wherein the image composition method is any one of claims 7 to 9. A program to be executed by a central processing unit in a photographing device,
An acquisition process for acquiring a plurality of compositing images having the same size as the shooting size when the shutter is pressed;
A creation process for creating a reduced image for comparison by performing a reduction process to the same size as the preview image output smaller than the shooting size with respect to the composition image acquired in the acquisition process;
A labeling process for comparing each of the plurality of comparative reduced images created in the creation process with the preview image, and labeling a portion having a large difference as an NG region and a portion having a small difference as an OK region in the comparison result; ,
An indexing process for determining the OK area and the NG area of the image for synthesis by mapping the OK area and the NG area to the image for synthesis acquired in the acquisition process;
Generation processing for generating a storage image by combining the OK regions in the plurality of combining images based on the determined result;
The program characterized by including.

Images