JP4291182B2 - Electronic device with camera and photographing method - Google Patents

Description

  The present invention relates to an electronic device with a camera and a photographing method, and more particularly, to an electronic device with a camera and a photographing method for preventing whiteout of a photographed image due to overexposure by a light during close-up photographing.

  For example, in a portable electronic device such as a mobile phone (hereinafter, also simply referred to as an electronic device), a camera-equipped type has become widespread. According to such a camera-equipped mobile phone, in addition to transmitting and receiving audio, which is the original function of the phone, it is possible to transmit a photographed image by the camera, so that the information transmission capability can be dramatically improved. . In such camera-equipped mobile phones, most types of mobile phones are provided with a display unit for displaying character information or image information and an information input key group except when transmitting and receiving to improve portability. The operation unit is configured to be foldable together.

  In photographing with a camera including a mobile phone with a camera as described above, it is generally important to optimally adjust exposure. For example, Patent Document 1 discloses a digital camera configured to obtain optimal exposure in a short time. Has been. In addition, when a close-up photographing is performed with a camera-equipped mobile phone in a state where the brightness of the surroundings of the subject is insufficient, there is a device equipped with a light as a device for compensating for insufficient exposure. According to the camera-equipped mobile phone having such a configuration, it is possible to solve the shortage of exposure by turning on the light during close-up photography. However, in actual photographing, when the light is flashed during close-up photography, the light is often overexposed due to excessive light, so that the photographed image becomes white, so-called whiteout occurs.

  Here, dedicated cameras such as single-lens reflex cameras have an AF (Auto-Focus) function, so the distance from the camera to the subject (subject distance) is automatically measured during shooting. Since the optimum exposure amount is always adjusted based on the subject distance and shooting is performed, whiteout as described above does not occur. However, a camera provided in a camera-equipped mobile phone is generally composed of a simple single-focus camera due to cost restrictions and the light is also an LED (Light Emitting Diode) or the like, and the amount of light is at most. Only 3 steps of off, weak lighting and strong lighting can be adjusted. Accordingly, since the camera-equipped mobile phone does not have an AF function, the subject distance cannot be recognized, and as described above, it is impossible to prevent the occurrence of white-out of a photographed image due to flash of light during close-up photography.

In order to prevent red-eye during shooting, for example, Patent Document 2 discloses a digital camera configured to capture a subject a plurality of times and acquire an image under appropriate shooting conditions without missing a photo opportunity. . As shown in FIG. 4, the digital camera includes an imaging unit 200 that acquires a first main image of a subject, a determination unit 120 that determines whether or not the first main image satisfies a predetermined condition, When the determination unit 120 determines that the first main image does not satisfy the predetermined condition, the imaging unit 200 includes an imaging control unit 400 that causes the imaging unit 200 to acquire the second main image, and a strobe 360.
Here, when the determination unit 120 determines that the first main image does not satisfy the predetermined condition, the imaging control unit 400 causes the imaging unit 200 to acquire the second main image. Specifically, the determination unit 120 determines whether or not the spatial frequency of the first main image is equal to or higher than a predetermined value, or determines whether or not the brightness of the first main image satisfies a predetermined condition. to decide. When the spatial frequency of the first main image is not equal to or higher than the predetermined value, or when the brightness of the first main image does not satisfy the predetermined condition, the imaging control unit 400 displays the second main image on the imaging unit 200. Get it. The imaging unit 200 may acquire a plurality of continuous pre-flash images in a state where the strobe 360 emits light, and the imaging control unit 400 acquires the main image based on the plurality of pre-flash images. The focus position may be determined.

Further, for example, Patent Document 3 discloses a camera configured so that more appropriate shooting can be performed without bothering the photographer regardless of the state of the subject. The camera determines whether or not the first shooting is overexposed when the flash device flashes the subject during the first shooting, and determines that the first shooting is overexposed. In this case, the second photographing is automatically performed. According to such a configuration, it is not necessary for the photographer himself to determine whether or not the first shooting is overexposure. If the CPU (Central Processing Unit) determines that the exposure is overexposure, the photographer immediately starts again. Since photographing is performed, the second photographing is performed without changing the composition of the photographing, so that a photo opportunity can be utilized.
JP 2002-232771 A Japanese Patent Laid-Open No. 2003-234953 JP 2003-270698 A

Incidentally, the conventional cameras disclosed in Patent Documents 2 and 3 have the following problems.
First, the digital camera described in Patent Document 2 captures a subject a plurality of times and acquires an image under an appropriate photographing condition without missing a photo opportunity for the purpose of preventing red eyes during photographing. . Next, the camera described in Patent Document 3 automatically performs the second shooting when the first shooting is overexposed for the purpose of performing more appropriate shooting without bothering the photographer. Like to do.

However, in any of the cameras described in Patent Documents 2 and 3, it is common in that an image is acquired without missing a photo opportunity by shooting an object a plurality of times and setting appropriate shooting conditions. It is not possible to efficiently prevent whiteout of a photographed image due to a flash of light during close-up photography, which is a weak point of a camera-equipped mobile phone.
In particular, as in the camera described in Patent Document 3, when the first shooting is performed and it is determined that the first shooting is overexposed based on the image information, the second shooting is performed and the second shooting is performed. It is too efficient to perform a series of processes of performing the process.

  The present invention has been made in view of the above circumstances, and is capable of efficiently preventing the occurrence of whiteout of a photographed image due to flash of light during close-up photography without providing an autofocus function. It is an object to provide a mobile phone with a camera and a photographing method.

In order to solve the above problems, the invention described in claim 1 is provided with a light without an autofocus function, and the camera unit always outputs an optimum image based on an input image signal. It is configured to adjust the exposure, and even when the light is turned on and whites out in the first frame, it is configured to have an exposure adjustment value that matches the lighting state of the light in the next frame. In a camera-equipped electronic device, in close-up photography with the camera, a working memory that temporarily stores two images taken with the light turned on in two frame periods, and a working memory A stored image memory for storing one of the two stored images as a stored image; and the two images stored in the working memory from the camera. An image calculation unit that integrates the magnitudes of the luminance information included in the images and outputs the first integrated value of the first image and the second integrated value of the second image to the working memory; The first integrated value and the second integrated value are compared, and if the absolute value of the difference between the two integrated values is smaller than a preset threshold value, the first image is read from the working memory. When the absolute value of the difference between the two integrated values is greater than or equal to the threshold value, the second image is transferred from the working memory to the saved image memory. And a central processing unit that controls to transfer and store the data

  According to a second aspect of the present invention, there is provided the electronic device with camera according to the first aspect, wherein the work memory is the two images taken in a state where the light is turned on during the two consecutive frame periods. These images are temporarily stored.

  According to a third aspect of the invention, there is provided the camera-equipped electronic device according to the second aspect, wherein the image calculation unit integrates the magnitudes of the luminance information respectively included in the two consecutive images. It is a feature.

  According to a fourth aspect of the present invention, there is provided the camera-equipped electronic device according to the second or third aspect, wherein the central processing unit includes the first integrated value and the first integrated value of each of the two consecutive images. The integrated value of 2 is compared.

The invention according to claim 5 is provided with a light without an autofocus function, and the camera unit performs exposure adjustment so that an optimum image is always output based on an input image signal. Even when the light is turned on and whitened out in the first frame, an electronic device with a camera that is configured to have an exposure adjustment value that matches the light lighting state in the next frame is used. According to a photographing method, during close-up photographing by the camera, a step of taking a first image by exposing the light in a first frame period and a second image after the first frame period Exposing the light in the frame period to capture a second image and simultaneously outputting image data of the first image; and the second frame period Outputting the second image data in a third frame period after falling; storing the image data of the first image and the second image in a working memory over time; and The magnitudes of luminance information included in the first image and the second image are integrated, and the first integrated value of the first image and the second integrated value of the second image are obtained. The step of outputting to the working memory over time, the step of comparing the first integrated value and the second integrated value, and the absolute value of the difference between the two integrated values as a result of the comparison in advance. When the threshold value is smaller than the set threshold, the absolute value of the difference between the two integrated values is determined in advance as a result of the step of transferring and storing the first image from the working memory to the storage image memory. If it is greater than or equal to the set threshold, the second image is It is characterized in that the working memory and a step of storing is transferred to the storage image memory.

  According to a sixth aspect of the present invention, there is provided a photographing method using the camera-equipped electronic device according to the fifth aspect, wherein the first to third frame periods are continuous.

  According to a seventh aspect of the present invention, there is provided a photographing method using the camera-equipped electronic device according to the fifth aspect, wherein a luminance signal is used as the luminance information.

According to the camera-equipped electronic device and the photographing method of the present invention, even if the autofocus function is not provided, two images are continuously photographed at different photographing timings, and the influence of the influence of the light is adjusted. To avoid whiteout. In other words, two images are taken each time, and one of the images with the correct exposure is selected and used as a saved image.
Therefore, even if the autofocus function is not provided, it is possible to efficiently prevent the occurrence of whiteout of a captured image caused by the flash of light during close-up shooting.

  In an electronic device with a camera that does not have an autofocus function and is equipped with a light that is turned on during close-up photography, two images taken with the light turned on during two frame periods during close-up photography with the camera A working memory for temporarily storing, a saved image memory for storing one of the two images stored in the working memory as a saved image, and two sheets stored in the working memory from the camera An image calculation unit that integrates the size of the luminance information included in each image and outputs the first integrated value of the first image and the second integrated value of the second image to the working memory; When the first integrated value is compared with the second integrated value and the absolute value of the difference between the two integrated values is smaller than a preset threshold value, the first image is transferred from the working memory to the saved image memory. Forward to case On the other hand, if the absolute value of the difference between the two integrated values is greater than or equal to the threshold value, a central processing unit that controls to transfer and store the second image from the working memory to the saved image memory; It has.

FIG. 1 is a block diagram showing the configuration of an electronic device with a camera according to an embodiment of the present invention, FIG. 2 is a timing chart showing the operation of the electronic device with the camera, and FIG. 3 is a photographing method using the electronic device with the camera. It is a flowchart which shows. In this example, a mobile phone will be described as an example of the electronic device.
As shown in FIG. 1, a camera-equipped mobile phone 10 in this example includes a CPU (Central Processing Unit) 1 that performs overall control operations and a ROM (Read that stores various control programs to be executed by the CPU 1. Program memory 2 composed of a non-volatile memory such as “Only Memory”, and two consecutively photographed images before being selected as saved images, which are data temporarily required for the CPU 1 to execute the program A work memory 3 composed of a volatile memory such as a RAM (Random Access Memory) for storing an image, and a nonvolatile memory such as an EEP (Electrically Erasable and Programmable) for storing a selected image among continuously shot images as a stored image And a stored image memory 4 comprising a memory.

  The working memory 3 temporarily stores two images A and B taken in the state in which the lights 12 are turned on during two frame periods, as will be described later, during close-up photography by the camera unit 5. As will be described later, the saved image memory 4 stores one of the two images A and B stored in the work memory 3 as a saved image. The light 12 can be turned on at times other than close-up photography.

  As shown in FIG. 1, the camera-equipped mobile phone 10 includes an artificial retina LSI (Large Scale Integration) such as a CMOS (Complementary Metal Oxide Semiconductor) and a CCD (Charge Coupled Device), a lens, and a signal processing ASIC (Application Specific). An electronic camera module comprising an integrated circuit), a camera unit 5 for photographing a subject and outputting a photographed image signal in YUV format, a camera control unit 6 for controlling the operation of the camera unit 5, and an LCD (Liquid Crystal Device) : A liquid crystal display device), a display device such as an organic EL (Electroluminescence), display the input image from the camera unit 5 and use it as a finder, and control the operation of the display unit 7 Display control unit 8, a key unit 9 in which a plurality of information input keys are arranged, and a key having an input circuit for receiving key inputs from the key unit 9. And an input unit 11. Here, the signal processing ASIC is configured to automatically perform exposure adjustment and white balance adjustment so that an optimum subject is displayed.

  Further, as shown in FIG. 1, the camera-equipped mobile phone 10 includes a light 12 made of a light emitting element such as an LED, a light control unit 13 that controls turning on and off of the light 12, and memories 2 to 4. A DMA (Direct Memory Access) control unit 14 that performs data transfer between the components 5 to 9 and 11 to 13 on behalf of the CPU 1 so as not to place a burden on the CPU 1, and a camera control unit 5 to a DMA control unit 14 is integrated with the luminance signal Y, which is luminance information, of the imaged image data in the YUV format transferred to the work memory 3 through 14, and an image calculation unit 15 for outputting the integrated value.

  Next, with reference to the timing chart of FIG. 2, the camera photographing operation performed by flashing the light with the camera-equipped mobile phone 10 of this example will be described. This camera shooting is characterized in that two continuous shootings of the subject are performed, and in order to turn on the light during the exposure period of two shot images in this way, as shown in FIG. It is necessary to turn on the light by adjusting the exposure period and the timing of light emission.

  In the timing chart of FIG. 2, among the frame periods determined by the synchronization signal S1, the image A (first image) that is captured by exposure in the frame A period is image data (output data Do) in the next frame B period. Is output as Similarly, an image B (second image) captured by exposure in the frame B period is output as image data in the next frame C period. Accordingly, when images A and B are continuously shot in the shooting of this example, in order to turn on the light during the exposure period T in which both images A and B are shot, The light emission signal S2 is set to H (High level), and the light is turned on.

Next, a specific example of performing camera shooting using the camera-equipped mobile phone 10 of FIG. 1 will be described with reference to the flowchart of FIG.
First, when a user requests shooting by pressing a key assigned to shooting among a plurality of information input keys arranged in the key unit 9 of the camera-equipped mobile phone 10 (step S1), the light is written. The control unit 13 inputs a signal according to the request through the key input unit 11, and based on this, supplies the H level light emission signal S2 to turn on the light 12 (step S2).

  Next, during the period in which the light emission signal S2 is set to the H level, the camera control unit 6 captures the captured image data (first captured image data) of the image A that is captured by the camera unit 5 during the frame A period. ) Is transferred to the work memory 3 and stored in YUV format data in the frame B period (step S3). At the same time, the image calculation unit 15 integrates the luminance signal Y included in the first captured image data in the YUV format stored in the work memory 3 from the camera control unit 6, The integrated value (referred to as the first integrated value A1) is output to the work memory 3.

  Similarly, during the period in which the light emission signal S2 is set to the H level, the camera control unit 6 captures the captured image data (second captured image) of the image B that is captured by the camera unit 5 during the frame B period. Data) is transferred and stored in the work memory 3 as data in the YUV format in the frame C period (step S4). At the same time, the image calculation unit 15 integrates the luminance signal Y included in the second photographed image data in the YUV format transferred from the camera control unit 6 to the work memory 3, The integrated value (second integrated value B1) is output to the work memory 3. Next, when the exposure period T of the frame A period and the frame B period ends, the light control unit 13 turns off the light 12 by supplying an L level light emission signal S2 (step S5).

  Next, the CPU 1 compares the first integrated value A1 of the luminance signal Y of the image A with the second integrated value B1 of the luminance signal Y of the image B (step S6). Here, the images A and B are determined to be brighter images as the first integrated value A1 and the second integrated value B1 are larger. That is, it is determined that the first integrated value A1 and the second integrated value B1 become brighter as they become closer to (FFh: data value in hexadecimal notation) and darker as they get closer to (00h). Here, a threshold C serving as a reference for the luminance signal Y is set in advance. This threshold value C is determined according to a request such as luminance required for an image.

Next, the CPU 1 compares the absolute value of the difference between the images A and B with the floor price C (step S7). That is, it is determined whether or not the absolute value of the difference between the images A and B exceeds the floor price C. Here, the following (Expression 1) and (Expression 2) are set in advance, and one of them corresponds to the result of comparison.
| (Integrated value A1) − (Integrated value B1) | <Threshold C (Formula 1)
| (Integrated value A1) − (Integrated value B1) | ≧ Threshold value C (Formula 2)

  As a result of the comparison, if (Expression 1) is true, the image A is selected and used as a captured image (step S8). That is, as a result of comparison by the CPU 1, if the difference between the first integrated value A1 of the luminance signal Y of the image A and the second integrated value B1 of the luminance signal Y of the image B is smaller than the threshold C, continuous shooting is performed. Since the difference in brightness between the two images A and B is small, it is determined that no white-out has occurred, and the two images A and B stored in the work memory cell 4 by the DMA control unit 14 are determined. Of these, the first image A taken is selected as a saved image and transferred to the saved image memory 4 for storage. On the other hand, the second image B taken out of the two images A and B is regarded as an unnecessary image and discarded.

  As a result of the comparison, if (Expression 2) is true, the image B is selected and used as a captured image (step S9). That is, as a result of the comparison by the CPU 1, when the difference between the first integrated value A1 of the luminance signal Y of the image A and the second integrated value B1 of the luminance signal Y of the image B is greater than or equal to the threshold C, Since the brightness difference between the two images A and B taken continuously is large, it is determined that the first image A is white-out and stored in the work memory cell 4 by the DMA control unit 14. Of the two images A and B, the image B taken as the second image is selected as a saved image, transferred to the saved image memory 4 and stored. On the other hand, the image A photographed as the second of the two images A and B is regarded as an unnecessary image and discarded.

  In general, in the camera-equipped mobile phone 10 having a light, the internal adjustment value of the exposure amount of the camera unit 5 and the light lighting timing are set on the premise that shooting is performed at a certain subject distance or more. In this case, the first image A is selected as the saved image. The camera unit 5 performs exposure adjustment internally so that an optimum image is always output based on the input image signal. Further, even when the light is turned on and white-out, the adjustment value suitable for the light-on state is obtained in the next frame, and the photographed image is not in the white-out state. Therefore, as described above, when the comparison result corresponds to (Equation 2), it is determined that the first image A is too bright compared to the second image B, and is therefore white-out. , Image B can be selected and used as a stored image.

  As described above, even if the camera-equipped mobile phone 10 of this example does not have an autofocus function, the image capturing timing is shifted and the two images A and B are continuously captured, and the light Whiteout can be avoided by adjusting the acceptance of the influence. In other words, with conventional camera phones, there is no autofocus function, so if you shoot with the light turned on when the subject distance is too close, the saved image will be whitened out, so the user will give up or close-up shooting Sometimes the trouble of re-setting the light so that it does not flash and shooting again is inevitable. In this way, giving up shooting or redoing shooting misses a photo opportunity, so the meaning of carrying the camera is lost. In this regard, as described above, according to the camera-equipped mobile phone 10 of this example, it is possible to reliably capture an image that is not whitened out by a single photographing action, so that a photo opportunity is not missed.

Thus, according to the configuration of the camera-equipped mobile phone 10 in this example, the close-up photography by the camera unit 5 in the camera-equipped cellular phone 10 that does not have the autofocus function and includes the light 12 that is turned on during the close-up photography. Sometimes, the work memory 3 for temporarily storing two images A and B taken with the lights 12 turned on in the two frame A and B periods are stored in the work memory 3. Luminances included in the saved image memory 4 for storing either one of the two images A and B as saved images, and the two images A and B stored in the working memory 3 from the camera unit 5. An image calculation unit 15 that integrates the magnitude of the information Y and outputs the first integrated value A1 of the first image A and the second integrated value B1 of the second image B to the work memory 3; , First integrated value 1 and the second integrated value B1 are compared, and if the absolute value of the difference between the two integrated values A1 and B1 is smaller than a preset threshold C, the first image A is stored from the work memory 3 and stored. When the absolute value of the difference between the integrated values A1 and B1 is greater than or equal to the threshold value C, the second image A and image B are saved from the work memory 3 and stored. Since at least the central processing unit 1 that controls to transfer to the storage memory 4 is provided, it is possible to perform close-up photography by lighting the light without causing whiteout.
Therefore, even if the autofocus function is not provided, it is possible to efficiently prevent the occurrence of whiteout of a captured image caused by the flash of light during close-up shooting.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and the present invention can be changed even if there is a design change without departing from the gist of the present invention. include. For example, in the embodiment, the two captured images to be compared and selected have been described by taking two consecutive images as an example, but the target image does not need to be two consecutive images, and the second image The image may be from the second frame period, and three images may be targeted. The setting of the interval and the number of images can be determined according to the performance of the camera and the light. Further, in the embodiment, the example in which the comparison and selection processing of the captured image is performed by the image calculation unit and the integration value comparison processing is performed by the CPU in the luminance signal calculation processing, but both the integration processing and the comparison processing are performed by the CPU. You can also do it. Further, the calculation process may be performed by the CPU, and the comparison process may be performed by the image calculation unit.

  In the embodiment, the signal output from the camera is described as the YUV format, and the luminance signal included in the signal is used as the luminance information. However, the signal output from the camera is not limited to this and is output in the YCbCr format. Other formats such as RGB format may be used. In this case, the signal used for the calculation is a signal that can be used as luminance information among the image data of each format. Further, as an example of determining whether or not white-out has been performed, the example in which the luminance signals of the two photographed images are used has been described. However, a register that stores a value for detecting the current exposure amount inside the camera module is used. If you have one, you can use that value. Moreover, although the example applied to the mobile phone has been described in the embodiments, the mobile phone is not limited to the mobile phone, and is not limited to a mobile phone such as a PDA (Personal Digital Assistants) and a portable GPS (Global Positioning System). It can be applied to general electronic equipment.

It is a block diagram which shows the structure of the electronic device with a camera which is an Example of this invention. It is a timing chart which shows operation | movement of the electronic device with a camera. It is a flowchart which shows the imaging | photography method using the electronic device with the camera. It is a block diagram which shows roughly the structure of an example of the conventional digital camera.

Explanation of symbols

1 CPU (Central processing unit)
2 Memory for Program 3 Memory for Work 4 Memory for Stored Image 5 Camera Unit 6 Camera Control Unit 7 Display Unit 8 Display Control Unit 9 Key Unit 10 Mobile Phone with Camera 11 Key Input Unit 12 Light 13 Write Control Unit 14 DMA (Direct Memory) Access) Control unit 15 Image calculation unit

Claims (7)

It is not equipped with an autofocus function, is equipped with a light, and the camera unit is configured to adjust exposure so that an optimal image is always output based on the input image signal, and the first frame. in even when the light is whiteout lit, in the next frame a camera-equipped electronic apparatus is configured such that the exposure adjustment value that matches the light lit,
A working memory for temporarily storing two images taken with the light on in two frame periods during close-up photography by the camera;
A saved image memory for storing one of the two images stored in the working memory as a saved image;
The magnitudes of the luminance information included in the two images stored in the working memory from the camera are integrated, and the first integrated value of the first image and the second of the second image are added. An image calculation unit that outputs the integrated value of
The first integrated value and the second integrated value are compared, and if the absolute value of the difference between the two integrated values is smaller than a preset threshold value, the first image is extracted from the work memory. If the absolute value of the difference between the two integrated values is greater than or equal to the threshold value, the second image is transferred from the working memory to the saved image memory. A central processing unit which controls to transfer and store;
An electronic device with a camera, comprising:   2. The electronic device with camera according to claim 1, wherein the working memory temporarily stores the two images taken in a state where the light is turned on in each of the two consecutive frame periods. .   The camera-equipped electronic device according to claim 2, wherein the image calculation unit integrates the magnitudes of the luminance information included in the two consecutive images.   4. The camera-equipped electronic device according to claim 2, wherein the central processing unit compares the first integrated value and the second integrated value of each of the two consecutive images. . It is not equipped with an autofocus function, is equipped with a light, and the camera unit is configured to adjust exposure so that an optimal image is always output based on the input image signal, and the first frame. in even when the light is whiteout lit, in the next frame a shot method using a camera-equipped electronic apparatus that is configured such that the exposure adjustment value that matches the light lit,
During close-up photography by the camera, a step of taking a first image by exposing the light during a first frame period; and
Exposing the light in a second frame period after the first frame period to capture a second image and outputting image data of the first image at the same time;
Outputting the second image data in a third frame period after the second frame period;
Storing the image data of the first image and the second image in a working memory over time,
The first integrated value of the first image and the second integrated value of the second image are integrated by integrating the magnitudes of luminance information included in the first image and the second image, respectively. Each of which is output to the working memory over time,
Comparing the first integrated value and the second integrated value;
As a result of the comparison, when the absolute value of the difference between the two integrated values is smaller than a preset threshold value, the first image is transferred from the working memory to the saved image memory and stored;
As a result of the comparison, if the absolute value of the difference between the two integrated values is greater than or equal to a preset threshold value, the second image is transferred from the working memory to the saved image memory and stored. Stages,
An imaging method using an electronic device with a camera, comprising:   6. The photographing method using an electronic apparatus with a camera according to claim 5, wherein the first to third frame periods are continuous. The photographing method using an electronic device with a camera according to claim 5 or 6, wherein a luminance signal is used as the luminance information.

Images