JP6180728B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, and a program.

  White balance processing for adjusting white balance by adjusting the gain of RAW data is performed as one of image processing for RAW data. As a technique for performing white balance processing on RAW data, for example, a technique described in Patent Document 1 can be cited.

JP 2011-155696A

  In recent years, instead of causing the imaging device that performed imaging to perform image processing on RAW data (data indicating a RAW image, which is a raw image obtained by imaging, hereinafter the same), RAW data is used. An increasing number of users have recorded on a recording medium and have a computer such as a PC (Personal Computer) perform image processing on RAW data stored in the recording medium.

  Here, since the RAW data is raw data before image processing such as white balance processing, contrast adjustment processing, and noise removal processing, for example, the size is generally large. Therefore, if the RAW data can be more efficiently compressed and the size of the RAW data can be reduced, for example, the number of images that can be recorded on the recording medium (the number of images indicated by the RAW data) can be increased. . Therefore, for example, the user convenience can be improved by more efficiently compressing the RAW data.

  Further, even when image processing is performed on RAW data in the imaging apparatus, the RAW data obtained from the imaging element is transferred to a memory such as an SDRAM (Synchronous Dynamic Random Access Memory) and recorded, and then stored in the memory. It is common to perform image processing on stored RAW data.

  As described above, even when image processing is performed on RAW data in the imaging apparatus, power consumption generated by data transfer can be reduced by compressing the RAW data more efficiently and reducing the size of the RAW data. It becomes possible to suppress. Therefore, for example, when the imaging device is driven by an internal power source such as a battery, the driving time of the imaging device can be further increased by reducing the size of the RAW data.

  Therefore, compressing RAW data more efficiently is important for improving user convenience and reducing power consumption for data transfer.

  Here, for example, in the technique described in Patent Document 1, white balance processing is performed on RAW data, and then the image after white balance processing is compressed. However, for example, in the technique described in Patent Document 1, no consideration is given to more efficiently compressing RAW data. Therefore, for example, even if the technique described in Patent Document 1 is used, there is a risk of inefficient compression.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a new and improved image processing apparatus, image processing method, and image processing apparatus capable of more efficiently compressing RAW data. And to provide a program.

  In order to achieve the above object, according to the first aspect of the present invention, an overexposed region included in the RAW image is generated based on RAW data indicating a RAW image that is a raw image obtained by imaging. A determination unit that determines a corresponding region, and a conversion unit that selectively performs distribution conversion of an image spatial frequency on the RAW data corresponding to the determined region corresponding to the overexposed region, An image processing apparatus is provided in which the conversion unit reduces a luminance change of pixels other than the set reference pixel in an area corresponding to the overexposed area.

  With this configuration, unnecessary information in the RAW data can be reduced, so that the RAW data can be compressed more efficiently.

  Further, the conversion unit may set the luminance change of pixels other than the reference pixel to zero in an area corresponding to the overexposure area.

  In addition, the determination unit multiplies the comparison result between the pixel value of the reference pixel and a predetermined threshold, the pixel value of each pixel other than the reference pixel and the corresponding white balance coefficient, and the predetermined value. An area corresponding to the overexposed area may be determined based on a comparison result with a threshold value.

  The image processing apparatus may further include a compression processing unit that performs an image compression process on the RAW data that has been subjected to the image spatial frequency distribution conversion.

  In order to achieve the above object, according to the second aspect of the present invention, the overexposure included in the RAW image is based on the RAW data indicating the RAW image which is a raw image obtained by imaging. A step of determining a region corresponding to a region, and a step of selectively performing distribution transformation of an image spatial frequency on the RAW data corresponding to the determined region corresponding to the overexposed region, In the step of performing the distribution conversion of the image spatial frequency, there is provided an image processing method for reducing a luminance change of pixels other than the set reference pixel in the region corresponding to the overexposed region.

  By using such a method, unnecessary information in the RAW data can be reduced, so that the RAW data can be compressed more efficiently.

  In order to achieve the above object, according to the third aspect of the present invention, the overexposure included in the RAW image is based on the RAW data indicating the RAW image which is a raw image obtained by imaging. The computer functions as a determination unit that determines a region corresponding to a region, and a conversion unit that selectively performs distribution conversion of an image spatial frequency on the RAW data corresponding to the determined region corresponding to the overexposed region. The converting means is provided with a program for reducing a change in luminance of pixels other than the set reference pixel in an area corresponding to the overexposed area.

  By using such a program, unnecessary information in the RAW data can be reduced, so that the RAW data can be compressed more efficiently.

  According to the present invention, RAW data can be compressed more efficiently.

It is explanatory drawing for demonstrating an example of the problem in the existing image compression with respect to RAW data. It is a block diagram which shows an example of a structure of the image processing apparatus which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the determination part which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the conversion part which concerns on embodiment of this invention. It is explanatory drawing for demonstrating an example of the process of the conversion process part with which the image processing apparatus which concerns on embodiment of this invention is provided. 5 is a flowchart showing an example of processing in the image processing apparatus according to the embodiment of the present invention. It is explanatory drawing for demonstrating an example of the result of the image compression process with respect to RAW data by which distribution conversion of the image spatial frequency which concerns on embodiment of this invention was selectively performed.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the following, a case where the RAW data is image data including an R (Red) pixel, a G (Green) pixel, and a B (Blue) pixel obtained by a Bayer array image sensor or a color filter will be described as an example. explain. Note that the RAW data to which the image processing method according to the embodiment of the present invention can be applied is not limited to the image data obtained by the Bayer array image sensor or the like. For example, the image processing method according to the embodiment of the present invention can be applied to various image data such as “RGB + W”, “RGB + IR”, and “RGB + S”.

[1] Problems in Existing Image Compression Before describing an example of a configuration of an image processing apparatus according to an embodiment of the present invention that can perform processing according to an image processing method according to an embodiment of the present invention, first, An example of a problem in existing image compression for RAW data will be described.

  FIG. 1 is an explanatory diagram for explaining an example of a problem in existing image compression for RAW data. 1 shows an example of pixel values of RAW data before white balance processing is performed, and B shown in FIG. 1 shows an example of pixel values of RAW data after white balance processing is performed. Is shown. Here, in FIG. 1, the horizontal axis indicates the intensity level of the light source (1 <2 <3 <4 <5), and the vertical axis indicates the pixel value. FIG. 1 shows an example in which the RAW data is an image indicating gray (1: gray closer to black,..., 5: gray closer to white). Further, the saturation value shown in FIG. 1 indicates an upper limit value of the number of bits of RAW data (for example, 256 if it is 8 bits).

  For example, as shown in FIG. 1A, when the RAW data is an image indicating gray, the pixel value has a characteristic of “G> R> B”. In view of the above characteristics, white balance processing is realized, for example, as shown in FIG. 1B by performing processing of multiplying the gain coefficient so that the R pixel / G pixel / B pixel have the same value. Is done.

  The image sensor has a range that can be expressed, and the RAW data of the white balance processing has a constant luminance of the subject as shown in “3”, “4”, and “5” shown in FIG. When it becomes larger, the G pixel is saturated.

  When the white balance processing is performed, for example, each of the R pixel / G pixel / B pixel is multiplied by a gain coefficient for adjusting the white balance (hereinafter sometimes referred to as “white balance coefficient”). As a result, for example, as indicated by “3”, “4”, and “5” shown in FIG. 1B, the R pixel / G pixel / B pixel is saturated. Here, for example, as shown by “3”, “4”, and “5” shown in FIG. 1B, after white balance processing, the area where the R pixel / G pixel / B pixel is saturated is “whiteout”. It is called “Area”.

  In general, since an image that a user views is an image indicated by image data after image processing including white balance processing, an “area corresponding to a whiteout region” in RAW data (a whiteout region after white balance processing) The luminance difference (image spatial frequency component) of the R pixel / B pixel in the region (hereinafter, the same shall apply) can be said to be unnecessary information (data).

  However, the image compression processing for RAW data according to the existing technology performs compression including the unnecessary information. Therefore, it can be said that the image compression processing for RAW data according to the existing technology is inefficient compression.

[2] Image processing apparatus according to an embodiment of the present invention Therefore, an image processing apparatus according to an embodiment of the present invention is a RAW image based on RAW data as a process related to an image processing method according to an embodiment of the present invention. An area corresponding to the overexposed area included in the image is determined (determination process). An example of the determination process according to the embodiment of the present invention will be described later.

  In addition, the image processing apparatus according to the embodiment of the present invention selectively performs distribution conversion of image spatial frequency on RAW data corresponding to a region corresponding to a whiteout region determined in the determination process (conversion). processing). More specifically, the image processing apparatus according to the embodiment of the present invention reduces the luminance change of pixels other than the set reference pixel in, for example, a region corresponding to a whiteout region.

  Here, as the reference pixel according to the embodiment of the present invention, for example, a pixel of a color that first saturates when the luminance of the subject exceeds a certain level. For example, in the example illustrated in FIG. 1, the reference pixel according to the embodiment of the present invention to be set includes a G pixel. Note that the reference pixel according to the embodiment of the present invention is not limited to the G pixel. For example, depending on the type of light source, the R pixel and the B pixel can be set as the reference pixels according to the embodiment of the present invention. Here, the reference pixel according to the embodiment of the present invention may be set in advance, or may be set (reset) based on a user operation, a light source detection result, or the like.

  As described above, the image processing apparatus according to the embodiment of the present invention performs determination processing and conversion processing as processing related to the image processing method according to the embodiment of the present invention. Here, the image processing apparatus according to the embodiment of the present invention reduces the luminance change (information) of pixels other than the reference pixel in the area corresponding to the overexposed area in the conversion process. Therefore, when the image compression processing is performed on the RAW data subjected to the conversion processing in the image processing apparatus according to the embodiment of the present invention, the above-described unnecessary information in the RAW data is compressed on the RAW data. Can be done.

  Therefore, the image processing apparatus according to the embodiment of the present invention can compress the RAW data more efficiently by performing the process (determination process and conversion process) according to the image processing method according to the embodiment of the present invention. .

  Note that the image compression processing for the RAW data according to the embodiment of the present invention may be performed by, for example, the image processing apparatus according to the embodiment of the present invention or performed by an external apparatus capable of performing the image compression processing. It may be broken.

  Hereinafter, processing (determination processing and conversion processing) according to the image processing method according to the embodiment of the present invention will be described more specifically while describing an example of the configuration of the image processing apparatus according to the embodiment of the present invention.

  Hereinafter, an example of the configuration of the image processing apparatus according to the embodiment of the present invention will be described by taking as an example the case where the G pixel is a reference pixel and the R pixel and the B pixel are pixels other than the reference pixel. As described above, the reference pixel according to the embodiment of the present invention is not limited to the G pixel.

  FIG. 2 is a block diagram showing an example of the configuration of the image processing apparatus 100 according to the embodiment of the present invention. The image processing apparatus 100 includes, for example, a determination unit 102 and a conversion unit 104.

  Here, in FIG. 2, RAW data processed by the image processing apparatus 100, that is, RAW data before processing related to the image processing method according to the embodiment of the present invention is indicated as “input RAW data”. In FIG. 2, the RAW data processed by the image processing apparatus 100, that is, the RAW data after the processing related to the image processing method according to the embodiment of the present invention is performed is indicated as “output RAW data”.

  Examples of the input RAW data according to the embodiment of the present invention include RAW data transmitted in accordance with imaging from an imaging unit (described later) or an external imaging device. Note that the input RAW data according to the embodiment of the present invention is not limited to the above. For example, input RAW data according to an embodiment of the present invention is read from a recording medium such as an external recording medium included in a storage unit (described later) or an external device connected via a network (or directly). RAW data stored in the recording medium may be used.

  The image processing apparatus 100 includes, for example, a control unit (not shown), a ROM (Read Only Memory, not shown), a RAM (Random Access Memory, not shown), a storage unit (not shown), and a user. An operation unit (not shown) that can be operated, a display unit (not shown) that displays various screens on a display screen, a communication unit (not shown) for communicating with an external device, and an imaging unit (not shown) Etc.) may be provided. For example, the image processing apparatus 100 connects the above-described components by a bus as a data transmission path.

  Here, the control unit (not shown) includes, for example, a CPU (Central Processing Unit) and various processing circuits, and controls the entire image processing apparatus 100. In addition, the control unit (not shown) may serve as, for example, the determination unit 102 and the conversion unit 104 (or either one). Needless to say, the determination unit 102 and the conversion unit 104 may be configured by a dedicated (or general-purpose) processing circuit capable of realizing the processing of each unit.

  In addition, the control unit (not shown) plays a role of performing processing on the output RAW data, such as processing to record output RAW data in a storage unit (not shown) and processing related to a compression processing unit described later. Also good.

  A ROM (not shown) stores control data such as programs and operation parameters used by a control unit (not shown). A RAM (not shown) temporarily stores programs executed by a control unit (not shown).

  The storage unit (not shown) is a storage unit included in the image processing apparatus 100, and stores various data such as image data and applications. Here, examples of the storage unit (not shown) include a magnetic recording medium such as a hard disk, a non-volatile memory such as a flash memory, and the like. Further, the storage unit (not shown) may be detachable from the image processing apparatus 100.

  Examples of the operation unit (not shown) include a button, a direction key, or a combination thereof. Further, the image processing apparatus 100 can be connected to, for example, an operation input device (for example, a keyboard or a mouse) as an external apparatus of the image processing apparatus 100.

  Examples of the display unit (not shown) include a liquid crystal display (Liquid Crystal Display) and an organic EL display (Organic Electro-Luminescence display). The display unit (not shown) may be a device capable of display and user operation, such as a touch screen. The image processing apparatus 100 can also be connected to a display device (for example, an external display) as an external apparatus of the image processing apparatus 100 regardless of the presence or absence of a display unit (not shown).

  A communication unit (not shown) is a communication unit included in the image processing apparatus 100, and performs wireless / wired communication with an external apparatus via a network (or directly). Here, as a communication unit (not shown), for example, a communication antenna and an RF (Radio Frequency) circuit (wireless communication), an IEEE 802.15.1 port and a transmission / reception circuit (wireless communication), an IEEE 802.11b port and a transmission / reception Examples include a circuit (wireless communication), a LAN (Local Area Network) terminal, a transmission / reception circuit (wired communication), and the like. In addition, as a network according to the embodiment of the present invention, for example, a wired network such as a LAN or a WAN (Wide Area Network), a wireless LAN (WLAN; Wireless Local Area Network) or a wireless WAN (WWAN; Wireless) via a base station is used. Examples include a wireless network such as a wide area network, or the Internet using a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol).

  The imaging unit (not shown) serves to capture a still image, for example. When the imaging unit (not shown) is provided, the image processing apparatus 100 can process RAW data generated by imaging in the imaging unit (not shown), for example.

  Here, as an imaging unit (not shown) according to the embodiment of the present invention, for example, a lens / imaging device can be cited. The lens / imaging device is composed of, for example, an optical lens and an image sensor using a plurality of imaging devices such as CMOS (Complementary Metal Oxide Semiconductor).

  The determination unit 102 plays a role of performing a determination process according to the embodiment of the present invention, and determines an area corresponding to a whiteout area included in the RAW image based on the input RAW data.

  FIG. 3 is a block diagram illustrating an example of the configuration of the determination unit 102 according to the embodiment of the present invention. The determination unit 102 includes, for example, pixel determination units 110A, 110B, and 110C and an overexposed region determination unit 112.

  The pixel determination unit 110A includes a pixel value corresponding to the R pixel in the RAW data (hereinafter, sometimes referred to as “R-Pixel” or “pixel value R-Pixel”), and a white balance coefficient corresponding to the R pixel. (Hereinafter may be referred to as “WB-RGine” or “white balance coefficient WB-RGane”). Here, the white balance coefficient WB-RGain is determined based on the result of detecting RAW data in a control unit (not shown) or a detection circuit, for example.

  The pixel determination unit 110A compares a value obtained by multiplying a pixel value R-Pixel (pixel value of a pixel other than the reference pixel) and a white balance coefficient WB-RGain (corresponding white balance coefficient) with a predetermined threshold Full-Range. To do.

  Here, the predetermined threshold Full-Range according to the embodiment of the present invention may be, for example, a preset fixed value or a variable value that can be changed by a user operation or the like. As a specific example, the predetermined threshold Full-Range includes, for example, a maximum value that can be expressed (for example, “4095” for 12-bit expression and “1023” for 10-bit expression).

  In addition, the pixel determination unit 110 </ b> A transmits, for example, data indicating the comparison result to the overexposed region determination unit 112. Here, as the data indicating the comparison result according to the embodiment of the present invention, for example, whether or not the multiplied value is greater than a predetermined threshold Full-Range (or the multiplied value is a predetermined threshold Full-Range). Flag R-Pixel_Full_flag indicating whether or not the above is the same, and so on). Needless to say, the data indicating the comparison result according to the embodiment of the present invention is not limited to the flag.

  The pixel determination unit 110B includes a pixel value corresponding to the B pixel in the RAW data (hereinafter, sometimes referred to as “B-Pixel” or “pixel value B-Pixel”), and a white balance coefficient corresponding to the B pixel. (Hereinafter may be referred to as “WB-BGain” or “white balance coefficient WB-BGain”). Here, the white balance coefficient WB-BGain is determined based on the result of detecting the RAW data in a control unit (not shown) or a detection circuit, for example.

  Similarly to the pixel determination unit 110A, the pixel determination unit 110B multiplies a pixel value B-Pixel (pixel value of a pixel other than the reference pixel) and a white balance coefficient WB-BGain (corresponding white balance coefficient), and a predetermined value. To the threshold Full-Bang.

  In addition, the pixel determination unit 110B transmits data indicating the comparison result to the overexposed region determination unit 112, for example, similarly to the pixel determination unit 110A. Here, as the data indicating the comparison result according to the embodiment of the present invention, for example, a flag B-Pixel_Full_flag indicating whether or not the multiplied value is larger than a predetermined threshold Full-Range. Needless to say, the data indicating the comparison result according to the embodiment of the present invention is not limited to the flag.

  The pixel value corresponding to the G pixel in the RAW data (hereinafter may be referred to as “G-Pixel” or “pixel value G-Pixel”) is input to the pixel determination unit 110C.

  The pixel determination unit 110C compares the pixel value G-Pixel (the pixel value of the reference pixel) with a predetermined threshold Full-Range. Then, the pixel determination unit 110C transmits data indicating the comparison result to the overexposed region determination unit 112, for example, similarly to the pixel determination unit 110A. Here, the data indicating the comparison result according to the embodiment of the present invention includes, for example, a flag G-Pixel_Full_flag indicating whether or not the pixel value G-Pixel is larger than a predetermined threshold Full-Range. Needless to say, the data indicating the comparison result according to the embodiment of the present invention is not limited to the flag.

  The whiteout region determination unit 112 determines a region corresponding to the whiteout region included in the RAW image based on the data indicating the comparison results transmitted from the pixel determination units 110A, 110B, and 110C.

  More specifically, for example, when the whiteout region determination unit 112 indicates that the flag R-Pixel_F, the flag B-Pixel_F, and the flag G-Pixel_F are all greater than a predetermined threshold Full-Range, It is determined that the pixel corresponding to the flag is a pixel included in the region corresponding to the whiteout region.

Note that the processing in the whiteout region determination unit 112 is not limited to the above. For example, the overexposed area determination unit 112 can determine that the pixel is included in an area corresponding to the overexposed area when the following conditional expression is satisfied.
Conditional expression:
(G-Pixel == Full-Range) &&
(R-Pixel * WB-RGine> Full-Range) &&
(B-Pixel * WB-BGain> Full-Range)

  When the determination is made, the overexposed region determination unit 112 transmits data indicating the determination result of the region corresponding to the overexposed region to the conversion unit 104. Here, as the data indicating the determination result according to the embodiment of the present invention, for example, a flag Full_flag indicating whether or not it is an area corresponding to a whiteout area. Needless to say, the data indicating the determination result according to the embodiment of the present invention is not limited to the flag.

  The determination unit 102 includes, for example, the pixel determination units 110A, 110B, and 110C and the overexposure region determination unit 112, thereby determining an area corresponding to the overexposure region based on the input RAW data. Then, the determination unit 102 transmits a flag Full_flag (an example of data indicating a determination result; hereinafter the same) to the conversion unit 104.

  With reference to FIG. 2 again, an example of the configuration of the image processing apparatus 100 will be described. The conversion unit 104 performs the conversion process according to the embodiment of the present invention, and based on the transmitted flag Full_flag, the distribution of the image spatial frequency with respect to the RAW data corresponding to the region corresponding to the overexposure region Perform the conversion selectively. More specifically, the conversion unit 104 reduces the luminance change of pixels other than the set reference pixel, for example, in a region corresponding to a whiteout region.

  FIG. 4 is a block diagram showing an example of the configuration of the conversion unit 104 according to the embodiment of the present invention. The conversion unit 104 includes, for example, conversion processing units 114A, 114B, and 114C.

  The conversion processing unit 114A selectively performs distribution conversion of the image spatial frequency on the pixel value R-Pixel based on the transmitted flag Full_flag. Then, the conversion processing unit 114A outputs the pixel value Rout on which the distribution conversion of the image spatial frequency has been selectively performed.

  More specifically, the conversion processing unit 114A performs image spatial frequency distribution conversion when the flag Full_flag indicates a region corresponding to a whiteout region, and the flag Full_flag is used in a region corresponding to a whiteout region. If it does not indicate that there is, it does not perform image spatial frequency distribution conversion. In the example of FIG. 4, when the flag Full_flag indicates “1”, the example indicates that the flag Full_flag is an area corresponding to a whiteout area.

  Here, when the state changes from “whiteout region” to “non-whiteout region”, the luminance change is the smallest. Therefore, the conversion processing unit 114A, for example, performs the calculation shown in the following Equation 1 to make the luminance change of the pixel value R-Pixel (pixel value of pixels other than the reference pixel) zero.

Rout = (Full_Range / WB-RGine)
... (Formula 1)

  FIG. 5 is an explanatory diagram for explaining an example of processing of the conversion processing unit 114A included in the image processing apparatus 100 according to the embodiment of the present invention. 5A shows an example of the pixel value R-Pixel before processing by the conversion processing unit 114A, and B shown in FIG. 5 shows an example of the pixel value Rout after processing by the conversion processing unit 114A. Show.

  For example, as illustrated in B of FIG. 5, the conversion processing unit 114 </ b> A converts, for example, pixel values determined to be areas corresponding to overexposed areas to the same value. That is, the conversion processing unit 114A deletes unnecessary luminance change (information) in an area corresponding to an overexposed area, for example.

  Therefore, the unnecessary information in the input RAW data can be reduced by the conversion processing unit 114A performing, for example, the calculation shown in Equation 1 below.

  Note that the processing in the conversion processing unit 114 </ b> A is not limited to the processing for performing the calculation shown in Equation 1 above. The conversion processing unit 114A can perform an arbitrary calculation that can reduce the luminance change of the pixel value R-Pixel (pixel value of a pixel other than the reference pixel).

  With reference to FIG. 4 again, an example of the configuration of the conversion unit 104 according to the embodiment of the present invention will be described. The conversion processing unit 114B selectively performs distribution conversion of the image spatial frequency on the pixel value B-Pixel based on the transmitted flag Full_flag. Then, the conversion processing unit 114B outputs the pixel value Bout on which the distribution conversion of the image spatial frequency has been selectively performed.

  More specifically, like the conversion processing unit 114A, the conversion processing unit 114B performs image spatial frequency distribution conversion when the flag Full_flag indicates a region corresponding to an over-exposure region, and the flag Full_flag is If it is not indicated that the area corresponds to the overexposed area, the distribution conversion of the image spatial frequency is not performed.

  Here, when the state changes from “whiteout region” to “non-whiteout region”, the luminance change is the smallest. Therefore, for example, the conversion processing unit 114B performs the calculation shown in the following Equation 2 to make the luminance change of the pixel value B-Pixel (pixel values of pixels other than the reference pixel) zero.

Bout = (Full_Range / WB-BGain)
... (Formula 2)

  In addition, the process in the conversion process part 114B is not restricted to the process which performs the calculation shown to the said Numerical formula 2. The conversion processing unit 114B can perform an arbitrary calculation that can reduce the luminance change of the pixel value B-Pixel (pixel value of a pixel other than the reference pixel).

  The conversion processing unit 114C outputs the pixel value G-Pixel (the pixel value of the reference pixel) as the pixel value Gout regardless of the transmitted flag Full_flag.

  The conversion unit 104 includes, for example, conversion processing units 114A, 114B, and 114C, and based on the transmitted flag Full_flag, the distribution of the image spatial frequency with respect to the RAW data corresponding to the region corresponding to the overexposure region Perform the conversion selectively.

  Here, as illustrated with reference to FIG. 5, for example, in the pixels other than the reference pixel, the conversion unit 104 converts a pixel value determined to be a region corresponding to a whiteout region into the same value. (When the luminance change is zero). In other words, for example, the conversion unit 104 erases unnecessary luminance change in the region corresponding to the overexposure region in pixels other than the reference pixel (when the luminance change is set to zero). Note that the processing in the conversion unit 104 is not limited to the above. For example, the conversion unit 104 may reduce an unnecessary luminance change in a region corresponding to a whiteout region in pixels other than the reference pixel.

  Therefore, the conversion unit 104 performs the conversion process according to the embodiment of the present invention, whereby the unnecessary information in the input RAW data can be reduced. Therefore, the conversion unit 104 can perform the conversion process according to the embodiment of the present invention, and can compress the RAW data more efficiently.

  For example, as illustrated in FIG. 2, the image processing apparatus 100 includes a determination unit 102 and a conversion unit 104, thereby determining a region corresponding to the overexposed region and corresponding to the determined overexposed region. Image spatial frequency distribution conversion is selectively performed on the RAW data.

  Here, an example of processing in the image processing apparatus 100 is shown.

  FIG. 6 is a flowchart showing an example of processing in the image processing apparatus 100 according to the embodiment of the present invention. Here, for example, the processes in steps S102 and S104 shown in FIG. 6 correspond to the process in the determination unit 102, and the processes in steps S106 and S108 shown in FIG. FIG. 6 illustrates an example of processing when the G pixel is a reference pixel and the R pixel and the B pixel are pixels other than the reference pixel.

  The image processing apparatus 100 determines whether input RAW data has been input (S100). If it is not determined in step S100 that input RAW data has been input, the image processing apparatus 100 does not proceed with the process.

  If it is determined in step S100 that the input RAW data has been input, the image processing apparatus 100 applies the white balance coefficient (WB-RGane) corresponding to each of the pixel value R-Pixel and the pixel value B-Pixel. , WB-BGain) (S102). Here, the process of step S102 corresponds to a process of generating a pixel value used for determining an area corresponding to a whiteout area.

  The image processing apparatus 100 determines an area corresponding to the overexposure area using the pixel value G-Pixel, the pixel value generated in step S102, and the predetermined threshold Full-Range (S104).

Here, in step S104, for example, the image processing apparatus 100 determines that the region corresponds to the overexposure region when the following conditional expression is satisfied.
Conditional expression:
(G-Pixel == Full-Range) &&
(R-Pixel * WB-RGine> Full-Range) &&
(B-Pixel * WB-BGain> Full-Range)

  Note that the process in step S104 is not limited to the process using the conditional expression. For example, as described above, the image processing apparatus 100 may determine whether the region corresponds to the over-exposure region using the flag R-Pixel_F, the flag B-Pixel_F, and the flag G-Pixel_F.

  If it is determined in step S104 that the region corresponds to the overexposure region, the image processing apparatus 100 performs distribution conversion of the image spatial frequency (S106). For example, the image processing apparatus 100 performs distribution conversion of the image spatial frequency by performing the calculation shown in the above Formula 1 and Formula 2.

  On the other hand, if it is not determined in step S104 that the region corresponds to the overexposure region, the image processing apparatus 100 does not perform image spatial frequency distribution conversion (S108).

  When the process of step S106 or the process of step S108 is performed, the image processing apparatus 100 outputs output RAW data (RAW data on which image spatial frequency distribution conversion has been selectively performed) (S110).

  Here, for example, the image processing apparatus 100 may output the output RAW data to a recording medium capable of storing the RAW data, or can perform an image compression process on the RAW data. The output RAW data may be output to a simple compression processing unit (described later) or an external device.

  The image processing apparatus 100 includes, for example, a determination unit 102 and a conversion unit 104. The image processing apparatus 100 determines a region corresponding to a whiteout region by performing the processing illustrated in FIG. 6, and a region corresponding to the determined whiteout region. The image spatial frequency distribution conversion is selectively performed on the RAW data corresponding to.

  Here, as described above, for example, the image processing apparatus 100 erases or reduces unnecessary luminance change (information) in a region corresponding to a whiteout region in pixels other than the reference pixel.

  Therefore, since the image processing apparatus 100 can reduce the unnecessary information in the input RAW data, the RAW data can be compressed more efficiently.

  FIG. 7 is an explanatory diagram for explaining an example of a result of the image compression process on the RAW data in which the distribution conversion of the image spatial frequency according to the embodiment of the present invention is selectively performed.

  Here, A shown in FIG. 7 is compressed image data obtained as a result of image compression processing performed on output RAW data (RAW data on which image spatial frequency distribution conversion has been selectively performed). An example of the size is shown. Further, B shown in FIG. 7 shows a comparative example with respect to A of FIG. Specifically, B shown in FIG. 7 is obtained as a result of image compression processing being performed on RAW data that has not been subjected to image spatial frequency distribution conversion (that is, RAW data corresponding to input RAW data). 2 shows an example of the data size of a compressed image according to an existing technique.

  FIG. 7 shows the pixel value of the R pixel in the region corresponding to the overexposed region or the pixel value of the B pixel in the region corresponding to the overexposed region.

  FIG. 7 shows an example of the data size of the image after compression when the output RAW data is compressed by calculating the difference value with the adjacent pixel and encoding with the Huffman code.

  In the existing technology, encoding is performed by calculating a difference value regardless of whether or not it is an overexposed region. Therefore, as shown in B of FIG. 7, the data size after encoding when using the existing technology is 46 bits.

  On the other hand, the output RAW data according to the embodiment of the present invention is RAW data in which the distribution conversion of the image spatial frequency is selectively performed in the conversion unit 104. Here, as described above, the conversion unit 104 reduces the luminance change of pixels other than the reference pixel in the region corresponding to the determined overexposure region. Therefore, in the output RAW data transmitted from the conversion unit 104, The difference value in the area corresponding to the whiteout area becomes smaller. For example, when the conversion unit 104 sets the luminance change of pixels other than the reference pixel to zero in the area corresponding to the determined overexposure area, for example, as illustrated in FIG. The difference value in the area to be performed is zero.

  Therefore, as shown in FIG. 7A, encoded data obtained as a result of image compression processing performed on output RAW data (RAW data on which image spatial frequency distribution conversion has been selectively performed). The size is, for example, 8 bits.

  As apparent from comparing the encoded data size shown in A of FIG. 7 with the encoded data size shown in B of FIG. 7, output RAW data (the distribution conversion of the image spatial frequency is selectively performed). By performing the image compression process on the RAW data), the RAW data compression efficiency can be further improved. That is, the size of the compressed image data can be further reduced by performing the image compression process on the output RAW data (RAW data on which the distribution conversion of the image spatial frequency is selectively performed).

  Therefore, for example, as illustrated in FIG. 7, the image processing apparatus 100 more efficiently compresses the RAW data by performing processing (determination processing and conversion processing) according to the image processing method according to the embodiment of the present invention. You can see that

  In addition, since the image processing apparatus 100 can compress the RAW data more efficiently by performing the processing (determination processing and conversion processing) according to the image processing method according to the embodiment of the present invention, for example, It is possible to increase the number of images (number of images indicated by the RAW data) that can be recorded on the recording medium. Therefore, the convenience of the user can be improved by the image processing apparatus 100 performing the processing (determination processing and conversion processing) related to the image processing method according to the embodiment of the present invention.

  In addition, since the image processing apparatus 100 can compress the RAW data more efficiently by performing the processing (determination processing and conversion processing) according to the image processing method according to the embodiment of the present invention, for example, The power consumption generated by the data transfer of the RAW data can be suppressed, and the bandwidth required for the data transfer of the RAW data can be reduced.

  The configuration of the image processing apparatus according to the embodiment of the present invention is not limited to the configuration shown in FIG.

  For example, an image processing apparatus according to an embodiment of the present invention includes a compression processing unit that performs image compression processing on RAW data (output RAW data) that has been subjected to image space frequency distribution conversion selectively performed by the conversion unit 104. (Not shown) may further be provided.

  A compression processing unit (not shown) compresses an image indicated by the output RAW data by, for example, calculating a difference value with an adjacent pixel and encoding it with a Huffman code. Here, the encoding is a process for converting information with a high appearance probability into a short code length and converting information with a low appearance probability into a long code length. In the case of a natural image, there is a feature that the difference value between adjacent pixels is small, so the difference value “0” is converted into a minimum code length of 2 bits. Further, when the difference value becomes large (when the appearance probability becomes small), the code length to be converted becomes long.

  Note that the image compression process in the compression processing unit (not shown) is not limited to the process of encoding with the Huffman code. A compression processing unit (not shown) can compress the image indicated by the output RAW data using, for example, an arbitrary encoding method capable of compressing the image.

  As described above, the image processing apparatus has been described as an embodiment of the present invention, but the embodiment of the present invention is not limited to such an embodiment. Embodiments of the present invention include, for example, an imaging device such as a digital camera, a computer such as a PC or a server, a display device such as a television receiver, a communication device such as a mobile phone or a smartphone, a video / music playback device (or video / music). The present invention can be applied to various devices capable of processing RAW data, such as a recording / reproducing apparatus and a game machine. The embodiment of the present invention can also be applied to, for example, an image processing IC (Integrated Circuit) that can be incorporated in the above devices.

(Program according to an embodiment of the present invention)
A program for causing a computer to function as an image processing apparatus according to an embodiment of the present invention (for example, a program capable of causing a computer to function as a determination unit and a conversion unit) is executed on the computer, thereby causing RAW Data can be compressed more efficiently.

  In addition, a program capable of causing the computer to further function as a compression processing unit (not shown) is executed in the computer, so that the RAW data is more efficiently compressed, and the size of the compressed image data is reduced. It can be made smaller.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, in the above description, it is shown that a program (computer program) for causing a computer to function as the image processing apparatus according to the embodiment of the present invention is provided. However, the embodiment of the present invention further includes the above program. A stored recording medium can also be provided.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 102 Determination part 104 Conversion part 110A, 110B, 110C Pixel determination part 112 Overexposed area determination part 114A, 114B, 114C Conversion processing part

Claims (5)

A determination unit that determines a region corresponding to a whiteout region included in the RAW image based on RAW data indicating a RAW image that is a raw image obtained by imaging;
A conversion unit that selectively performs distribution conversion of an image spatial frequency on the RAW data corresponding to the region corresponding to the determined whiteout region;
With
The determination unit multiplies a comparison result between a pixel value of a set reference pixel and a predetermined threshold value, a white balance coefficient corresponding to each pixel value other than the reference pixel, and the predetermined value. Based on the comparison result with the threshold value of, determine the area corresponding to the overexposed area,
The converting unit, in a region corresponding to the overexposed area, characterized by small changes in luminance of the pixels other than the standard pixel, the image processing apparatus.   The image processing apparatus according to claim 1, wherein the conversion unit sets a change in luminance of pixels other than the reference pixel to zero in an area corresponding to the overexposure area. The image on the RAW data distribution converter is selectively performed in the spatial frequency, further comprising a compression processing unit that performs image compression processing, the image processing apparatus according to claim 1 or 2. Determining a region corresponding to a whiteout region included in the RAW image based on RAW data indicating a RAW image that is a raw image obtained by imaging;
Selectively performing image spatial frequency distribution conversion on the RAW data corresponding to the determined area corresponding to the overexposed area;
Have
In the determining step, a comparison result between a pixel value of a set reference pixel and a predetermined threshold value, a value obtained by multiplying a pixel value of each pixel other than the reference pixel and a corresponding white balance coefficient, Based on a comparison result with a predetermined threshold, an area corresponding to the overexposed area is determined,
Wherein in the step of distribution converting the image spatial frequency, in the region corresponding to the overexposed area, characterized by small changes in luminance of the pixels other than the standard pixel, the image processing method. Determination means for determining an area corresponding to the overexposure area included in the RAW image based on RAW data indicating a RAW image that is a raw image obtained by imaging;
Conversion means for selectively performing distribution conversion of image spatial frequency on the RAW data corresponding to the determined region corresponding to the overexposed region;
Function as a computer
The determination unit multiplies a comparison result between a pixel value of a set reference pixel and a predetermined threshold value, a pixel value of each pixel other than the reference pixel and a corresponding white balance coefficient, and the predetermined value. Based on the comparison result with the threshold value of, determine the area corresponding to the overexposed area,
And the converting means, in the region corresponding to the overexposed area, characterized by small changes in luminance of the pixels other than the standard pixel, program.

Images