JP6252014B2 - Gamma characteristic estimation apparatus, method and program - Google Patents

Description

  The present invention relates to a gamma characteristic estimation apparatus, method, and program.

  Measurements using digital cameras are performed in various fields. In the photometric stereo technique, the object is irradiated with light from three or more directions, a normal vector is obtained from the shadow state of the object, and the shape of the object is reproduced. For this reason, when performing a measurement such as a photometric stereo method using a digital camera, the relationship between the output value of the digital camera (for example, the digital RGB value of the photographed image) with respect to the amount of light incident on the digital camera is not known. It is difficult to measure accurately.

  In general, the amount of light incident on a digital camera and the output value of the digital camera have a non-linear relationship, and this non-linear relationship is also referred to as a gamma characteristic. When using a digital camera for measurement, if this gamma characteristic is acquired and correction using the gamma characteristic (so-called gamma correction) is not performed so that the incident light quantity and the output value have a linear relationship, the measurement accuracy is reduced. Resulting in. However, in general, what kind of image correction the digital camera is performing is not written in the operation manual or the like, so the user of the digital camera cannot know the details of the image correction from the operation manual or the like.

  In addition, the gamma characteristics of digital cameras often differ from one digital camera to another due to differences in production lots and camera application versions. For this reason, for example, when a measurement such as a photometric stereo method is performed using a digital camera, it is necessary to acquire a gamma characteristic for each measurement system.

  Gamma characteristics can be acquired by acquiring the gamma characteristics from changes in the output value when the input light quantity is changed by adjusting the aperture or exposure time of a digital camera, or by using a color such as a Macbeth chart with a known reflectance. There are a method for obtaining gamma characteristics from changes in output values when a chart is photographed, a method for obtaining gamma characteristics from changes in output values when changing the amount of emitted light from a light source whose accurate amount of emitted light is known, etc. . The gamma characteristic can be acquired by specifying the amount of light incident on the digital camera as in these methods.

  However, in many cases, a digital camera such as a compact digital camera or a digital camera mounted on a smartphone does not have a function of adjusting an aperture or exposure time, and it is expensive to newly provide an adjustment function. Further, when a color chart such as a Macbeth chart whose reflectance is measured in advance is used, a cost for preparing the color chart is required (for example, Non-Patent Document 1). Furthermore, when using a light source with a known accurate amount of emitted light, the light source itself is relatively expensive. This is because in the case of a light source that adjusts the amount of emitted light according to the voltage to be applied, the voltage and the amount of emitted light are not necessarily proportional, so the relationship between the voltage and the amount of emitted light is specified unless the change in the amount of emitted light is measured using an illuminometer or the like. This is because it cannot be done, and the cost increases accordingly. In other words, the exact amount of emitted light is known by adjusting the voltage applied by feeding back the change in the amount of emitted light measured by an illuminometer or the like to the light source. However, the cost increases because the configuration is complicated. As described above, in the gamma characteristic estimation method as described above, since the gamma characteristic is obtained from the plot of the output value with respect to the accurate incident light quantity to the digital camera, the cost is high and the calibration may be difficult.

JP 60-257673 A JP 2005-34169 A

Young-Chang Chang and John F. Reid, "RGB Calibration for Color Image Analysis in Machine Vision", IEEE Transactions on Image Processing, Vol. 5, No. 10, pp. 1414-1422, October 1996

  In the conventional method for obtaining gamma characteristics, since the gamma characteristics are obtained by specifying the amount of light incident on the camera, the cost is high.

  Therefore, an object of the present invention is to provide a gamma characteristic estimation apparatus, method, and program capable of estimating a gamma characteristic at a relatively low cost.

  According to one aspect of the present invention, a camera that captures an object irradiated with light from the first and second light sources, a control unit that controls on / off of the first and second light sources, When the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are on A data processing unit that applies an output value of the camera when the object is photographed to an expected function, and estimates a gamma characteristic from a function in which the amount of light obtained from the inverse function of the expected function is most likely An apparatus for estimating gamma characteristics is provided.

  According to the disclosed gamma characteristic estimation apparatus, method, and program, the gamma characteristic can be estimated at a relatively low cost.

It is a figure explaining the gamma characteristic estimation method in one Example. It is a figure explaining a gamma characteristic. It is a figure explaining an example of the output characteristic of a light source. It is a figure explaining an example of the procedure of a gamma characteristic estimation process. It is a figure explaining a gamma characteristic estimation process. It is a flowchart explaining an example of a gamma characteristic estimation process. It is a figure which shows an example of the light quantity distribution of a light source. It is a figure which shows an example of an output RGB value. It is a figure which shows an example of a function of a polygonal line. It is a figure which shows an example of a quadratic function. It is a flowchart explaining the other example of a gamma characteristic estimation process. It is a figure which shows an example of a function of a polygonal line. It is a figure which shows an example of an output RGB value. It is a figure explaining the gamma characteristic estimation method in a modification. It is a block diagram which shows the 1st example of a gamma characteristic estimation apparatus. It is a block diagram which shows an example of a smart phone. It is a block diagram which shows an example of a personal computer. It is a block diagram which shows the 2nd example of a gamma characteristic estimation apparatus. It is a block diagram which shows the 3rd example of a gamma characteristic estimation apparatus. It is a block diagram which shows the 4th example of a gamma characteristic estimation apparatus. It is a block diagram which shows the 5th example of a gamma characteristic estimation apparatus. It is a block diagram which shows the 6th example of a gamma characteristic estimation apparatus.

  In the disclosed gamma characteristic estimation apparatus, method, and program, the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second By applying the output value of the digital camera when the object is photographed with the light source on to an expected function, the light quantity obtained from the inverse function of this expected function is the most likely Gamma characteristics are estimated from the “likelihood function”.

  Embodiments of the disclosed gamma characteristic estimation apparatus, method, and program will be described below with reference to the drawings.

  FIG. 1 is a diagram for explaining a gamma characteristic estimation method in one embodiment. In FIG. 1, a gamma characteristic estimation apparatus 1 includes a power source 2, switches 3 and 4, a first light source 5-1, a second light source 5-2, a digital camera 6 that is an example of a camera, and a data processing unit 7. Have. The first and second light sources 5-1 and 5-2 are examples of at least two light sources, and three or more light sources may be provided. As will be described later, the digital camera 6 and the data processing unit 7 may be formed by separate devices or may be formed by a device in which these are integrally provided. As will be described later, an electronic device such as a smartphone, a tablet computer, or a personal computer may include a part or all of the gamma characteristic estimation device 1. Furthermore, for example, in order to reduce the load on the data processing unit 7, at least a part of the functions of the data processing unit 7 may be realized by an external device such as a cloud computing system.

  The first light source 5-1 can be turned on / off via the switch 4. On the other hand, the second light source 5-2 can be turned on / off via the switch 3. The on / off of the switches 3 and 4 is manually controlled, for example, controlled by the digital camera 6, controlled by the data processing unit 7, or controlled by the combination of the digital camera 6 and the data processing unit 7. May be. That is, the data processing unit 7 or the digital camera 6 may form an example of a control unit that controls on / off of the switches 3 and 4. The digital camera 6 captures the object 9 by receiving the reflected light from the object 9 irradiated with the light emitted from the light sources 5-1 and 5-2. The digital camera 6 outputs a digital output value representing the photographed object 9 to the data processing unit 7. The data processing unit 7 can be formed by a general-purpose processor or the like, and performs data processing including various calculations including gamma characteristic estimation processing and measurement based on the digital output value. The estimated gamma characteristic can be used to correct the digital output value of the digital camera 6, and the corrected digital output value may be used for, for example, recognition of the three-dimensional shape of the object 9.

  In the present embodiment, the output characteristics (or emitted light amounts) of the first and second light sources 5-1 and 5-2 may be unknown. Further, the reflectance of the object 9 may be unknown.

First, the gamma characteristic will be described. The incident light quantity to the digital camera 6 is represented by x, the constant is represented by α, the digital output RGB value (hereinafter simply referred to as “output RGB value”) as an example of the output value of the digital camera 6 is represented by y, and the gamma characteristic is represented by γ. The gamma characteristic γ is generally in a relationship of y = αx ^γ with respect to the incident light quantity x, for example, the relationship of FIG. FIG. 2 is a diagram for explaining the gamma characteristic, where the vertical axis indicates the output RGB values output from the digital camera 6, and the horizontal axis indicates the amount of light incident on the digital camera 6 in arbitrary units. The gamma characteristic is not limited to a function as shown in FIG. 2, but is often unknown. In the example described below, the output RGB value of the digital camera 6 is, for example, 8 bits, and the output RGB value output from the digital camera 6 can take values from “0” to “255”, for example.

  FIG. 3 is a diagram illustrating an example of output characteristics of the light source. In FIG. 3, the vertical axis (for example, the y axis) represents the amount of light emitted from the light source, and the horizontal axis (for example, the x axis) represents the voltage applied to the light source in arbitrary units. Each of the first and second light sources 5-1 and 5-2 has output characteristics as shown in FIG. 3, for example, but there are variations in solids, and the actual output characteristics are often unknown. In this embodiment, the gamma characteristics can be estimated by calculation (or calculation) even if the output characteristics of the light sources 5-1 and 5-2 are unknown.

  FIG. 4 is a diagram for explaining an example of the procedure of gamma characteristic estimation processing. In FIG. 4, in step ST <b> 1, the switches 4 and 3 turn off the first and second light sources 5-1 and 5-2, and the digital camera 6 captures the object 9. In step ST2, the second light source 5-2 is turned off, the switch 4 turns on the first light source 5-1, and emits light with a constant first emitted light amount. The digital camera 6 causes the object 9 to move. Take a picture. In step ST3, the switch 4 turns off the first light source 5-1, and the switch 3 turns on the second light source 5-2 to emit light with a constant second emitted light amount. Photograph the object 9. In step ST4, in a state where the second light source 5-2 emits light with the second emitted light amount, the switch 4 turns on the first light source 5-1 to emit light with the first emitted light amount. Shoots the object 9. That is, in step ST4, the digital camera 6 images the object 9 in a state where both the first and second light sources 5-1 and 5-2 emit light. The maximum value of the output RGB value is “255” if the output RGB value of the digital camera 6 is, for example, 8 bits as described above. The digital camera 6 may shoot the object 9 at least once in each step ST1 to ST4. For example, the digital camera 6 may shoot the object 9 at regular intervals or continuously shoot in each step ST1 to ST4. .

  Control of the switches 3 and 4 and the camera 6 in steps ST1 to ST4 may be performed by a processor or the like as will be described later, and the processor or the like may further perform gamma characteristic estimation processing of the data processing unit 7. .

  The first emitted light amount may be the same as the second emitted light amount or may be different from the second emitted light amount. In the latter case, the first emitted light amount may be higher than the second emitted light amount or lower than the second emitted light amount.

  FIG. 5 is a diagram for explaining the above gamma characteristic estimation processing. In FIG. 5, the vertical axis indicates the amount of light irradiated on the object 9, and the horizontal axis indicates time (transition of steps ST <b> 2 to ST <b> 4) in arbitrary units. In FIG. 5, the light amount indicated by the left-down hatching indicates the first emitted light amount by the first light source 5-1 irradiated on the object 9 in each of steps ST2 and ST4, and is indicated by the right-down hatching. The amount of light indicates the second amount of light emitted from the second light source 5-2 irradiated on the object 9 in each of steps ST3 and ST4.

  In FIG. 5, it is assumed that the amount of light emitted to the object 9 is zero (0) when both the first and second light sources 5-1 and 5-2 are off in step ST 1. However, the amount of light may not be zero by a light source (not shown) other than the first and second light sources 5-1 and 5-2.

At the right end of FIG. 5, is represented by a function gamma characteristics becomes y = .alpha.x ^gamma, and the total emission amount of light radiated in the case of gamma = 1 to the object 9 in step ST4, by the broken line STL for comparison Show. When γ = 1 and the incident light amount x to the digital camera 6 and the output RGB value y are in a linear relationship, the first light source 5-1 is on and the second light source 5-2 is off. A value (STL) obtained by adding the output RGB value y (ST2) and the output RGB value y (ST3) when the second light source 5-2 is on and the first light source 5-1 is off is The output RGB value y is the same when both the first and second light sources 5-1 and 5-2 are on.

  However, for example, when γ <1, the output RGB value y (ST2) when the first light source 5-1 is on and the second light source 5-2 is off as shown in FIG. The value (STL) obtained by adding the output RGB value y (ST3) when the second light source 5-2 is on and the first light source 5-1 is off is the first and second values. Output RGB value y (ST4) when both of the light sources 5-1 and 5-2 are on. In this embodiment, the value (STL) added as described above is greater than the output RGB value y (ST4) when both the first and second light sources 5-1 and 5-2 are on. , Γ values are estimated using part or all of the captured image. For example, from the estimated value of γ, the output RGB value y (ST2) when only the first light source 5-1 is on and the output RGB value y (ST2) when only the second light source 5-2 is on. ST3) is estimated. The output RGB when both the first and second light sources 6-1 and 5-2 that can be calculated from the light amount obtained by adding the estimated light amounts of these output RGB values y (ST 2 and ST 3) are on. The calculated value y is compared with the actual measured value of the output RGB value y obtained by actual shooting by the digital camera 6 when both the first and second light sources 5-1 and 5-2 are on. The square of the difference between the calculated value and the actually measured value (that is, the square error) is calculated. By expressing the process of adding all the square errors as an evaluation function, the evaluation function can be obtained by the least square method for comparing the calculation model and the actual measurement model.

By minimizing the above evaluation function, the calculation model can be brought close to the actual measurement model. If a function of the measured model y = αx ^γ, variable one because (gamma), theoretically can be estimated by calculating the value of gamma in 1 pixel. However, in practice, it is difficult to accurately estimate the value of γ with one pixel due to noise, quantization error, and the like. In the digital camera 6, the gamma characteristic may be a polygonal line, a quadratic function or the like in order to simplify the correction calculation. Furthermore, when the function of the calculation model is a polygonal line, a quadratic function, etc., the number of parameters is relatively large, and it is difficult to accurately estimate the value of γ by calculation with one pixel. Therefore, it is desirable to use a plurality of pixels for the calculation of the value of γ. For example, if an average value of a plurality of pixels is used, the influence of noise of each pixel can be reduced.

  In the above-described embodiment, a certain function is predicted, and the most likely coefficient of the certain function is estimated from iterative calculation and experimental design from at least three photographed images, and the gamma characteristic is estimated. For example, if the information to be handled is an 8-bit output RGB value, the range of values that the output RGB value can take is determined to be 0 to 255, and the output RGB value is continuously monotonically increased as the light quantity increases. This is possible because of

  On the other hand, when one of the two light sources is turned on / off and the digital camera is used to take a picture while changing the light quantity of the other light source, the change in the light quantity when one light source is turned on / off is constant. However, a method for obtaining the gamma characteristic using the change in the difference between the output RGB values is conceivable. In this case, any function of the calculation model can be used to calculate the gamma characteristic. However, in this case, since a certain number of photographed images are photographed while turning on / off one light source and changing the light quantity of the other light source, it is difficult to reduce costs because it takes time and effort for photographing. Here, a certain number of photographed images is a larger number than the three photographed images.

  Therefore, the present inventors generally limited the function corresponding to the gamma characteristic of the camera to some extent, and applied the residual error to some of the expected functions rather than considering it as an entirely unknown function shape. I found that the function can be specified by looking. That is, in one embodiment, a gamma characteristic is estimated by predicting a certain function and estimating the most likely coefficient of the certain function from iterative calculation and experimental design from at least three captured images. As described above, the gamma characteristic can be estimated only by taking images of at least three illumination states of the first illumination, the second illumination, and the first and second illuminations. In addition, since the first and second illuminations can be controlled by turning on and off the first and second light sources, the light quantity of the first light source or the second light source is not changed accurately, Compared with a conceivable method, the control of the first and second light sources is simpler, and it is possible to reduce the labor and time of photographing, thereby reducing the cost.

FIG. 6 is a flowchart for explaining an example of the gamma characteristic estimation process. Figure 6 is a flowchart showing processing executed when a function of computational models that are expected to be y = αx ^γ. In this example, the process of FIG. 6 is executed by the data processing unit 7. When the gamma characteristic estimation process is started, in step S1, the data processing unit 7 sets the initial value of γ to 0.5. In step S <b> 2, the data processing unit 7 controls the switch 4 to be turned on and the switch 3 to be turned off, so that only the first light source 5-1 is turned on and the object 9 is photographed by the digital camera 6. In step S <b> 3, the data processor 7 controls the switch 4 to be turned off and the switch 3 to be turned on, so that only the second light source 5-2 is turned on and the object 9 is photographed by the digital camera 6. In step S4, the data processor 7 controls both the switches 3 and 4 to be turned on, so that both the first and second light sources 5-1 and 5-2 are turned on. Shoot. In step S5, the data processing unit 7 starts from the initial value 0.5 of γ and uses the simplex method so that the least square error is minimized, and the first and second light sources 5-1 are used. , 5-2 is estimated when the value of γ that is most likely to be on when both are on, and the process ends. The data processing unit 7 may execute the process of step S5 using, for example, numerical analysis software (for example, MATLAB (registered trademark)). In this case, the least square error is used as the evaluation function, and the initial value of γ The value of γ that minimizes the error may be estimated by the fminsearch function that minimizes the value by 0.5.

  Thus, first, the output RGB value y when the first light source 5-1 is on and the second light source 5-2 is off, and the second light source 5-2 is on and the first light source is on. The degree to which the sum of the output RGB value y when 5-1 is off is greater than the output RGB value y when both the first and second light sources 5-1 and 5-2 are on From the digital camera 6 when only the first light source 5-1 is on and RGB output from the digital camera 6 when only the second light source 5-2 is on. Estimate the amount of value. Then, the RGB value output from the digital camera 6 when both the first and second light sources 5-1 and 5-2 are turned on, calculated from the light amount obtained by adding the light amounts of these estimated RGB values. The calculated value is compared with the measured RGB values of the captured image of the object 9 actually captured by the digital camera 6 when both the first and second light sources 5-1 and 5-2 are on. Furthermore, the evaluation function using the least square method that compares the calculation model with the actual measurement model is represented by an evaluation function that squares the difference between the calculated value of the RGB value and the actual measurement value and adds all the square errors. Can be requested. By minimizing the evaluation function obtained in this way, the calculation model can be brought close to the correct actual measurement model.

  FIG. 7 is a diagram illustrating a simulation result of an example of the light amount distribution of the light source when it is predicted that γ = 0.46. In FIG. 7, the vertical axis represents the amount of light emitted from the light source, and the horizontal axis represents the x coordinate value and the y coordinate value on the xy plane in arbitrary units. 7, (a) is a light amount distribution of the first light source 5-1, (b) is a light amount distribution of 5-2 of the second light source, and (c) is the first and second light sources 5-1,. The light quantity distribution combining 5-2 is shown.

  FIG. 8 is a diagram illustrating a simulation result of an example of the output RGB value when it is predicted that γ = 0.46. In FIG. 8, the vertical axis represents RGB values output from the digital camera 6, and the horizontal axis represents x-coordinate values and y-coordinate values on the xy plane in arbitrary units. 8, (a) is an output RGB value when only the first light source 5-1 is on, (b) is an output RGB value when only the second light source 5-2 is on, and (c) is The output RGB values when both the first and second light sources 5-1 and 5-2 are on are shown.

In step S5 described above, the data processing unit 7 applies the values of the three RGB values (8a), (8b), and (8c) shown in (a), (b), and (c) in FIG. The value of γ is calculated by performing inverse gamma correction. For example, when the data processing unit 7 obtains a function that minimizes the sum of the values of the pixels represented by Σ [(8c) − {(8a) + (8b)}] ² by the simplex method, the value of γ Was estimated to be 0.46, and it was confirmed that the gamma characteristic could be accurately estimated. Further, when the data processing unit 7 predicts that γ is a value other than 0.46 for other calculation models having different values of α, it is estimated that the value of γ is an expected value, and the gamma characteristic is It was confirmed that it can be estimated accurately.

In the above example, the data processing unit 7 estimates the maximum likelihood function for y = αx ^γ . However, the gamma characteristic is not limited to y = αx ^γ, and a linear function or a quadratic function may be used. good. In other words, the maximum likelihood function, function expressed by y = .alpha.x ^gamma, polygonal line function, and may be used at least one of quadratic function. Furthermore, the maximum likelihood function, function expressed by y = .alpha.x ^gamma, polygonal line function, and using two or more of the quadratic function, the function is evaluation function squared error is minimized as a function of the calculation model You may choose.

  FIG. 9 is a diagram illustrating an example of a polygonal line function. In FIG. 9, the vertical axis indicates the RGB values output from the digital camera 6, and the horizontal axis indicates the amount of light incident on the digital camera 6 in arbitrary units. The broken line function may include a multi-stage broken line.

  FIG. 10 is a diagram illustrating an example of a quadratic function. In FIG. 10, the vertical axis represents the RGB values output from the digital camera 6, and the horizontal axis represents the amount of light incident on the digital camera 6 in arbitrary units.

If the gamma characteristic is a function other than y = .alpha.x ^gamma as shown in Figure 9 or Figure 10 for example, it is possible to obtain the ML function if define this function. In addition, when a plurality of functions of a calculation model that are likely to be used in advance can be predicted, a plurality of functions that can be predicted are stored in a table, and for example, the function of the calculation model that minimizes the square error that is an evaluation function is estimated. What is necessary is just to select as a gamma characteristic.

  In the above example, the least square error is used as the evaluation function. However, the evaluation function is not limited to the least square error, and is not particularly limited as long as it is an evaluation function that can estimate the maximum likelihood function such as Euclidean distance. Not. Further, the method for obtaining the value of γ so that the least square error is minimized is not limited to the simplex method, and for example, the golden section method may be used. That is, as a method for obtaining the value of γ so that the least square error is minimized, at least one of the least square method, the Euclidean distance, the simplex, and the golden section method may be used. Furthermore, as a method for obtaining the value of γ so that the least square error is minimized, a method that uses two or more of the least square method, the Euclidean distance, the simplex, and the golden section method and adopts the method that minimizes the least square error. May be.

  FIG. 11 is a flowchart for explaining another example of gamma characteristic estimation processing. FIG. 11 shows a process when the expected function of the calculation model is a polygonal line as shown in FIG. In this example, the processing in FIG. 11 is executed by the data processing unit 7. When the gamma characteristic estimation process is started, in step S11, the data processing unit 7 sets the initial value of the inflection point of the RGB value output from the digital camera 6 to 150 and the slope ratio after bending to 0.5. . In step S <b> 12, the data processing unit 7 controls the switch 4 to be turned on and the switch 3 to be turned off, so that only the first light source 5-1 is turned on and the object 9 is photographed by the digital camera 6. In step S <b> 13, the data processing unit 7 controls the switch 4 to be turned off and the switch 3 to be turned on, so that only the second light source 5-2 is turned on and the object 9 is photographed by the digital camera 6. In step S <b> 14, the data processing unit 7 controls both the switches 3 and 4 to be turned on, thereby turning on both the first and second light sources 5-1 and 5-2. Shoot. In step S15, the data processor 7 starts from the initial value 150 of the inflection point of the RGB value and the slope ratio 0.5 after bending, and uses the simplex method so that the least square error is minimized. Thus, a polygonal line that is most likely when both the first and second light sources 5-1 and 5-2 are on is estimated, and the process ends.

  FIG. 12 is a diagram illustrating an example of a polygonal line function. In FIG. 12, the vertical axis represents RGB values output from the digital camera 6, and the horizontal axis represents the amount of light incident on the digital camera 6 in arbitrary units. In FIG. 12, P1 indicates the inflection point of the RGB value, and is 100, for example. P2 represents a portion for obtaining the inclination ratio after bending (that is, after the inflection point P1 of the RGB value), and the inclination at the portion P2 is, for example, half of the inclination to the inflection point P1 of the RGB value. .

  FIG. 13 is a diagram showing a simulation result of an example of output RGB values when the gamma characteristic is predicted to be the function of the polygonal line shown in FIG. 12 for the light amount distribution of the light source similar to FIG. In FIG. 12, the vertical axis represents RGB values output from the digital camera 6, and the horizontal axis represents x-coordinate values and y-coordinate values on the xy plane in arbitrary units. In FIG. 13, (a) is an output RGB value when only the first light source 5-1 is on, (b) is an output RGB value when only the second light source 5-2 is on, and (c) is The output RGB values when both the first and second light sources 5-1 and 5-2 are on are shown.

In step S15 described above, the data processing unit 7 applies the values of the three RGB values (13a), (13b), and (13c) shown in (a), (b), and (c) in FIG. The value of γ is calculated by performing inverse gamma correction. For example, when the data processing unit 7 obtains a function that minimizes the sum of the values of each pixel represented by Σ [(13c) − {(13a) + (13b)}] ² by the simplex method, The inflection point P1 is 100, and the slope at the portion P2 after bending is estimated to be half (= 0.5) of the slope of the RGB value up to the inflection point P1, and the gamma characteristic can be accurately estimated. Was confirmed. Further, when the data processing unit 7 predicts that the gradient at the inflection point P1 of the RGB value and the portion P2 after the bending is a value other than the above, the inflection point P1 of the RGB value and the portion P2 after the bending are performed. It was estimated that the value of the slope was an expected value, and it was confirmed that the gamma characteristic could be estimated accurately.

  Although the case where the gamma characteristic is a polygonal line function has been described with reference to FIGS. 11 to 13, the gamma characteristic can be similarly estimated even if the gamma characteristic is a quadratic function or the like. That is, the function of the calculation model can be calculated to estimate the gamma characteristic no matter what function is expected. When a plurality of functions of a calculation model that are likely to be used in advance can be predicted, for example, a function of a calculation model that minimizes a square error as an evaluation function may be selected as an estimated gamma characteristic. That is, when a plurality of predictable calculation model functions are stored in the table, for example, the gamma characteristic estimated by the data processing unit 7 in the process of FIG. 6, the gamma characteristic estimated in the process of FIG. For example, the gamma characteristic obtained by estimating the function of the calculation model that minimizes the square error, which is the evaluation function, may be selected as the final result estimated by the gamma characteristic estimation process. For example, in the case of the process of FIG. 6, in step S5, the value of γ that the data processing unit 7 makes most likely when both the first and second light sources 5-1 and 5-2 are turned on is a function of a plurality of calculation models. For example, a function of a calculation model that minimizes a square error as an evaluation function is selected as an estimated gamma characteristic. Similarly, in the case of the process of FIG. 11, for example, in step S15, the data processing unit 7 sets a polygonal line that becomes most likely when both the first and second light sources 5-1 and 5-2 are turned on to a plurality of calculation models. For example, a function of a calculation model that minimizes a square error as an evaluation function is selected as an estimated gamma characteristic.

  As described above, it was confirmed that if the calculation model can be predicted, the gamma characteristic of the digital camera 6 can be estimated based on at least three captured images.

  In the example shown in FIG. 7, it is assumed that there is a light quantity distribution. However, even if there is no light quantity distribution, if the reflectance of the measurement object is not uniform, the RGB value and the RGB value obtained by adding the light quantity are offset. The value of γ can be estimated. Even if the amount of light is uniform as in an integrating sphere, the value of γ, that is, the gamma characteristic can be estimated if the amount of light can be added in a plurality of stages by combining a plurality of light sources.

  FIG. 14 is a diagram illustrating a gamma characteristic estimation method according to a modification. In FIG. 14, the same parts as those in FIG. An adjustment unit 303 is provided in the gamma characteristic estimation apparatus 1 shown in FIG. The amount of light emitted from the second light source 5-2 is variable stepwise via the adjustment unit 303 and the switch 3. The adjustment unit 303 may be controlled by the digital camera 6 or the data processing unit 7, for example.

  In the case of this modification, after executing the processing of FIG. 6 or FIG. 11 once, the adjustment unit 303 increases or decreases the amount of light emitted from the second light source 5-2 by a certain amount, and the same processing of FIG. Try again. As a result, the gamma characteristic can be estimated based on the amount of change in the output RGB value of the digital camera 6 that occurs in response to a certain amount of change in the amount of light emitted from the second light source 5-2. Note that the number of times to re-execute the processing of FIG. 6 or FIG. 11 is not particularly limited.

  FIG. 15 is a block diagram illustrating a first example of a gamma characteristic estimation apparatus. In FIG. 15, the smartphone 31-1 is communicably connected to a personal computer (PC) 41 via a wired or wireless interface. On / off control of a light emitting diode (LED: Light Emitting Diode) 5-1 that is an example of a first light source is performed by a user operation, for example, and on / off control of an LED 5-2 that is an example of a second light source. Is performed, for example, by a user operation. The camera 6 of the smartphone 31-1 captures the object 9 by receiving reflected light from the object 9 (not shown) irradiated with the light from the LEDs 5-1 and 5-2. The shooting timing of the camera 6 can be determined by a user operation, for example. The output data of the camera 6 is output to the data processing unit 7 in the PC 41. In this example, for example, when the user manually controls the LEDs 5-1 and 5-2 and captures with the camera 6 of the smartphone 31-1, the data processing unit 7 in the PC 41 has the gamma characteristic shown in FIG. 6 or FIG. Perform estimation processing.

  FIG. 16 is a block diagram illustrating an example of a smartphone. In FIG. 16, a smartphone 31 that is an example of a first electronic device includes a CPU (Central Processing Unit) 311, a storage unit 312, a communication unit 313, an input unit 314, a display unit 315, and a camera 6 connected via a bus 320. Have a configuration. The battery 316 is an example of the power supply 2 shown in FIG. 1 and supplies a power supply voltage to each part in the smartphone 31. The storage unit 312 may store a program executed by the CPU 311, various data used in the calculation executed by the CPU 311, intermediate results of the calculation executed by the CPU 311, a table storing functions of a plurality of predictable calculation models, and the like. . The communication unit 313 enables communication with an external device such as a PC via a wired or wireless interface 330. The input unit 314 may include a keyboard, for example. The display unit 315 may display a message for the user, a gamma characteristic estimated by the gamma characteristic estimation process, and the like. The input unit 314 and the display unit 315 may be integrated as a touch panel. The connection between the CPU 311 and each unit in the smartphone 31 is not limited to the connection via the bus 320.

  FIG. 17 is a block diagram illustrating an example of a personal computer (PC). In FIG. 17, a PC 41 that is an example of a second electronic device has a configuration in which a CPU 411, a storage unit 412, a communication unit 413, an input unit 414, and a display unit 415 are connected by a bus 420. The PC 41 may further include a camera 6 and may further include a battery (not shown). The storage unit 412 may store a program executed by the CPU 411, various data used in the calculation executed by the CPU 411, an intermediate result of the calculation executed by the CPU 411, a table storing a plurality of predictable calculation model functions, and the like. . The communication unit 413 enables communication with an external device such as a smartphone via a wired or wireless interface 430. The input unit 414 may include a keyboard, for example. The display unit 415 may display a message for the user, a gamma characteristic estimated by the gamma characteristic estimation process, and the like. The input unit 414 and the display unit 415 may be integrated as a touch panel. The connection between the CPU 411 and each unit in the PC 41 is not limited to the connection via the bus 420.

  In the case of the PC 41 shown in FIG. 15, the function of the data processing unit 7 can be realized by the CPU 411 shown in FIG. Moreover, you may implement | achieve some functions of the data processing part 7 by CPU311 in the smart phone 31 shown in FIG. In the example shown in FIG. 15, the gamma characteristic estimation process can be executed by executing a program stored in the storage unit 412, which is an example of a computer-readable storage medium, by the CPU 411 in the PC 41.

  FIG. 18 is a block diagram illustrating a second example of the gamma characteristic estimation apparatus. In FIG. 18, the same parts as those in FIG. 15 are denoted by the same reference numerals, and the description thereof is omitted. In FIG. 18, the smartphone 31-2 includes switches 3 and 4 and a camera 6. Therefore, the control of the switches 3 and 4 and the control of the photographing timing of the camera 6 can be executed by the CPU 311 shown in FIG. The function of the data processing unit 7 can be realized by the CPU 411 in the PC 41 shown in FIG. Further, some functions of the data processing unit 7 may be realized by the CPU 311 shown in FIG. In the example shown in FIG. 18, the gamma characteristic estimation process can be executed by executing a program stored in the storage unit 412, which is an example of a computer-readable storage medium, by the CPU 411 in the PC 41.

  FIG. 19 is a block diagram illustrating a third example of the gamma characteristic estimation apparatus. In FIG. 19, the same parts as those in FIG. In FIG. 19, the smartphone 31-2 is connected to the cloud computing system 51 via a wired or wireless interface such as the Internet so as to be communicable. The function of the data processing unit 7 can be realized by one or more computers in the cloud computing system 51 which is an example of the second electronic device. Further, some functions of the data processing unit 7 may be realized by the CPU 311 shown in FIG. In the case of the example shown in FIG. 19, the gamma characteristic estimation process can be executed by one or more computers in the cloud computing system 51 executing a program stored in a computer-readable storage medium.

  FIG. 20 is a block diagram illustrating a fourth example of the gamma characteristic estimation apparatus. In FIG. 20, the same parts as those in FIG. 15 are denoted by the same reference numerals, and the description thereof is omitted. In FIG. 20, the smartphone 31-3 includes switches 3 and 4, a camera 6, and a data processing unit 7. Therefore, the control of the switches 3 and 4, the function of the data processing unit 7, and the control of the photographing timing of the camera 6 can be executed by the CPU 311 shown in FIG. In the case of the example shown in FIG. 20, the gamma characteristic estimation process can be executed by the CPU 311 executing a program stored in the storage unit 312 which is an example of a computer-readable storage medium.

  FIG. 21 is a block diagram illustrating a fifth example of the gamma characteristic estimation apparatus. In FIG. 21, the same parts as those in FIG. In FIG. 21, the smartphone 31-4 further includes LEDs 5-1 and 5-2.

  FIG. 22 is a block diagram illustrating a sixth example of the gamma characteristic estimation apparatus. In FIG. 22, the same parts as those in FIG. In FIG. 22, the smartphone 31-4 displays an illumination display 15-1 that functions as a first light source and an illumination display 15-2 that functions as a second light source on the display unit 315. The displays 15-1 and 15-2 displayed on the display unit 315 can be turned on / off by being controlled by the CPU 311 shown in FIG. For example, the display 15-1 and the display 15-2 may be displayed in an area on the display unit 315 that is a certain distance away from each other.

  In the case of the example shown in FIGS. 15 and 18, since the gamma characteristic estimation process is executed by the CPU 411 on the PC 41 side, the load on the CPU 311 on the smartphone 31-1, 31-2 side can be reduced. In the case of the example shown in FIG. 19, since the gamma characteristic estimation process is executed by the computer on the cloud computing system 51 side, the load on the CPU 311 on the smartphone 31-2 side can be reduced.

  In the case of the examples shown in FIGS. 20 to 22, control of switches 3 and 4 and control of shooting timing of the camera 6 are realized by one smartphone 31-3 to 31-5 and gamma characteristic estimation processing is executed. Can do. In addition, the smartphones 31-3 to 31-5 illustrated in FIGS. 20 to 22 can be replaced with electronic devices such as a tablet computer and a personal computer (PC) having the same configuration. However, an electronic apparatus such as a tablet computer or a PC in this case needs to include the camera 6. Furthermore, in the case of the example shown in FIG. 22, since the display unit 315 of the smartphone 31-5 is used as the first and second light sources, it is not necessary to provide two additional light sources.

  18 to 22, when the adjustment unit 303 of FIG. 14 is used, a combination of the adjustment unit 303 and the switch 3 may be provided instead of the switch 3.

  By the way, when the object is irradiated with light from three or more directions, the normal vector is obtained from the shaded state of the object, and the shape of the object is reproduced, for example, a photometric stereo method is used using a digital camera. May be measured. Since the photometric stereo method uses at least three light sources, it is suitable for applying the gamma characteristic estimation method in the above-described embodiments and modifications. By using two light sources among the three or more light sources used in the photometric stereo method in the gamma characteristic estimation method, accurate gamma characteristics are estimated without preparing a new light source for gamma characteristic estimation. It becomes possible.

  According to the embodiment and the modification described above, the gamma characteristic can be estimated relatively easily. Specifically, a photographed image when only the first light source is on, a photographed image when only the second light source 5-2 is on, and a photograph when both the first and second light sources are on. Based on the image, it is possible to estimate the gamma characteristic. In addition, according to the above-described embodiments and modifications, it is not necessary to use a light source whose accurate light amount is known, and calibration can be performed relatively easily.

The following additional notes are further disclosed with respect to the embodiment including the above examples.
(Appendix 1)
A camera for photographing an object irradiated with light from the first and second light sources;
A controller for controlling on / off of the first and second light sources;
When the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are on A data processing unit that applies an output value of the camera when the object is photographed to an expected function and estimates a gamma characteristic from a function in which the amount of light obtained from the inverse function of the expected function is most likely A gamma characteristic estimation device characterized by that.
(Appendix 2)
The gamma characteristic according to claim 1, wherein at least one of least square method, Euclidean distance, simplex, and golden section method is used to estimate the minimum error for obtaining the most likely function. Estimating device.
(Appendix 3)
As a function to be the expected amount of incident light x and the output value y of the camera, the function expressed by y = .alpha.x ^gamma when indicated by constant alpha, polygonal line function, and the use of at least one of quadratic function The gamma characteristic estimation apparatus according to appendix 1 or 2, characterized by:
(Appendix 4)
A storage unit storing a plurality of the expected functions;
The data processing unit selects a gamma characteristic obtained by estimating a function of a calculation model that minimizes a square error, which is an evaluation function, as a final result estimated by the gamma characteristic estimation process. The gamma characteristic estimation apparatus of any one of Claims.
(Appendix 5)
5. The gamma characteristic estimation apparatus according to claim 1, further comprising a single processor that functions as the control unit and the data processing unit.
(Appendix 6)
The gamma characteristic estimation apparatus according to any one of appendices 1 to 5, further comprising the first and second light sources.
(Appendix 7)
The gamma characteristic estimation apparatus according to appendix 6, further comprising a display unit that functions as the first and second light sources.
(Appendix 8)
The control unit is provided in a first electronic device having a communication function,
The gamma characteristic estimation device according to any one of appendices 1 to 7, wherein the data processing unit is provided in a second electronic device capable of communicating with the first electronic device.
(Appendix 9)
The gamma characteristic estimation device according to appendix 8, wherein the second electronic device is provided in a cloud computing system.
(Appendix 10)
Photographing an object irradiated with light from the first and second light sources with a camera,
The controller controls on / off of the first and second light sources,
In the data processor, the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are Applying the output value of the camera when the object is photographed in the ON state to an expected function, and estimating the gamma characteristic from the function that the light quantity obtained from the inverse function of the expected function is most likely A gamma characteristic estimation method characterized by
(Appendix 11)
The gamma characteristic according to appendix 10, wherein at least one of least square method, Euclidean distance, simplex, and golden section method is used to estimate a minimum error for obtaining the most likely function. Estimation method.
(Appendix 12)
As a function to be the expected amount of incident light x and the output value y of the camera, the function expressed by y = .alpha.x ^gamma when indicated by constant alpha, polygonal line function, and the use of at least one of quadratic function The gamma characteristic estimation method according to appendix 10 or 11, wherein
(Appendix 13)
A plurality of the expected functions are stored in the storage unit,
Any one of appendices 10 to 12, wherein a gamma characteristic obtained by estimating a function of a calculation model that minimizes a square error as an evaluation function in the data processing unit is selected as a final result estimated by the gamma characteristic estimation process. 2. The gamma characteristic estimation method according to claim 1.
(Appendix 14)
14. The gamma characteristic estimation method according to any one of appendices 10 to 13, wherein the functions of the control unit and the data processing unit are realized by a single processor.
(Appendix 15)
The gamma characteristic estimation method according to any one of appendices 10 to 14, wherein the functions of the first and second light sources are realized by a display unit.
(Appendix 16)
Realizing the function of the control unit in a first electronic device having a communication function;
The gamma characteristic estimation method according to any one of appendices 10 to 15, wherein the function of the data processing unit is realized in a second electronic device capable of communicating with the first electronic device.
(Appendix 17)
The gamma characteristic estimation method according to appendix 16, wherein the second electronic device is provided in a cloud computing system.
(Appendix 18)
A program for causing a computer to execute gamma characteristic estimation processing,
A procedure for photographing an object irradiated with light from the first and second light sources with a camera;
A procedure for controlling on / off of the first and second light sources;
When the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are on Applying the output value of the camera when the object is photographed to an expected function, and estimating the gamma characteristic from the function that makes the light quantity obtained from the inverse function of the expected function most likely A program characterized by being executed.
(Appendix 19)
The program according to appendix 18, wherein at least one of least square method, Euclidean distance, simplex, and golden section method is used to estimate the minimum error for obtaining the most likely function.
(Appendix 20)
As a function to be the expected amount of incident light x and the output value y of the camera, the function expressed by y = .alpha.x ^gamma when indicated by constant alpha, polygonal line function, and the use of at least one of quadratic function The program according to appendix 18 or 19, characterized by:
(Appendix 21)
A plurality of the expected functions are stored in the storage unit,
The appending steps 18 to 20 are characterized in that the estimating step selects a gamma characteristic obtained by estimating a function of a calculation model that minimizes a square error as an evaluation function as a final result estimated by the gamma characteristic estimation process. The program according to any one of the above.
(Appendix 22)
The program according to any one of appendices 18 to 21, further causing the computer to execute a procedure for causing a display unit to function as the first and second light sources.

  Although the disclosed gamma characteristic estimation apparatus, method, and program have been described with the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and improvements can be made within the scope of the present invention. Needless to say.

DESCRIPTION OF SYMBOLS 1 Gamma characteristic estimation apparatus 2 Power supply 3, 4 Switch 5-1 1st light source 5-2 2nd light source 6 Digital camera 7 Data processing part 15-1, 15-2 Display 31 Smartphone 41 PC
51 Cloud Computing System 303 Adjustment Units 311 and 411 CPU
312 and 412 storage units 315 and 415 display units

Claims (6)

A camera for photographing an object irradiated with light from the first and second light sources;
A controller for controlling on / off of the first and second light sources;
When the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are on A data processing unit that applies an output value of the camera when the object is photographed to an expected function and estimates a gamma characteristic from a function in which the amount of light obtained from the inverse function of the expected function is most likely A gamma characteristic estimation device characterized by that. Regarding the output value of the camera, the pixel value when the first light source is on and the second light source is off from the pixel value when the first and second light sources are on. And the least-squares function , where the sum of the squares of the values obtained by subtracting the sum of pixel values when the first light source is off and the second light source is on is minimized. Euclidean distance, simplex, and wherein Rukoto determined using at least one of golden section, the gamma characteristic estimation apparatus according to claim 1. As a function to be the expected amount of incident light to the camera x, y the output value of the camera, the constant alpha, the gamma characteristics in gamma tables list, function expressed by y = .alpha.x ^gamma, polygonal line The gamma characteristic estimation apparatus according to claim 1, wherein at least one of a function and a quadratic function is used. A storage unit storing a plurality of the expected functions;
The data processing unit, a gamma characteristic value of the evaluation function adding the squared error to estimate the function of calculation model to minimize the difference the squared output values and the value of the expected function of the camera, gamma 4. The gamma characteristic estimation apparatus according to claim 1, wherein the gamma characteristic estimation apparatus is selected as a final result estimated by the characteristic estimation process. Photographing an object irradiated with light from the first and second light sources with a camera,
The controller controls on / off of the first and second light sources,
In the data processor, the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are Applying the output value of the camera when the object is photographed in the ON state to an expected function, and estimating the gamma characteristic from the function that the light quantity obtained from the inverse function of the expected function is most likely A gamma characteristic estimation method characterized by A program for causing a computer to execute gamma characteristic estimation processing,
A procedure for photographing an object irradiated with light from the first and second light sources with a camera;
A procedure for controlling on / off of the first and second light sources;
When the first light source is on and the second light source is off, the first light source is off and the second light source is on, and the first and second light sources are on Applying the output value of the camera when the object is photographed to an expected function, and estimating the gamma characteristic from the function that makes the light quantity obtained from the inverse function of the expected function most likely A program characterized by being executed.