JP6790970B2 - Image correction device, imaging system, control method and program - Google Patents

Description

The present invention relates to an image correction device, an imaging system, a control method and a program.

When an image of an object is output, the gradation value of the image differs depending on the surrounding environment in which the image pickup device is placed. There are various techniques to correct for differences in such images.
Patent Document 1 describes, as a related technique, a technique for correcting an image signal based on a correction result in a previous frame.

Japanese Unexamined Patent Publication No. 2016-134908

By the way, even when the image pickup apparatus is in an environment where the temperature changes, the gradation value of the image is different. At this time, even if the instantaneous temperatures are the same, the deviation amounts are not always the same.
Therefore, there has been a demand for a technique capable of correctly correcting the gradation value of an image when the image pickup apparatus is in an environment where the temperature changes.

An object of the present invention is to provide an image correction device, an imaging system, a control method and a program capable of solving the above problems.

In order to achieve the above object, the aspects of the present invention include a temperature acquisition unit that acquires the temperature of the image pickup device, an image acquisition unit that acquires an image from the image pickup device, and a timing prior to the acquisition of the image. An image correction device including a correction unit that corrects the image based on the transition of the temperature acquired by the temperature acquisition unit during the period up to the timing of imaging.

Another aspect of the present invention is an imaging system including the above-mentioned image correction device and an imaging device that outputs an captured image to the image correction device.

Further, another aspect of the present invention is to acquire the temperature of the imaging device, to acquire an image from the imaging device, and to acquire the image in a period from a timing before the imaging of the image to the timing of the imaging. It is a control method including correcting the image based on the transition of the temperature.

Further, another aspect of the present invention is to acquire the temperature of the image pickup apparatus by a computer, acquire an image from the image pickup apparatus, and from a timing before the image pickup to the image pickup timing. It is a program for correcting the image and executing the image based on the transition of the temperature acquired during the period.

According to the present invention, when the image pickup apparatus is in an environment where the temperature changes, the gradation value of the image can be corrected correctly.

It is a figure which shows the structure of the imaging system by one Embodiment of this invention. It is a figure which shows the example of the relationship between the temperature and the gradation value in one Embodiment of this invention. It is the first figure which shows the processing flow of the imaging system by one Embodiment of this invention. It is a 2nd figure which shows the processing flow of the imaging system by one Embodiment of this invention. It is a figure which shows the example of the gradation correction value in one Embodiment of this invention. It is a figure which shows the minimum structure of the image correction apparatus by embodiment of this invention. It is a schematic block diagram which shows the structure of the computer which concerns on at least one Embodiment.

<Embodiment>
Hereinafter, embodiments will be described in detail with reference to the drawings.
The configuration of the imaging system 1 according to the embodiment of the present invention will be described.
As shown in FIG. 1, the image pickup system 1 according to an embodiment of the present invention includes an image pickup device 10 and an image correction device 20.

The image pickup device 10 is a device that acquires an image of the object O to be captured. As shown in FIG. 1, the image pickup device 10 includes an image pickup element 101, an A / D conversion unit 102, and a temperature measurement unit 103.

The image sensor 101 captures an image of the object O. The image sensor 101 outputs the image information of the captured object O to the A / D conversion unit 102.

The A / D conversion unit 102 acquires the image information of the object O imaged by the image sensor 101. The A / D conversion unit 102 converts the acquired analog image information into digital image information.
The temperature measuring unit 103 measures the temperature of the imaging device 10. The temperature measuring unit 103 measures, for example, the temperature of the image sensor 101 or the A / D conversion unit 102.
The temperature change of the image sensor 101 is one that directly affects the gradation value of the image. Therefore, when the temperature measuring unit 103 measures the temperature of the image sensor 101, it is considered that the image pickup system 1 can more accurately correct the gradation value of the image. Further, when the temperature measuring unit 103 cannot directly measure the temperature of the image sensor 101 but measures the temperature of the A / D conversion unit 102 provided at a position affected by the temperature of the image sensor 101, the image sensor 1 uses A. Since the / D conversion unit 102 is greatly affected by the temperature of the image sensor 101 and changes in conjunction with the temperature, it is considered that the gradation value of the image can be accurately corrected without directly measuring the temperature of the image sensor 101.
The image pickup apparatus 10 is generally provided with a temperature sensor for avoiding operation outside the guaranteed operation temperature range, for example, on the side of the A / D conversion unit 102. The temperature measuring unit 103 may utilize the temperature sensor.

The image correction device 20 is a device that corrects the gradation value of an image. As shown in FIG. 2, even if the temperature of the image pickup apparatus 10 is the same, the gradation value of the image is not always the same. For example, when the temperature of the image pickup apparatus 10 (camera temperature in FIG. 2) is 36 degrees, in FIG. 2A, the gradation value of R is about 227, the gradation value of G is about 226, and the gradation value of B is about 226. Is about 228. On the other hand, in FIG. 2B, the gradation value of R is about 225, the gradation value of G is about 225, and the gradation value of B is about 226. The acquisition of the temperature transition of the image pickup device 10 by the image correction device 20 is not the instantaneous temperature measurement result of one measurement point of the temperature measurement unit 103, but the temperature of one measurement point of the temperature measurement unit 103. It means that the continuous temperature measurement results including multiple affected points have been obtained. As a result, the image correction device 20 corrects the gradation value of the image with a gradation correction value including the influence of more accurate temperature.
As shown in FIG. 1, the image correction device 20 includes an image acquisition unit 201, a temperature acquisition unit 202, a correction unit 203, and a storage unit 204.

The image acquisition unit 201 acquires an image from the image pickup device 10.
The temperature acquisition unit 202 acquires the temperature of the image pickup device 10. Specifically, the temperature acquisition unit 202 acquires the temperature measured by the temperature measurement unit 103.
The correction unit 203 corrects the image based on the transition of the temperature of the image pickup device 10 acquired by the temperature acquisition unit 202 during the period from the timing before the image acquisition of the image pickup device 10 to the timing of the image pickup. Specifically, the correction unit 203 corrects the gradation value of the image that changes with the change in the temperature of the image pickup device 10.
The storage unit 204 stores correction information indicating the relationship between the temperature transition of the image pickup apparatus 10 and the correction value of the gradation value of the image.

Next, the processing of the imaging system 1 according to the embodiment of the present invention will be described.
Here, a correction data storage process for accumulating correction data in which the imaging system 1 uses the gradation value of the image for correction and a gradation value correction process in which the imaging system 1 corrects the gradation value of the image will be described.

(Correction data storage process)
First, the correction data storage process performed by the imaging system 1 will be described.
Here, the processing flow of the imaging system 1 shown in FIG. 3 will be described.
The correction data storage process performed by the imaging system 1 is a process for acquiring in advance the correspondence between the temperature transition and the gradation value as shown in FIGS. 2A and 2B.

It is set to an arbitrary initial temperature of the image pickup apparatus 10 (step S1). For example, the imaging device 10 is placed in a constant temperature bath set to a desired temperature (that is, an initial temperature, for example, the temperature indicated by T0 in FIG. 2). The temperature measuring unit 103 acquires the temperature of the imaging device 10. Wait until the temperature acquired by the temperature measuring unit 103 reaches the initial temperature (for example, about one hour to one and a half hours).

The setting of the constant temperature bath is changed to an arbitrary temperature (for example, the temperature indicated by T1 in FIG. 2) (step S2).

The image sensor 101 images a color sample (step S3). A color swatch is a swatch that shows a color standard consisting of a plurality of colors (for example, white, purple, yellow, etc.) indicated by the ratio of RGB (Red, Green, Blue). The image sensor 101 constantly outputs the image information of the captured object O to the A / D conversion unit 102.

The A / D conversion unit 102 acquires analog image information indicating an analog image of a color sample imaged by the image sensor 101 at a predetermined timing (for example, every minute). The A / D conversion unit 102 converts the acquired analog image information into digital image information indicating a digital image. The A / D conversion unit 102 outputs the converted digital image information to the image correction device 20.

Further, the temperature measuring unit 103 constantly acquires the temperature of the imaging device 10 (step S4). The temperature measuring unit 103 outputs the acquired temperature to the image correction device 20.

The image acquisition unit 201 acquires digital image information from the image pickup device 10. The image acquisition unit 201 calculates the average gradation value for each color of the color sample image indicated by the acquired digital image information (step S5). The average gradation value is an average value of gradation values indicated by each of a plurality of pixels showing the same color acquired by the image acquisition unit 201. The image acquisition unit 201 outputs the calculated average gradation value to the correction unit 203.

The temperature acquisition unit 202 acquires the temperature of the image pickup apparatus 10 from the temperature measurement unit 103 at a predetermined timing (for example, every minute). The temperature acquisition unit 202 outputs the temperature information of the image pickup device 10 to the correction unit 203.

The correction unit 203 acquires the average gradation value and the temperature information of the image pickup apparatus 10 at a predetermined timing (for example, every minute). The correction unit 203 writes the gradation value information in which the acquired average gradation value and the temperature information of the image pickup apparatus 10 are associated with each other in the storage unit 204 (step S6).

The temperature acquisition unit 202 determines whether or not the acquired temperature is the same as the previously acquired temperature within an error range (step S7).
The temperature acquisition unit 202 determines that the acquired temperature is not the same as the previously acquired temperature within the margin of error (for example, the difference between the acquired temperature and the previously acquired temperature is not less than or equal to the threshold value indicating the error). If this is the case (NO in step S7), the process returns to step S3. Each time the temperature acquisition unit 202 returns to the process of step S3, for example, the temperature information shown as T2, T3, T4 in FIG. 2 increases, and the average gradation value information associated with the temperature increases. To increase.
Further, when the temperature acquired by the temperature acquisition unit 202 is the same as the temperature acquired last time within the range of the error (for example, the difference between the acquired temperature and the temperature acquired last time is equal to or less than the threshold value indicating the error). When the determination is made (YES in step S7), the correction unit 203 is shown in FIG. 5, for example, based on the gradation value indicated by the gradation value information stored in the storage unit 204 and the gradation value of the color sample. As described above, the correspondence between each color of the color sample and the gradation correction value indicating the correction amount of the gradation value is generated for each RGB in association with the temperature transition (step S8). Specifically, the correction unit 203 uses, for example, the difference between the gradation value indicated by the gradation value information stored in the storage unit 204 and the gradation value of each color of the color sample as the gradation correction value.
The correspondence between each color of the color sample and the gradation correction value for each RGB is not limited to that shown in FIG. For example, the correspondence between each color of the color sample and the gradation correction value for each RGB may be indicated by a function. Further, the correspondence between each color of the color sample and the gradation correction value for each RGB may be updated by machine learning or the like.
The correction unit 203 associates the generated correspondence with the temperature transition and writes it to the storage unit 204 for each RGB (step S9).

(Gradation value correction processing)
Next, the gradation value correction process performed by the imaging system 1 will be described.
Here, the processing flow of the imaging system 1 shown in FIG. 4 will be described.

The image pickup device 10 is activated. The temperature measuring unit 103 constantly acquires the temperature of the imaging device 10 (step S11). The temperature measuring unit 103 outputs the acquired temperature to the image correction device 20.

The temperature acquisition unit 202 acquires the temperature of the image pickup apparatus 10 from the temperature measurement unit 103 at a predetermined timing (for example, every minute). The temperature acquisition unit 202 outputs the temperature information of the image pickup device 10 to the correction unit 203.

The correction unit 203 acquires the temperature information of the image pickup apparatus 10 from the temperature acquisition unit 202 at a predetermined timing (for example, every minute). The correction unit 203 writes the acquired temperature information of the imaging device 10 in the storage unit 204 (step S12).

When a predetermined time (for example, one hour to one and a half hours) has elapsed since the image pickup device 10 was started, the image pickup device 10 is in a state where it can take an image.
The correction unit 203 determines whether or not to acquire digital image information from the image acquisition unit 201 (step S13).
When the correction unit 203 determines that the digital image information has not been acquired from the image acquisition unit 201 (NO in step S13), the correction unit 203 returns to the process of step S11.

In such a state, the user performs an operation of photographing the image pickup apparatus 10 at an arbitrary timing.
The image sensor 101 images the object O (step S13). The image sensor 101 outputs the image information of the captured object O to the A / D conversion unit 102.

The A / D conversion unit 102 acquires analog image information of the object O from the image sensor 101. The A / D conversion unit 102 converts the acquired analog image information into digital image information. The A / D conversion unit 102 outputs digital image information to the image correction device 20.

The image acquisition unit 201 acquires digital image information from the image pickup device 10. The image acquisition unit 201 outputs the acquired digital image information to the correction unit 203.

In this case, the correction unit 203 acquires digital image information from the image acquisition unit 201 (YES in step S13), and in the temperature transition stored in the storage unit 204, the temperature transition and error obtained from the temperature acquisition unit 202. The same temperature transition is specified within the range (step S14). Specifically, for example, the correction unit 203 uses a numerical analysis method such as a mean square error to specify changes in temperature that can be regarded as the same within a predetermined error range in the corresponding relationship. In addition, the correction unit 203 according to another embodiment of the present invention may specify the temperature transition in the corresponding relationship with respect to the one whose temperature transition is expanded and contracted in the time axis direction and the temperature axis direction.
The correction unit 203 identifies the correspondence generated by the correction unit 203 associated with the temperature transition specified in the storage unit 204 (step S15).
The correction unit 203 corrects the gradation value of the image indicated by the digital image information by using the gradation correction value indicated by the specified correspondence (step S16).

The image pickup system 1 according to the embodiment of the present invention has been described above. In the image correction device 20 according to the embodiment of the present invention, the temperature acquisition unit 202 acquires the temperature of the image pickup device 10. The image acquisition unit 201 acquires an image from the image pickup device 10. The correction unit 203 is the image pickup device 10 acquired by the temperature acquisition unit 202 during the period from the timing before the image acquisition (that is, the timing retroactive from the timing of starting the image pickup to the timing of a certain period of time) to the timing of the image pickup. The image is corrected based on the transition of temperature.
By doing so, the image correction device 20 can correctly correct the gradation value of the image when the image pickup device is in an environment where the temperature changes.

Next, the image correction device 20 having the minimum configuration according to the embodiment of the present invention will be described.
As shown in FIG. 6, the image correction device 20 having the minimum configuration according to the embodiment of the present invention includes an image acquisition unit 201, a temperature acquisition unit 202, and a correction unit 203.
The temperature acquisition unit 202 acquires the temperature of the image pickup device 10.
The image acquisition unit 201 acquires an image from the image pickup device 10.
The correction unit 203 corrects the image based on the temperature transition of the image pickup device 10 acquired by the temperature acquisition unit 202 during the period from the timing before the image acquisition to the image acquisition timing.
In this way, the image correction device 20 can correctly correct the gradation value of the image when the image pickup device is in an environment where the temperature changes.

In the processing according to the embodiment of the present invention, the order of the processing may be changed as long as the appropriate processing is performed.

The storage unit 204, each of the other storage units and storage devices (including registers and latches) in the embodiment of the present invention may be provided anywhere as long as appropriate information is transmitted and received. Further, each of the storage unit 204, the other storage units, and the storage device may exist in a plurality of units within a range in which appropriate information is transmitted and received, and the data may be distributed and stored.

Although the embodiment of the present invention has been described, the above-mentioned image pickup system 1, image pickup device 10, image correction device 20, and other control devices may have a computer system inside. The process of the above-mentioned processing is stored in a computer-readable recording medium in the form of a program, and the above-mentioned processing is performed by the computer reading and executing this program. A specific example of a computer is shown below.
FIG. 7 is a schematic block diagram showing a configuration of a computer according to at least one embodiment.
As shown in FIG. 7, the computer 5 includes a CPU 6, a main memory 7, a storage 8, and an interface 9.
For example, each of the above-mentioned image pickup system 1, image pickup device 10, image correction device 20, and other control devices is mounted on the computer 5. The operation of each processing unit described above is stored in the storage 8 in the form of a program. The CPU 6 reads a program from the storage 8, expands it into the main memory 7, and executes the above processing according to the program. Further, the CPU 6 secures a storage area corresponding to each of the above-mentioned storage units in the main memory 7 according to the program.

Examples of the storage 8 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versaille Disk Read). , Semiconductor memory and the like. The storage 8 may be internal media directly connected to the bus of computer 5, or external media connected to computer 5 via an interface 9 or a communication line. When this program is distributed to the computer 5 via a communication line, the distributed computer 5 may expand the program in the main memory 7 and execute the above processing. In at least one embodiment, the storage 8 is a non-temporary tangible storage medium.

Further, the above program may realize a part of the above-mentioned functions. Further, the program may be a file that can realize the above-mentioned functions in combination with a program already recorded in the computer system, that is, a so-called difference file (difference program).

Although some embodiments of the present invention have been described, these embodiments are examples and do not limit the scope of the invention. These embodiments may be subject to various additions, various omissions, various replacements, and various modifications without departing from the gist of the invention.

1 ... Imaging system 5 ... Computer 6 ... CPU
7 ... Main memory 8 ... Storage 9 ... Interface 10 ... Image pickup device 20 ... Image correction device 101 ... Image sensor 102 ... A / D converter 103 ... Temperature measurement Unit 201 ... Image acquisition unit 202 ... Temperature acquisition unit 203 ... Correction unit 204 ... Storage unit

Claims (6)

A temperature acquisition unit that acquires the temperature of the image pickup device,
An image acquisition unit that acquires an image from the image pickup device,
A correction unit that corrects the image based on the transition of the temperature acquired by the temperature acquisition unit during the period from the timing prior to the imaging of the image to the timing of the imaging.
An image correction device comprising. A temperature measuring unit provided at a location away from the image sensor in the image pickup device,
With
The temperature acquisition unit
Acquires the temperature detected by the temperature measuring unit,
The image correction device according to claim 1. The correction unit
The correction amount of the image is specified based on the transition of the temperature, and the image is corrected using the specified correction amount.
The image correction device according to claim 1 or 2. The image correction device according to any one of claims 1 to 3,
An image pickup device that outputs the captured image to the image correction device, and
Imaging system with. Acquiring the temperature of the image pickup device and
Acquiring an image from the image pickup device and
Correcting the image based on the transition of the temperature acquired in the period from the timing prior to the imaging of the image to the timing of the imaging.
Control methods including. On the computer
Acquiring the temperature of the image pickup device and
Acquiring an image from the image pickup device and
Correcting the image based on the transition of the temperature acquired in the period from the timing prior to the imaging of the image to the timing of the imaging.
A program that executes.

Images