JP7555748B2 - Image processing device and image processing program - Google Patents

Description

The present invention relates to an image processing device and an image processing program.

Conventionally, there is known a technique for capturing images of the wall surface of a structure using an infrared camera or the like, and then inspecting the tile wall surface using the thermal image obtained.

For example, a technique is known that automatically switches from the current measurement temperature range to a measurement temperature range whose upper limit temperature is higher than the upper limit temperature of the current measurement temperature range and is closest to the upper limit temperature of the current measurement temperature range (Patent Document 1).

There is also known a technology that specifies a specific area of a display image, detects the maximum and minimum temperatures in the specified area, and displays an infrared image using the difference between the detected maximum and minimum temperatures as a temperature range (Patent Document 2).

A technology is also known that displays floating tiles by adding color information to a target pixel that is related to the temperature difference between the temperature indicated by the temperature data of the target pixel included in a certain area of a thermal image and a reference temperature indicated by the temperature data of another reference pixel other than the target pixel (Patent Document 3).

There is also known technology that acquires output data from an infrared camera to generate a thermal image of the tunnel wall and detects areas of the tunnel wall with abnormal temperatures (Patent Document 4).

Japanese Patent Application Publication No. 9-101207 Japanese Unexamined Patent Publication No. 62-039737 JP 2012-221235 A JP 2016-223796 A

When detecting loose tiles from thermal images, it is necessary to recognize the tiles from the thermal images. Because the temperature distribution in thermal images varies, it is difficult to accurately recognize tiles unless an appropriate temperature range is set according to the thermal image. For this reason, it is desirable to set an appropriate temperature range according to the thermal image.

For example, the above-mentioned Patent Documents 1 and 2 disclose a technology for setting a temperature range for detecting significant temperature differences in infrared images. However, neither technology is applicable to the detection of loose tiles.

Furthermore, the above-mentioned Patent Document 3 discloses a technique for manually setting the temperature range of a thermal image when detecting loose tiles. However, with the technique described in the above-mentioned Patent Document 3, since the setting is manual, it may not be possible to set an appropriate temperature range. Furthermore, the technique described in the above-mentioned Patent Document 4 does not take into consideration setting an appropriate temperature range according to the thermal image.

Therefore, the present invention aims to set an appropriate temperature range when detecting loose tiles from thermal images of a tiled wall surface.

In order to achieve the above object, the image processing device according to the present invention includes an acquisition unit that acquires a thermal image of a tile wall surface, an enhanced image generation unit that generates an enhanced image of the thermal image acquired by the acquisition unit by setting upper and lower limit candidates for the temperature of the thermal image, an edge detection unit that performs edge detection processing on the enhanced image generated by the enhanced image generation unit and generates an edge image corresponding to the enhanced image, and a generation unit that performs matching processing on the edge image obtained by the edge detection unit using a rectangular template representing a tile, and generates a tile image including a plurality of rectangular tiles. and a temperature acquisition unit that repeats the generation of the enhanced image by the enhanced image generation unit, the generation of the edge image by the edge detection unit, and the generation of the tile image by the generation unit, and obtains the upper and lower limit candidates for the temperature when the number of tiles included in the tile image generated by the generation unit becomes equal to or greater than a predetermined threshold value as the upper and lower limits of the temperature when generating the enhanced image, and the enhanced image generation unit generates an enhanced image of the thermal image by setting the upper and lower limits of the temperature acquired by the temperature acquisition unit to the temperature of the thermal image acquired by the acquisition unit.

According to the present invention, by setting an appropriate temperature range when detecting loose tiles, it is possible to accurately detect loose tiles.

1 is a block diagram showing an example of a hardware configuration of a temperature range setting device according to a first embodiment. 1 is a block diagram showing an example of a functional configuration of a temperature range setting device according to a first embodiment; 1A is a diagram showing an example of a thermal image of a tile joint input to a temperature range setting device according to an embodiment, and FIG. 1B is a diagram showing an example of a tile image generated by the temperature range setting device according to an embodiment. 1 is a graph showing an example of a correspondence relationship between an upper limit temperature, a lower limit temperature, and the number of detected tiles of a temperature range according to the first embodiment. FIG. 11 is a diagram showing an example of a tile image when a temperature range according to the first embodiment is set. 5 is a flowchart showing an example of a process flow of a temperature range setting program according to the first embodiment. FIG. 11 is a block diagram showing an example of a functional configuration of a temperature range setting device according to a second embodiment. 13A to 13C are diagrams illustrating an N-dimensional parameter search method using a grid search algorithm according to a second embodiment. 13A to 13C are diagrams illustrating an example of a tile image when a temperature range and a block size according to the second embodiment are set. 10 is a flowchart showing an example of a process flow of a temperature range setting program according to a second embodiment.

Below, a temperature range setting device according to an embodiment of the present invention will be described with reference to the drawings. The temperature range setting device is an example of an image processing device of the present invention.

[First embodiment]
FIG. 1 is a block diagram showing an example of a hardware configuration of a temperature range setting device 10 according to the first embodiment.

As shown in FIG. 1, the temperature range setting device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a monitor 16, and a communication interface (I/F) 17. Each component is connected to each other via a bus 18 so as to be able to communicate with each other.

The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or storage 14, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculation processes according to the program stored in the ROM 12 or storage 14. In this embodiment, a temperature range setting program is stored in the ROM 12 or storage 14. The temperature range setting program is an example of an image processing program.

The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is composed of a HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including the operating system and various data.

The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs to the device.

The monitor 16 is, for example, a liquid crystal display that displays various information. The monitor 16 may be a touch panel type and function as the input unit 15.

The communication interface 17 is an interface for the device to communicate with other external devices, and uses standards such as Ethernet (registered trademark), FDDI (Fiber Distributed Data Interface), and Wi-Fi (registered trademark).

Next, the functional configuration of the temperature range setting device 10 will be described with reference to FIG. 2.

Figure 2 is a block diagram showing an example of the functional configuration of the temperature range setting device 10 according to the first embodiment.

As shown in FIG. 2, the temperature range setting device 10 has, as its functional components, an acquisition unit 101, an enhanced image generation unit 102, an edge detection unit 103, a generation unit 104, and a temperature acquisition unit 105. Each functional component is realized by the CPU 11 reading out a temperature range setting program stored in the ROM 12 or storage 14, expanding it in the RAM 13, and executing it.

The acquisition unit 101 acquires a thermal image of a tile wall surface. This thermal image is, for example, an image of the tile wall surface of a structure or the like captured by an infrared camera, and is an image that represents the temperature distribution of the tile wall surface. This thermal image is, for example, a thermal image of a tile joint that is uniformly partitioned by a group of straight lines in two directions in a real space that includes the tile wall surface.

The enhanced image generating unit 102 generates an enhanced image of the thermal image by setting upper and lower limit candidates for the temperature of the thermal image for the thermal image acquired by the acquiring unit 101.

The edge detection unit 103 performs edge detection processing on the enhanced image generated by the enhanced image generation unit 102, and generates an edge image corresponding to the enhanced image. This edge image is an image that includes edges that correspond to the tile joints.

The generation unit 104 performs a matching process using a rectangular template representing a tile on the edge image obtained by the edge detection unit 103, and generates a tile image including multiple rectangular tiles. Note that this tile image is generated using a known image processing technique such as template matching. The generation unit 104 detects the number of tiles included in the tile image.

The temperature acquisition unit 105 repeats the generation of an enhanced image by the enhanced image generation unit 102, the generation of an edge image by the edge detection unit 103, and the generation of a tile image by the generation unit 104, and acquires upper and lower limit candidates for the temperature when the number of tiles included in the tile image generated by the generation unit 104 becomes equal to or greater than a predetermined threshold value, as the upper and lower limits of the temperature when generating the enhanced image.

Then, the enhanced image generating unit 102 generates an enhanced image of the thermal image by setting the upper and lower limits of the temperature acquired by the temperature acquiring unit 105 to the temperature of the thermal image acquired by the acquiring unit 101. Then, the edge detecting unit 103 generates an edge image from the enhanced image generated by the enhanced image generating unit 102, and the generating unit 104 generates a tile image from the edge image generated by the edge detecting unit 103.

Next, a method for generating a tile image according to this embodiment will be described in detail with reference to Figures 3(A) and 3(B).

Figure 3 (A) shows an example of a thermal image of a tile joint input to the temperature range setting device 10 according to this embodiment, and Figure 3 (B) shows an example of a tile image generated by the temperature range setting device 10 according to this embodiment.

As an example, when a thermal image of a tile joint as shown in FIG. 3(A) is acquired, the temperature range setting device 10 performs smoothing (e.g., Gaussian smoothing) on the thermal image. The temperature range setting device 10 then calculates the gradient of brightness of the smoothed thermal image to detect edges. The temperature range setting device 10 also performs noise removal, such as reinforcing areas where edges are broken and removing excess edges that are not joints, to generate an edge image.

The temperature range setting device 10 performs pattern matching between the edges included in the edge image and a predetermined rectangular template, and applies the rectangular template to the edges to generate a tile image including multiple rectangular tiles, as shown in FIG. 3(B) as an example.

Next, with reference to Figures 4 and 5, we will specifically explain the method for searching for the upper and lower limits of the temperature range in the first embodiment.

Figure 4 is a distribution diagram showing an example of the correspondence between the upper and lower limits of the temperature range and the number of detected tiles according to the first embodiment.

The graph shown in FIG. 4 shows the relationship between the upper limit and lower limit temperatures and the number of detected tiles, with the lower limit of the temperature range on the vertical axis and the upper limit of the temperature range on the horizontal axis. In this graph, a specific color is associated with the number of detected tiles, and the number of detected tiles is represented by the difference in color (in the example of FIG. 4, the difference in color is represented by grayscale). Using this graph, the upper limit and lower limit values at which the number of detected tiles is, for example, maximum are determined as the optimal temperature range R _MAX . Note that the number of detected tiles is not limited to the maximum, and it is sufficient that the number of detected tiles is equal to or greater than a predetermined threshold. The predetermined threshold here is set to an empirically or analytically appropriate value (number of tiles). For example, the predetermined threshold may be set to a value (number of tiles) corresponding to a predetermined percentage (for example, a range of 60% to 90%) of the total number of tiles included in the thermal image.

More specifically, possible combinations (hereinafter referred to as "pairs") for the upper and lower temperature candidates of the thermal image are determined in advance. For example, N pairs are determined from the first pair to the nth pair. First, the first pair is set in the thermal image, and an emphasis image, an edge image, and a tile image corresponding to the first pair are generated. The number of tiles contained in this tile image is detected, and the number of detected tiles is registered in association with the first pair. Next, a second pair is set, and an emphasis image, an edge image, and a tile image corresponding to the second pair are generated. The number of tiles contained in this tile image is detected, and the number of detected tiles is registered in association with the second pair. The same process is repeated thereafter to detect the number of tiles corresponding to the nth pair, and the number of detected tiles is registered in association with the nth pair. As a result, in the graph of FIG. 4, the number of detected tiles is represented by different colors as described above.

Next, the graph in Fig. 4 is used to determine the pair of temperatures when the number of detection tiles is equal to or greater than the predetermined threshold. If there are multiple pairs when the number of detection tiles is equal to or greater than the predetermined threshold, any of the multiple pairs may be selected, but it is preferable to select the pair when the number of detection tiles is the maximum. In the example in Fig. 4, the pair when the number of detection tiles is the maximum is selected, and the temperature range R _MAX is determined based on this.

In the above, the number of tiles is detected for all pairs from the first pair to the nth pair, but this is not limited to this. For example, if the number of tiles detected for the kth pair (1<k<n) is equal to or greater than a predetermined threshold, the number of tiles for the (k+1)th pair and after may not be detected. In this case, the temperature of the kth pair is determined as the temperature range _RMAX .

FIG. 5 is a diagram showing an example of a tile image 30 when the temperature range R _MAX according to the first embodiment is set.

5 is a tile image obtained when the temperature range R _MAX is set. In this tile image 30, a plurality of tiles 31 are detected, and the number of detected tiles is the maximum (for example, 636).

Next, the operation of the temperature range setting device 10 according to the first embodiment will be described with reference to FIG.

Figure 6 is a flowchart showing an example of the processing flow of the temperature range setting program according to the first embodiment.

The steps shown in FIG. 6 are executed by the CPU 11 reading and executing the temperature range setting program from the ROM 12 or storage 14. The temperature range setting program starts to be executed, for example, when a thermal image of a tiled wall surface is input by the user.

First, in step S101, the CPU 11 acquires a thermal image of the tile wall surface input by the user. At this time, as described above, possible pairs are determined for the upper limit candidate and the lower limit candidate of the temperature of the thermal image. For example, N pairs from the first pair to the nth pair are determined.

In step S102, the CPU 11 sets a first pair for the thermal image acquired in step S101 and generates an enhanced image of the thermal image.

In step S103, the CPU 11 performs edge detection processing on the highlighted image generated in step S102 above, and generates an edge image corresponding to the highlighted image.

In step S104, the CPU 11 performs a matching process using a rectangular template representing a tile on the edge image generated in step S103, and generates a tile image including multiple rectangular tiles. At this time, the number of tiles included in the tile image is detected.

In step S105, the CPU 11 determines whether or not the number of tiles has been detected for all pairs determined in step S101. If it is determined that the number of tiles has been detected for all pairs (if the result is positive), the process proceeds to step S106, and if it is determined that the number of tiles has not been detected for all pairs (if the result is negative), the process proceeds to step S102 and the process is repeated.

In step S106, the CPU 11 searches for pairs in which the number of detected tiles is equal to or greater than the threshold value, for example, by using the graph shown in Fig. 4. In the example of Fig. 4, a pair in the temperature range R _MAX in which the number of detected tiles is the maximum is searched for.

In step S107, the CPU 11 obtains the upper and lower limit candidates for temperature obtained from the pairs searched for in step S106 as the upper and lower limits for temperature when generating the enhanced image.

In step S108, the CPU 11 sets upper and lower limits for the temperature acquired in step S107 for the thermal image acquired in step S101, and generates an enhanced image of the thermal image.

In step S109, the CPU 11 performs edge detection processing on the highlighted image generated in step S108 above, and generates an edge image corresponding to the highlighted image.

In step S110, the CPU 11 performs a matching process using a rectangular template representing the tile on the edge image generated in step S109, generates a tile image including multiple rectangular tiles, and ends the series of processes performed by this temperature range setting program.

As described above, the temperature range setting device 10 according to this embodiment sets an appropriate temperature range when detecting a loose tile. This allows loose tiles to be recognized with high accuracy.

Second Embodiment
In the above first embodiment, the upper and lower limits of the temperature range are searched for, but the parameters to be searched are not limited to these. For example, the block size of a rectangular template when detecting tiles also affects the accuracy of tile detection. For this reason, in the second embodiment, a parameter search is performed by including the block size of a rectangular template in the parameters to be searched, in addition to the upper and lower limits of the temperature range.

Figure 7 is a block diagram showing an example of the functional configuration of a temperature range setting device 10A according to the second embodiment.

As shown in FIG. 7, the temperature range setting device 10A has, as its functional components, an acquisition unit 101, an enhanced image generation unit 106, an edge detection unit 107, a generation unit 108, and a temperature acquisition unit 109. Each functional component is realized by the CPU 11 reading out a temperature range setting program stored in the ROM 12 or storage 14, and expanding and executing the program in the RAM 13. Note that components having the same functions as those in the temperature range setting device 10 shown in the first embodiment are given the same reference numerals, and repeated explanations will be omitted.

The enhanced image generating unit 106 sets a pair of thermal image temperatures (upper limit candidate and lower limit candidate) for the thermal image acquired by the acquiring unit 101, and generates an enhanced image of the thermal image. In addition to the pair of thermal image temperatures, the enhanced image generating unit 106 further sets the block size of a rectangular template representing a tile. Note that this block size represents the size of the rectangular template.

The edge detection unit 107 performs edge detection processing on the enhanced image generated by the enhanced image generation unit 106, and generates an edge image corresponding to the enhanced image.

The generation unit 108 performs a matching process on the edge image obtained by the edge detection unit 107 using a rectangular template whose block size is set by the enhancement image generation unit 106, and generates a tile image corresponding to the edge image.

The temperature acquisition unit 109 uses a grid search algorithm to derive the upper and lower limits of temperature and the block size of a rectangular template representing the tile when the number of tiles included in the tile image generated by the generation unit 108 is equal to or greater than a predetermined threshold. Note that the grid search algorithm is a method of trying all combinations of parameters through repeated calculations to search for the combination with the best evaluation accuracy. In this embodiment, the grid search algorithm is performed using the upper and lower limits of temperature and the block size as parameters.

Next, with reference to Figures 8 and 9, we will specifically explain the method of searching for N-dimensional parameters using the grid search algorithm according to the second embodiment.

Figure 8 is a diagram used to explain the N-dimensional parameter search method using the grid search algorithm according to the second embodiment.

As shown in Figure 8, the range of existence of N parameters is likened to an N-dimensional space. In the example of Figure 8, two parameters, α and β, are shown, but the same applies to three or more parameters.

Specifically, the score (e.g., the number of detected tiles) of lattice points (points indicated by circles) spaced at equal intervals within space B1 is calculated. If the point closest to the correct answer (e.g., the point with the highest score) is found in the kth trial of the repeated calculation, finer lattice points are added to the surrounding space B2, and the (k+1)th trial is performed in a similar manner in an even narrower range. Then, if the point closest to the correct answer is found in the (k+1)th trial, finer lattice points are added to the surrounding space B3, and the (k+2)th trial is performed in a similar manner in an even narrower range, and this operation is repeated. In this way, the upper and lower limits of the temperature range, where the number of detected tiles is, for example, maximized, and the block size are derived.

Figure 9 shows an example of a tile image 40 when the temperature range and block size according to the second embodiment are set.

The tile image 40 shown in FIG. 9 is a tile image obtained when the temperature range and block size are set. In this tile image 40, multiple tiles 41 are detected. The number of detected tiles is a maximum of, for example, 566 in the first layer, a maximum of, for example, 653 in the second layer, and a maximum of, for example, 672 in the third layer. Note that a layer represents a space, and in the example of FIG. 8 described above, space B1 corresponds to the first layer, space B2 corresponds to the second layer, and space B3 corresponds to the third layer. The more finely the lattice points are taken and the more layers are added, the more accurate the tile detection will be. However, since the calculation time increases as the number of layers increases, it is desirable to set an appropriate number of layers.

Next, the operation of the temperature range setting device 10A according to the second embodiment will be described with reference to FIG. 10.

Figure 10 is a flowchart showing an example of the processing flow of the temperature range setting program according to the second embodiment.

The CPU 11 reads out and executes a temperature range setting program from the ROM 12 or storage 14, thereby executing each step shown in FIG. 10. In this embodiment, the grid search algorithm described above is used.

First, in step S111, the CPU 11 calculates the scores of lattice points equally spaced in space B1, as shown in FIG. 8 above, by treating the range of existence of N parameters as an N-dimensional space. In this example, three parameters (upper limit candidate, lower limit candidate, and block size candidate for the temperature range) are associated with each lattice point. The score of a lattice point is, for example, the number of detected tiles. The number of detected tiles is obtained from the tile image.

In step S112, the CPU 11 detects, from among the multiple lattice points whose scores were calculated in step S111, the lattice point closest to the correct answer, for example, the lattice point with the maximum score.

In step S113, the CPU 11 determines whether the score calculation in step S111 has been repeated a predetermined number of times. If it is determined that the score calculation has not been repeated a predetermined number of times (if the result is negative), the process proceeds to step S114, and if it is determined that the score calculation has been repeated a predetermined number of times (if the result is positive), the process proceeds to step S115.

In step S114, the CPU 11 adds finer lattice points to the space B2 surrounding the lattice points detected in step S112, as shown in FIG. 8 above, for example, and returns to step S111 to repeat the process. Through this repeated calculation, the lattice point with the maximum score is detected for each layer representing each space.

On the other hand, in step S115, the CPU 11 determines the lattice point with the highest score among the multiple lattice points obtained for each layer representing each space by the above repeated calculation.

In step S116, the CPU 11 obtains the upper and lower limit candidates for the temperature range and the block size candidates corresponding to the lattice points determined in step S115 as the upper and lower limits of the temperature range and the block size to be applied to the thermal image, and ends the series of processes according to this temperature range setting program.

As described above, the temperature range setting device 10A according to this embodiment sets an appropriate temperature range and block size when detecting a floating tile. This allows floating tiles to be recognized with greater accuracy.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit of the invention.

In each of the above embodiments, a thermal image of a regular arrangement of tiles arranged vertically and horizontally was used. However, this is not limiting. Any thermal image having joints that are uniformly partitioned by straight lines in two directions in real space may be used. For example, a thermal image of a regular arrangement of tiles may be used, or any thermal image of tiles arranged periodically.

10, 10A Temperature range setting device 11 CPU
12 ROM
13 RAM
14 Storage 15 Input unit 16 Monitor 17 Communication I/F
18 bus 101 acquisition unit 102, 106 enhanced image generation unit 103, 107 edge detection unit 104, 108 generation unit 105, 109 temperature acquisition unit

Claims (4)

An acquisition unit for acquiring a thermal image of a tile wall surface;
an enhanced image generating unit that generates an enhanced image of the thermal image by setting upper limit candidates and lower limit candidates of the temperature of the thermal image acquired by the acquiring unit;
an edge detection unit that performs edge detection processing on the enhanced image generated by the enhanced image generation unit to generate an edge image corresponding to the enhanced image;
a generation unit that performs a matching process using a rectangular template representing a tile on the edge image obtained by the edge detection unit to generate a tile image including a plurality of rectangular tiles;
a temperature acquisition unit that repeats generation of the emphasized image by the emphasized image generation unit, generation of the edge image by the edge detection unit, and generation of the tile image by the generation unit, and obtains upper limit candidate and lower limit candidate temperatures when the number of tiles included in the tile image generated by the generation unit becomes equal to or greater than a predetermined threshold value as the upper limit and lower limit of the temperature when generating the emphasized image;
Equipped with
The emphasized image generating unit generates an emphasized image of the thermal image by setting an upper limit and a lower limit of the temperature acquired by the temperature acquiring unit to a temperature of the thermal image acquired by the acquiring unit. The enhancement image generating unit further sets a size of a rectangular template representing the tile;
The image processing device according to claim 1 , wherein the generation unit generates the tile image by using a rectangular template representing the tile, the size of which is set by the emphasis image generation unit. The image processing device according to claim 2 , wherein the temperature acquisition unit derives the upper and lower limits of the temperature and the size of a rectangular template representing the tile when the number of tiles included in the tile image is equal to or greater than a predetermined threshold, by using a grid search algorithm. Acquire a thermal image of the tile wall surface,
generating an enhanced image of the thermal image by setting upper limit candidates and lower limit candidates of the temperature of the thermal image;
performing an edge detection process on the enhanced image to generate an edge image corresponding to the enhanced image;
performing a matching process on the edge image using a rectangular template representing a tile to generate a tile image including a plurality of rectangular tiles;
generating the emphasized image, the edge image, and the tile image are repeated, and the upper limit candidate and the lower limit candidate of the temperature when the number of tiles included in the generated tile image becomes equal to or greater than a predetermined threshold are obtained as the upper limit and the lower limit of the temperature when generating the emphasized image;
When generating the emphasized image, an upper limit and a lower limit of the acquired temperature are set to a temperature of the thermal image, thereby generating an emphasized image of the thermal image;
An image processing program to be executed by a computer.