JP6869880B2 - Inspection image compositing device, inspection image compositing system, inspection image compositing method and program - Google Patents

Description

The present invention relates to an inspection image compositing apparatus, an inspection image compositing system, an inspection image compositing method and a program.

As a method of visually inspecting an inspection object in a place where workers cannot enter, a mobile imaging device is made to scan the surface of the inspection object to take an image, and the obtained image is used for visual inspection. There is a way to do it. In this method, a plurality of obtained images may be manually superposed to generate a panoramic image. In particular, in visual inspection, it is generally required to confirm the object with high resolution, so that the angle of view is narrowed, and scratches and the like may be photographed across a plurality of images. In such a case, a panoramic image may be manually generated in order to grasp the whole picture of scratches and the like.
Further, in Patent Document 1, the SN ratio is obtained by locally aligning a plurality of frames of an image containing noise that is almost independent of each frame obtained by photographing an inspection target and performing frame composition. Is stated to produce high frames.

Japanese Patent No. 5602530

It is a burden for the operator to manually generate the panoramic image, and it is preferable that the generation of the panoramic image can be automated. On the other hand, since the images obtained by visual inspection generally have a poor color difference, it is difficult or time-consuming to identify the overlapping portion of a plurality of images. It is preferable that the overlapping portion of a plurality of images taken for visual inspection can be identified in a shorter time.

Further, Patent Document 1 does not show a specific method for detecting overlapping portions of a plurality of images when the photographing device is moved to perform photographing. For example, Patent Document 1 describes that a frame is divided into local regions having the same shape for alignment, but does not describe which local region and which local region to attempt to align. ..

The present invention provides an inspection image compositing apparatus, an inspection image compositing system, an inspection image compositing method and a program capable of identifying an overlapping portion of a plurality of images taken for visual inspection in a shorter time.

According to the first aspect of the present invention, the inspection image synthesizer scans the surface of the inspection object and takes pictures at regular time intervals. an acquiring unit, from the imaging position information obtained when each of the adjacent images in sequence, a moving amount calculating unit for calculating the movement amount between these images, the range of pixels that overlap between the adjacent images in the time series, the Based on the movement amount and the scanning locus information set for causing the imaging device to scan the surface of the inspection object , a range having a width corresponding to the variation in the movement amount is calculated, and the calculated range is calculated. A search range setting unit set as a search range in template matching and a range of pixels that overlap between the images are specified by performing template matching in the search range, and a panoramic image in which the specified pixel ranges are overlapped is synthesized. It includes an image compositing unit.

The search range setting unit may set the search range having a width corresponding to the variation in the movement amount in each of the vertical direction and the horizontal direction of the pixels of the image.

The search range setting unit may calculate the width according to the variation in the movement amount from the time-series images taken by the photographing device in the past.

The search range setting unit may calculate the width according to the variation in the movement amount from the operation test data of the photographing device.

The search range setting unit sets the difference between the maximum or minimum value of the movement amount and the standard movement amount of the imaging device in the plurality of actual data of the movement amount as the maximum variation of the movement amount, and sets the movement amount. The width of the search range may be calculated according to the maximum value of the variation of.

The search range setting unit sets a value obtained by adding a margin to the magnitude of the variation in the movement amount obtained from the actual data of the movement amount as the maximum value of the variation in the movement amount, and sets the maximum variation in the movement amount. The width of the search range according to the value may be calculated.

The search range setting unit may calculate the width of the search range according to the maximum value of the variation in the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation in the movement amount.

According to the second aspect of the present invention, the inspection image synthesizing system scans the surface of the inspection object and captures images at regular time intervals, and the image captured by the imaging device and imaging position information. The image information acquisition unit to be acquired, the movement amount calculation unit that calculates the movement amount between these images from the shooting position information for each of the images adjacent to each other in time series, and the pixels overlapping between the images adjacent to each other in the time series. range, and the movement amount, based on the scanning trajectory information set in order to scan the surface of the inspection object on the imaging apparatus, is calculated as a range having a width corresponding to the variation of the amount of movement, A panorama in which the search range setting unit that sets the calculated range as the search range in template matching and the range of pixels that overlap between the images are specified by performing template matching in the search range, and the specified pixel ranges are overlapped. It includes an image compositing unit for compositing images.

According to the third aspect of the present invention, the inspection image synthesizing method is to acquire the captured image and the imaging position information from an imaging device that scans the surface of the inspection object and captures images at regular time intervals. when the photographing position information of each of the adjacent images in sequence, and computing the amount of movement between the images, the range of pixels that overlap between the time series at adjacent said image, and the moving amount, the photographing Based on the scanning locus information set for the device to scan the surface of the inspection object, it is calculated as a range having a width corresponding to the variation of the movement amount, and the calculated range is used as a search range in template matching. This includes setting, specifying a range of overlapping pixels between the images by performing template matching in the search range, and synthesizing a panoramic image in which the specified pixel ranges are overlapped.

According to the fourth aspect of the present invention, the program acquires the captured image and the capture position information from the imaging device that scans the surface of the inspection object and captures images at regular time intervals. when the photographing position information of each of the adjacent images in sequence, and computing the amount of movement between the images, the range of pixels that overlap between the time series at adjacent said image, and the moving amount, the photographing Based on the scanning locus information set for the device to scan the surface of the inspection object, it is calculated as a range having a width corresponding to the variation of the movement amount, and the calculated range is used as a search range in template matching. A program for setting and performing template matching in the search range to specify the range of overlapping pixels between the images and synthesizing a panoramic image in which the specified pixel ranges are overlapped. Is.

According to the inspection image compositing apparatus, the inspection image compositing system, the inspection image compositing method and the program described above, it is possible to identify the overlapping portion of a plurality of images taken for visual inspection in a shorter time.

It is a schematic block diagram which shows the functional structure of the inspection image synthesis system which concerns on one Embodiment of this invention. It is a figure which shows the example of the scanning pattern of the photographing apparatus which concerns on the same embodiment. It is a figure which shows the 2nd example of the scanning pattern of the photographing apparatus which concerns on this embodiment. It is a figure which shows the 3rd example of the scanning pattern of the photographing apparatus which concerns on the same embodiment. It is a figure which shows the example of the time series of the image photographed by the photographing apparatus which concerns on the same embodiment. It is a figure which shows the example of the moving amount of the photographing apparatus which concerns on the same embodiment. It is a figure which shows the example of superimposition of the image which concerns on the same embodiment. It is a figure which shows the example of superimposing images when the moving amount of the photographing apparatus which concerns on the same embodiment varies. It is a figure which shows the setting example of the template in the template matching which concerns on the same embodiment. It is a figure which shows the setting example of the search range in the template matching which concerns on the same embodiment. It is a flowchart which shows the example of the inspection procedure using the inspection image synthesis system which concerns on the same embodiment. It is a schematic block diagram which shows the structure of the computer which concerns on at least one Embodiment.

Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the inventions claimed. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.
FIG. 1 is a schematic block diagram showing a functional configuration of an inspection image synthesis system according to an embodiment of the present invention. As shown in the figure, the inspection image compositing system 1 includes a photographing device 100 and an inspection image compositing device 200. The inspection image synthesizer 200 includes a communication unit 210, a display unit 220, an operation input unit 230, a storage unit 280, and a control unit 290. The control unit 290 includes a movement amount calculation unit 291, a search range setting unit 292, and an image composition unit 293.

The inspection image synthesis system 1 provides a photographed image of the surface of the inspection object for visual inspection (VT) of the inspection object.
The imaging device 100 scans the surface of the inspection object and photographs at regular time intervals.
FIG. 2 is a diagram showing an example of a scanning pattern of the photographing apparatus 100. In FIG. 2, the scanning pattern of the photographing apparatus 100 on the surface of the inspection object 900 is shown by the line L11. Moreover, the scratch C11 on the surface of the inspection object 900 is shown.

In the scanning pattern of FIG. 2, the photographing apparatus 100 moves in the downward direction of the figure while reciprocating the surface of the inspection object 900 from side to side of the figure. The photographing device 100 takes an image on each of the outbound route and the inbound route.
However, the scanning pattern of the photographing apparatus 100 is not limited to a specific one. As the scanning pattern of the photographing device 100, various scanning patterns capable of photographing the entire area to be visually inspected can be used.

FIG. 3 is a diagram showing a second example of the scanning pattern of the photographing apparatus 100. In FIG. 3, the inspection object 900 has a shape in which a cylinder, a truncated cone, and a rectangular parallelepiped are combined. As shown by the arrow, the photographing apparatus 100 scans the inspection object 900 from the upper side of the inspection object 900 while rotating in the circumferential direction of the inspection object, and moves downward for each round to scan in the circumferential direction. repeat.
By scanning the imaging device 100 in this way, it is possible to scan even an inspection object having a circular shape such as a cylinder and a truncated cone.

FIG. 4 is a diagram showing a third example of the scanning pattern of the photographing apparatus 100. In FIG. 4, the scanning pattern of the photographing apparatus 100 on the surface of the inspection object 900 is shown by a solid line and a broken line. The image when the photographing apparatus 100 moves along the solid line is used for the visual inspection. On the other hand, the image when the photographing device 100 moves along the broken line is not used for the visual inspection.
Here, it is conceivable that the operation of the photographing device 100 may differ between the outward path and the returning path due to the backlash of the gears of the drive unit of the photographing device 100 and the like. In this case, by using only the image captured by the photographing device 100 in one direction, it is possible to reduce the influence of backlash and the like.

The inspection image compositing device 200 generates a panoramic image in which a plurality of images taken by the photographing device 100 on the surface of the inspection object are joined together. The panoramic image referred to here is an image corresponding to an image taken at an angle of view wider than the actual angle of view of the photographing apparatus 100 (thus, an image showing a wider range of the inspection object).
The inspection image synthesizer 200 is configured by using a computer such as a personal computer (PC) or an EWS (Engineering Workstation), for example.

The communication unit 210 communicates with another device. In particular, the communication unit 210 receives the shooting data from the shooting device 100. The shooting data includes an image shot by the shooting device 100 as image data, and also includes shooting position information of the image (position information of the shooting device 100 when the image is shot). The communication unit 210 corresponds to the example of the image information acquisition unit.
The shooting position information here may be such that the amount of movement of the shooting device 100 between shootings can be known. For example, the photographing position information may be the integrated movement distance information of the photographing apparatus 100. Alternatively, the photographing position information may indicate the position of the photographing apparatus 100 at the time of photographing by three-dimensional coordinate data or two-dimensional coordinate data.

The display unit 220 has a display screen such as a liquid crystal panel or an LED (Light Emitting Diode) panel, and displays various images. For example, the display unit 220 displays a panoramic image of an inspection object generated by the inspection image synthesizer 200.
The operation input unit 230 is configured by using an input device such as a keyboard and a mouse, and receives a user operation. For example, when the display unit 220 is displaying a panoramic image of an inspection object, the operation input unit 230 is a user operation for instructing a change in magnification of the image and a user operation for instructing a change (movement) of a display range. You may receive it.

The storage unit 280 stores various data. For example, the storage unit 280 stores the shooting data received by the communication unit 210 in chronological order.
The storage unit 280 is configured by using the storage device included in the inspection image synthesizer 200.
The control unit 290 controls each unit of the inspection image synthesizer 200 to perform various processes. The control unit 290 is configured by a CPU (Central Processing Unit) included in the inspection image compositing device 200 reading a program from the storage unit 280 and executing the program.

The movement amount calculation unit 291 calculates the movement amount between these images from the shooting position information for each image from the shooting device 100, which are adjacent to each other in time series. In particular, the movement amount calculation unit 291 calculates the movement amount by the number of pixels.
For example, the movement amount calculation unit 291 calculates the distance between these shooting positions (that is, the movement amount of the shooting device 100) by taking the difference of the shooting position information for each adjacent image in time series. Then, the movement amount calculation unit 291 converts the distance into the number of pixels by multiplying the calculated distance by the number of pixels per unit distance in the captured image of the photographing device 100.

FIG. 5 is a diagram showing an example of a time series of images taken by the photographing device 100. The horizontal axis of FIG. 5 indicates the time. In FIG. 5, the shooting time of each image is indicated by a frame number. The frame referred to here is an image obtained by one shooting of the photographing device 100 (hence, individual images taken by the photographing device 100).
Of the images shown in FIG. 5, a part of a flaw (scratch C11 in FIG. 2) is shown in each image of frame numbers t + ω, t + (ω + 1), and t + (ω + 2). In the scanning pattern shown in FIG. 2, the image pickup device 100 captures the image of FIG. 5 while constantly moving from left to right in the vertical direction of the drawing. As a result, when the images are viewed in the order of frame numbers t + ω, t + (ω + 1), t + (ω + 2), the image of the flaw is constant in height (vertical direction in the figure) and moves from the right side of the figure to the center and left side. doing.

FIG. 6 is a diagram showing an example of the amount of movement of the photographing device 100.
In FIG. 6, an image having a frame number t + ω and an image having a frame number t + (ω + 1) among the images shown in FIG. 5 are shown vertically arranged with their left and right edges aligned. In the following description, the image with the frame number t + ω is referred to as the first image, and the image with the frame number t + (ω + 1) is referred to as the second image.

The point P11 in the first image and the point P12 in the second image indicate the same point on the surface of the inspection object. The first image and the second image are not displaced in the vertical direction of the figure. The distance d corresponds to the amount of movement of the photographing device 100 from the acquisition of the first image to the acquisition of the second image. The movement amount calculation unit 291 calculates this distance d by the number of pixels p in the image captured by the photographing device 100.
After shifting the second image to the right by the number of pixels p from the state shown in FIG. 6, the second image is moved upward to align the upper and lower edges of the first image with the upper and lower edges of the second image. A panoramic image of these two images can be obtained.

The search range setting unit 292 calculates the range of pixels that are adjacent to each other in time series and overlap between the images from the photographing device 100, based on the movement amount calculated by the movement amount calculation unit 291. The search range setting unit 292 sets the calculated pixel range as the search range in the template matching performed by the image composition unit 293.
FIG. 7 is a diagram showing an example of superimposing images. In the example of FIG. 7, as described with reference to FIG. 6, the first image and the second image are shifted to the left and right by the number of pixels p, and the top and bottom are aligned and superposed.

Assuming that the photographing device 100 always moves from the left to the right in FIG. 2 by a distance D between shootings, and if there is no deviation in the vertical direction of FIG. 2 during that time, the obtained image as shown in FIG. A panoramic image (for one horizontal line) can be obtained by shifting the number of pixels p in the direction and superimposing the images. In this case, the range of pixels to be overlapped is uniquely determined, and it is not necessary to perform image matching.

However, in reality, it is difficult to constantly move the photographing device 100 at a constant speed, and the amount of movement of the photographing device 100 between shootings varies. Therefore, the search range setting unit 292 sets a search range having a width corresponding to the variation in the movement amount of the photographing device 100.
Further, usually, when the photographing apparatus 100 moves, not only the movement amount varies in the planned movement direction (horizontal direction in FIG. 2), but also the direction orthogonal to the planned movement direction (vertical direction in FIG. 2). (Variation occurs from 0 movement amount). Therefore, the search range setting unit 292 sets a search range having a width corresponding to the variation in the movement amount in each of the vertical direction and the horizontal direction of the pixels of the image.

FIG. 8 is a diagram showing an example of superimposing images when the amount of movement of the photographing apparatus 100 varies. In FIG. 8, the position of the image (first image in the example of FIG. 7) located on the left side of the figure by superposition is shown at position q1. In relation to this position q1, the positions of the images (second image in the example of FIG. 7) located on the right side of the figure by superposition are shown in three ways: positions q2, q2', and q2''.
In the following, the image located on the left side of the figure by superposition is referred to as a left image, and the image located on the right side of the figure by superposition is referred to as a right image.

The pixel number p _h, in which the search range setting section 292 indicates a standard amount of movement of the imaging device 100 assumed in the number of pixels. In the example of FIG. 8, the photographing apparatus 100 between the imaging and imaging is a standard to move to the right in Figure by the number of pixels p _h.
Number of pixels Delta] p _h, the search range setting unit 292 is assumed, the maximum value of the variation of the amount of movement planned moving direction (lateral direction in the drawing). In the example of FIG. 8, the photographing apparatus 100 is assumed to move to the right in FIG. Within the range of the pixel number p _h -Δp _h until p _{h +} Δp _h between the imaging and imaging.
The number of pixels Δp _v is the maximum value of the variation in the amount of movement in the direction perpendicular to the planned movement direction (vertical direction in the figure) assumed by the search range setting unit 292. In the example of FIG. 8, the photographing apparatus 100 is assumed to be displaced within the upper and lower either the pixel number Delta] p _V also in FIG between shooting and shooting.

Position q2 is, the photographing apparatus 100 is moved to the right in FIG. Is a scheduled movement direction by the number of pixels p _h, when no shift in the vertical direction and the position of the right image. Position q2 'is photographing apparatus 100 is moved by the number of pixels p _h -Δp _h in the right direction in the figure, when displaced by the number of pixels Delta] p _v upward in the figure, the position of the right image. Position q2 '' includes a photographing apparatus 100 is moved by the number of pixels p _h -Δp _h in the right direction in the figure, when displaced by the number of pixels Delta] p _v upward in the figure, the position of the right image.
The positions q2'and q2'' indicate the positions when there is the maximum expected fluttering on both the top, bottom, left and right of the figure. The region A11, which is the overlapping portion of the positions q1, q2', and q2'', indicates the portion where the two images always overlap within the assumed range.

FIG. 9 is a diagram showing an example of setting a template in template matching. In the example of FIG. 9, the search range setting unit 292 sets the area A11, which is a portion where the left image always overlaps with the right image within the assumed range of the variation in the movement amount of the photographing device 100, as the template in the template matching.
When the number of pixels of the left image in the horizontal direction in FIG. 9, _{p x, p n + Δp n} ( standard amount of movement within the expected range is the maximum value of the moving amount of the lateral direction of the imaging apparatus 100 of the search range setting section 292 + _p x -p _n + Δp _n of the maximum value) was subtracted from _{p x} of variation, the number of pixels in the horizontal direction of the template (area A11).
Further, the subtraction and the number of pixels left image in the vertical direction in FIG. 9, p _y, a double Delta] p _n is the maximum value of the longitudinal movement amount of the imaging apparatus 100 within the expected range of the search range setting section 292 _p y _{-2Δp n} that becomes the longitudinal direction number of pixels of the template (area A11). If the photographing apparatus 100 is shifted upward, it is subtracted twice the Delta] p _n assuming both be shifted downward.

The template matching referred to here is a pattern matching method for searching the appearance position of the pattern of the pixel value of the template within the search range. Specifically, pattern matching is performed by the following procedure.

(1) A part of one of the two images to be matched (the left image in the examples of FIGS. 8 to 8) (area A11 in the example of FIGS. 8 to 8) is set in the template.
(2) A part of the other (right image in the example of FIGS. 8 to 8) of the two images to be matched is set in the search range.
(3) Within the search range, search for a partial image whose pixel value best matches the template. As an evaluation score of the degree of coincidence of pixel values, a score capable of evaluating the magnitude of the difference in pixel values, such as the root mean square of the difference in pixel values in each pixel, is calculated, and the partial image (pixel value) having the smallest score is calculated. Search for a partial image) that is evaluated to have the smallest difference.

FIG. 10 is a diagram showing an example of setting a search range in template matching. The search range setting unit 292 sets a range of the right image that may overlap with the template (area A11 in FIG. 9) (an image or a partial image of the same part of the surface of the inspection object) as a search range. Set. In the example of FIG. 10, the search range setting unit 292 sets the area A12 as the search range.

P _n -Δp _n within expected range is the minimum value of the moving amount of the lateral direction of the imaging apparatus 100 of the search range setting section 292 - a _(standard movement amount maximum value of the variation), the right image in the horizontal direction in FIG. 10 of _p is subtracted from the number of pixels _{p x x - (p n -Δp} n) becomes the number of pixels in the horizontal direction of the search range (area A12).
In the vertical direction of FIG. 10, the entire right image (number of pixels _py ) is set in the search range.
As shown in the area A11 in the area A12 in FIG. 10, in the template matching, a partial image having the same shape, orientation and size as the template is set in the search range, and the evaluation score of the degree of matching of the pixel values is calculated. To do. Of all the sub-images that can be set, the position of the sub-image with the minimum evaluation score is detected.
The method of executing template matching here is not limited to a specific method.

To set the search range illustrated in the template and 10 are illustrated in Figure 9, the standard of the amount of movement of the imaging device 100 (the number of pixels p _h in the example of FIG. 8 to _8), the movement of the imaging device 100 maximum value of the variation amount (in the example of FIG. 8 to 8 Delta] p _h and Delta] p v number of _pixels) must be set.
Of these, the standard movement amount of the photographing device 100 is calculated by multiplying the moving speed set in the photographing device 100 and the shooting cycle (time interval between shootings) and converting it into the number of pixels. Can be done.

On the other hand, it is conceivable that the maximum value of the variation in the movement amount of the photographing device 100 is obtained based on some actual data of the operation of the photographing device 100.
For example, the search range setting unit 292 or a person calculates the movement amount of the shooting device 100 for each shooting from the time-series images taken by the shooting device 100 in the past, and the maximum value of the variation of the movement amount from the calculated movement amount. May be set. As a time-series image taken by the photographing device 100 in the past, it is conceivable to use an image taken at the time of past inspection or an image taken at the time of testing or adjustment at a factory or the like.
Alternatively, if the movement amount between shootings is recorded in the operation test of the photographing device 100, the maximum value of the variation in the moving amount of the photographing device 100 can be set from the operation test data.

When some actual data of the movement amount of the photographing device 100 is obtained, the search range setting unit 292 or a person detects the maximum value or the minimum value of the moving amount, and the obtained maximum value or the minimum value and the photographing device 100 are obtained. The difference from the standard movement amount of the above may be used as the maximum value of the variation in the movement amount of the photographing apparatus 100. Alternatively, the search range setting unit 292 or a person may use a value obtained by adding a margin (margin) to the size of the variation obtained from the actual data of the movement amount as the maximum value of the variation in the movement amount of the photographing device 100. It may be.
Alternatively, the search range setting unit 292 or a person obtains the frequency distribution of the movement amount of the photographing device 100, calculates 3σ, and uses it as the maximum value of the variation of the moving amount. The value may be obtained by a probabilistic method.

Further, when the search range setting unit 292 or a person obtains the maximum value of the variation in the movement amount of the photographing device 100, the moving based on the scanning locus information set for causing the photographing device 100 to scan the surface of the inspection object. The maximum value of the amount variation may be set.
For example, in the scanning pattern of FIG. 2, when the photographing apparatus 100 moves from left to right in the figure and when it moves from right to left, the moving direction of the image captured in the image is also opposite. For example, in each image of the frame numbers t + ω, t + (ω + 1), and t + (ω + 2) in FIG. 5, the image of the scratches in the image moves from the right to the left of the image. When the photographing device 100 moves from the right to the left in the figure, the movement of the image of the flaw is also reversed. Further, the image when the photographing device 100 moves in the vertical direction of FIG. 2 is not used for the visual inspection. When the photographing device 100 calculates the moving amount of the photographing device 100 based on the time series of the images captured in the past, the moving amount of the photographing device 100 is grasped by comparing the image with the scanning locus information and grasping the moving direction in the image. May be asked.

Further, in the case of the scanning pattern of FIG. 3, the length of the truncated cone portion of the inspection object 900 is longer than that of the cylindrical portion by one round, and the lower side of the truncated cone portion is one round. Will be longer. As described above, when the movement amount of the photographing device 100 differs depending on the portion of the inspection object 900, it is conceivable to associate the image with the position of the photographing device 100 by referring to the scanning locus information.
In the case of the scanning pattern of FIG. 4, the image when the photographing apparatus 100 moves along the broken line is not used for the visual inspection. In this case as well, it is conceivable to associate the image with the position of the photographing apparatus 100 by referring to the scanning locus information.

The image composition unit 293 performs template matching in the search range set by the search range setting unit 292, and specifies the range of pixels that overlap between adjacent images in chronological order. Then, the image synthesizing unit 293 synthesizes a panoramic image in which the specified pixel ranges are superimposed.
Here, as a method of image matching, in addition to template matching, there is also image matching by feature point extraction. However, the image obtained by photographing for visual inspection generally has a poor color tone difference, and it is difficult to extract appropriate feature points. From this, in image matching for images taken for visual inspection, it is expected that pattern matching accuracy is better when template matching is performed than when image matching is performed by feature point extraction.

In template matching, the larger the difference between the size of the search range and the size of the template, the larger the number of partial images that can be taken in the search range, and the longer the processing takes.
On the other hand, in the inspection image synthesizer 200, the search range setting unit 292 sets the template and sets the portion that may overlap with the template in the search range, so that the number of partial images that can be taken in the search range is small. The processing time is relatively short.

Next, the procedure of inspection using the inspection image synthesis system 1 will be described.
FIG. 11 is a flowchart showing an example of an inspection procedure using the inspection image synthesis system 1.
In the inspection procedure of FIG. 11, the photographing apparatus 100 photographs the surface of the inspection object (step S101).
Next, the inspection image synthesizer 200 panoramicly synthesizes an image of the surface of the inspection target by the photographing device 100 to generate an entire image of the visual inspection target range of the surface of the inspection target (step S102).

The inspection worker performs a visual inspection using the entire image generated by the inspection image synthesizer 200 (step S103).
Then, the inspection worker determines whether or not there is an instruction by visual inspection (step S104). The instruction here is an abnormal part such as a shape change or a scratch.
If it is determined that there is no instruction (step S104: NO), the inspection worker reports the inspection result (step S121).
After step S121, the process of FIG. 11 ends.

On the other hand, when it is determined that the instruction has been given (step S104: the instruction has been given), the inspection worker enlarges the part corresponding to the instruction in the entire image generated by the inspection image synthesizer 200 and performs a more detailed visual inspection. (Step S111).
Then, the inspection worker reports the inspection result (step S112), and determines and implements the response measures to the instructions such as inspection by another method, soundness evaluation or repair work (step S113).
After step S113, the process of FIG. 11 ends.

According to the inspection procedure of FIG. 11, the inspection image synthesizer 200 performs panoramic composition of the images to generate the entire image of the visual inspection target range, so that the entire image can be obtained without causing a heavy burden on the inspection worker. It holds on the premise.
In the inspection procedure of FIG. 11, the inspection worker can easily grasp the shape of the inspection target by obtaining the entire image of the visual inspection target range. Further, when the inspection object is instructed, the position of the instruction in the inspection object can be easily grasped. Further, even if the instruction is relatively large and cannot fit in the angle of view of the photographing apparatus 100, the inspection worker can easily grasp the entire shape of the instruction by referring to the entire image of the visual inspection target range.

Further, since the inspection image compositing device 200 generates the entire image of the visual inspection target range, the angle of view of each image taken by the photographing device 100 may be relatively small. In this respect, the photographing apparatus 100 can perform photographing at a high resolution. The photographing device 100 takes a picture at a high resolution, and the inspection image compositing device 200 panoramicly synthesizes the image to generate an entire image of the visual inspection target range, whereby a high resolution overall image can be obtained. When the instruction is found by the visual inspection, the inspection worker can enlarge the corresponding part of the whole image and perform a more detailed visual inspection.

In a general inspection that does not use the inspection image composition system 1, a visual inspection is performed using an image of the inspection object, and if an instruction is found, the magnification of the imaging is increased, the relevant part is photographed again, and the visual inspection is performed again. It is carried out. On the other hand, in the inspection using the inspection image synthesis system 1, it is not necessary to re-photograph the inspection object as described above. In this respect, the inspection can be performed efficiently by using the inspection image synthesis system 1.
Further, in a general inspection that does not use the inspection image composition system 1, the inspection worker refers to the image taken by the photographing apparatus in real time to confirm the presence or absence of an instruction. On the other hand, in the inspection using the inspection image synthesis system 1, the inspection worker only needs to confirm the presence or absence of the instruction by referring to the entire image of the visual inspection target range, and there is no time constraint due to the real-time property.
Further, in a general inspection that does not use the inspection image composition system 1, the image referred to by the inspection worker changes in time series. On the other hand, in the inspection using the inspection image synthesis system 1, the inspector refers to the entire image of the visual inspection target range, so that the image does not change in time series. In this respect, the inspection using the inspection image synthesis system 1 can improve the inspection accuracy. In particular, in the inspection using the inspection image synthesis system 1, a visual inspection can be performed using a still image instead of a moving image. Further, since the image does not change in time series, the observation time (time for visual inspection) is not limited, and as a result, the possibility of overlooking an abnormality or the like can be reduced.

As described above, the communication unit 210 acquires the photographed image and the photographing position information from the photographing apparatus 100 which scans the surface of the inspection object and photographs at regular time intervals. The movement amount calculation unit 291 calculates the movement amount between these images from the shooting position information for each adjacent image in chronological order. The search range setting unit 292 calculates the range of pixels that overlap between adjacent images in time series based on the movement amount of the photographing device 100, and sets it as the search range in template matching. The image synthesizing unit 293 performs template matching in the search range to specify the range of overlapping pixels between the images, and synthesizes a panoramic image in which the specified pixel ranges are overlapped.

The search range setting unit 292 can narrow down the search range by setting the search range based on the movement amount of the photographing device 100. In this respect, according to the inspection image synthesizer 200, it is possible to identify the overlapping portion of a plurality of images taken for visual inspection in a shorter time.
In addition, the image synthesizing unit 293 synthesizes panoramic images to generate an entire image of the visual inspection target range, so that the inspection worker can grasp the overall positional relationship and perform visual inspection efficiently and with high accuracy. It can be carried out.

Further, the search range setting unit 292 sets a search range having a width corresponding to the variation in the movement amount of the photographing device 100.
As a result, the search range setting unit 292 can narrow down the search range to such an extent that template matching can be performed even if the movement amount of the photographing device 100 varies. In this respect, according to the inspection image compositing apparatus 200, it is possible to secure the accuracy of template matching and reduce the processing time at the same time.

Further, the search range setting unit 292 sets a search range having a width corresponding to the variation in the movement amount in each of the vertical direction and the horizontal direction of the pixels of the image.
As a result, in the inspection image compositing device 200, in addition to the variation in the amount of movement of the photographing device 100 in the planned moving direction of the photographing device 100, it corresponds to the deviation in the direction perpendicular to the planned moving direction of the photographing device 100. Then, the search range can be narrowed down.

Further, the search range setting unit 292 calculates the width of the search range according to the variation in the movement amount of the photographing device 100 from the time-series images taken by the photographing device 100 in the past.
As a result, the search range setting unit 292 can set the search range based on the actual operation of the photographing device 100. At this point, the search range setting unit 292 can set the search range with high accuracy.

Further, the search range setting unit 292 calculates the width of the search range according to the variation in the movement amount of the photographing device 100 from the operation test data of the photographing device 100.
As a result, the search range setting unit 292 can set the search range based on the actual operation of the photographing device 100. In this respect, the search range setting unit 292 can set the search range with high accuracy.

Further, the search range setting unit 292 sets the difference between the maximum value or the minimum value of the movement amount in the plurality of actual data of the movement amount and the standard movement amount of the photographing apparatus 100 as the maximum variation of the movement amount, and sets the movement amount of the movement amount. Calculate the width of the search range according to the maximum value of the variation.
As a result, the search range setting unit 292 can set the width of the search range with high accuracy according to the maximum value or the minimum value of the variation in the movement amount.

Further, the search range setting unit 292 sets the value obtained by adding the margin to the size of the variation in the movement amount obtained from the actual data of the movement amount as the maximum value of the variation in the movement amount, and sets it as the maximum value of the variation in the movement amount. Calculate the width of the corresponding search range.
Since the search range setting unit 292 calculates the maximum value of the variation in the movement amount including the margin, the possibility of underestimating the magnitude of the variation in the movement amount can be reduced.

Further, the search range setting unit 292 calculates the width of the search range according to the maximum value of the variation in the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation in the movement amount.
The width of the search range can be set with high accuracy in that the search range setting unit 292 calculates the magnitude of the variation in the movement amount based on the probabilistic method.

Further, the search range setting unit 292 calculates the width of the search range according to the variation in the movement amount of the photographing device 100 based on the scanning locus information set for causing the photographing device 100 to scan the surface of the inspection object. To do.
As a result, the search range setting unit 292 can obtain variations in the amount of movement of the photographing device 100 in response to various operation patterns in scanning performed by the photographing device 100, and can set the search range based on this. it can. The search range setting unit 292 can set the search range with high accuracy in that a large amount of data for obtaining the variation in the movement amount of the photographing device 100 can be obtained.

FIG. 12 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.
The computer 10 includes a CPU 11, a main storage device 12, an auxiliary storage device 13, and an interface 14.
The inspection image synthesizer 200 described above is mounted on the computer 10. The operation of each unit of the control unit 290 described above is stored in the auxiliary storage device 13 in the form of a program. The CPU 11 reads the program from the auxiliary storage device 13 and expands it to the main storage device 12, and executes the above processing according to the program. Further, the CPU 11 secures a storage area corresponding to the above-mentioned storage unit 280 in the main storage device according to the program.

A program for realizing all or a part of the functions of the control unit 290 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by the computer system and executed to execute each unit. May be processed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices.
Further, the "computer system" includes a homepage providing environment (or a display environment) if a WWW system is used.
The "computer-readable recording medium" is a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, a hard disk built in a computer system, or a storage device such as an SSD (Solid State Drive). Say that. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included.

1 Inspection image composition system 100 Imaging device 200 Inspection image composition device 210 Communication unit 220 Display unit 230 Operation input unit 280 Storage unit 290 Control unit 291 Movement amount calculation unit 292 Search range setting unit 293 Image composition unit

Claims (10)

An image information acquisition unit that acquires images taken and shooting position information from a shooting device that scans the surface of the object to be inspected and shoots at regular time intervals.
A movement amount calculation unit that calculates the movement amount between these images from the shooting position information of each of the adjacent images in chronological order,
The range of pixels that overlap between the time series at adjacent said image, and the moving amount, based on the scanning trajectory information set in order to scan the surface of the inspection object on the imaging device, the moving amount A search range setting unit that calculates as a range having a width corresponding to the variation of the above and sets the calculated range as a search range in template matching.
An image synthesizing unit that performs template matching in the search range to specify the range of overlapping pixels between the images and synthesizes a panoramic image in which the specified pixel ranges are overlapped.
An inspection image synthesizer equipped with. The search range setting unit sets the search range having a width corresponding to the variation in the movement amount in each of the vertical direction and the horizontal direction of the pixels of the image.
The inspection image synthesizer according to claim 1. The search range setting unit calculates a width according to the variation in the amount of movement from the time-series images taken by the photographing device in the past.
The inspection image synthesizer according to claim 1 or 2. The search range setting unit calculates a width according to the variation in the movement amount from the operation test data of the photographing device.
The inspection image synthesizer according to any one of claims 1 to 3. The search range setting unit sets the difference between the maximum value or the minimum value of the movement amount and the standard movement amount of the imaging device in the plurality of actual data of the movement amount as the maximum variation of the movement amount, and sets the movement amount. Calculate the width according to the maximum value of the variation of
The inspection image synthesizer according to claim 3 or 4. The search range setting unit sets a value obtained by adding a margin to the magnitude of the variation in the movement amount obtained from the actual data of the movement amount as the maximum value of the variation in the movement amount, and sets the maximum of the variation in the movement amount. Calculate the width of the search range according to the value,
The inspection image synthesizer according to claim 3 or 4. The search range setting unit calculates the width of the search range according to the maximum value of the variation of the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation of the movement amount.
The inspection image synthesizer according to claim 3 or 4. An imaging device that scans the surface of the object to be inspected and photographs at regular time intervals.
An image information acquisition unit that acquires an image captured by the photographing apparatus and shooting position information, and an image information acquisition unit.
A movement amount calculation unit that calculates the movement amount between these images from the shooting position information of each of the adjacent images in chronological order,
The range of pixels that overlap between the time series at adjacent said image, and the moving amount, based on the scanning trajectory information set in order to scan the surface of the inspection object on the imaging device, the moving amount A search range setting unit that calculates as a range having a width corresponding to the variation of the above and sets the calculated range as a search range in template matching.
An image synthesizing unit that performs template matching in the search range to specify the range of overlapping pixels between the images and synthesizes a panoramic image in which the specified pixel ranges are overlapped.
Inspection image composition system equipped with. Acquiring captured images and imaging position information from an imaging device that scans the surface of the object to be inspected and photographs at regular time intervals.
From the shooting position information for each of the adjacent images in chronological order, the amount of movement between these images can be calculated.
The range of pixels that overlap between the time series at adjacent said image, and the moving amount, based on the scanning trajectory information set in order to scan the surface of the inspection object on the imaging device, the moving amount It is calculated as a range having a width corresponding to the variation of, and the calculated range is set as a search range in template matching.
Template matching is performed in the search range to specify the range of overlapping pixels between the images, and a panoramic image in which the specified pixel ranges are overlapped is synthesized.
Inspection image composition method including. On the computer
Acquiring captured images and imaging position information from an imaging device that scans the surface of the object to be inspected and photographs at regular time intervals.
From the shooting position information for each of the adjacent images in chronological order, the amount of movement between these images can be calculated.
The range of pixels that overlap between the time series at adjacent said image, and the moving amount, based on the scanning trajectory information set in order to scan the surface of the inspection object on the imaging device, the moving amount It is calculated as a range having a width corresponding to the variation of, and the calculated range is set as a search range in template matching.
Template matching is performed in the search range to specify the range of overlapping pixels between the images, and a panoramic image in which the specified pixel ranges are overlapped is synthesized.
A program to execute.

Images