JP5654402B2 - Equipment abnormality change determination device, equipment abnormality change judgment method, and program - Google Patents

Description

  The present invention relates to a technique for determining a temporal change of an abnormal part of equipment abnormality such as oil leakage or rust in power-related equipment such as a transformer.

  Insulating oil is used for electrical insulation in power-related facilities such as oil-filled transformers and oil breakers. Equipment malfunctions such as oil leakage and rust may occur due to aging and breakdown of power-related equipment. If oil leakage progresses, it will lead to machine failure, and if a failure occurs, it will spread to power transmission failure, causing problems in social life based on electricity. Also, in older types of power-related equipment, leakage of insulation oil increases the damage to the environment. Therefore, equipment inspection is being carried out in order to detect equipment abnormalities such as oil leakage and rust.

  Patent Document 1 discloses a method for determining whether or not oil leakage has occurred by comparing temporally continuous images that are captured with the installation position of the visible light camera and the capturing location fixed. . Moreover, in patent document 2 and non-patent document 1, under normal illumination such as an indoor fluorescent lamp, the fluorescence intensity emitted from the oil leakage part is measured when pulse light is irradiated with a light source such as a xenon flash lamp, A method for measuring the film thickness of oil leakage is disclosed. During this measurement, pulsed light is repeatedly irradiated in a short time, and the measured fluorescence intensity is integrated. As a result, the influence of a signal that is a random disturbance in time is reduced, and the S / N ratio of the measurement result is improved.

Japanese Patent Laid-Open No. 11-304634 (see, for example, paragraph 0013 and FIG. 1) JP-A-10-311771 (see, for example, paragraphs 0037 and 0074)

Masahiko Kuroki, 3 others, “Imaging technology of oil leakage by pulse fluorescence method”, Electrical Engineering E, The Institute of Electrical Engineers of Japan, 1998, 118, 10, p. 449-454

  However, facility inspections of power-related facilities such as transformers placed outdoors are performed after a certain period of time. In particular, when photographing a location where equipment is abnormal outdoors, the intensity of sunshine varies depending on the season and further on the photographing time. Therefore, even when the techniques described in Patent Document 2 and Non-Patent Document 1 are used, when an equipment abnormality that occurs outdoors is photographed, the influence of background light is stronger than in an indoor environment. In particular, when photographing is performed using a camera that senses the reflection wavelength of a special fluorescence caused by oil leakage or rust outdoors, the S / N ratio is low. That is, with the techniques described in Patent Document 2 and Non-Patent Document 1, it is difficult to reduce the influence of background light and the like over a long period of time, and there is also a problem that it is difficult to detect equipment abnormality itself. Also, outdoor equipment inspection is usually performed by human visual inspection. Therefore, even if an equipment abnormality can be detected during equipment inspection, it is not possible to immediately determine whether or not the abnormal state has progressed on the spot. That is, it is necessary to compare the current situation with the past situation. However, the technology for determining the change with time of an abnormal portion outdoors using an image is not shown in the above-described prior art documents.

  Further, unlike an indoor case as described in Patent Literature 2 and Non-Patent Literature 1, photographing of an abnormal portion in an outdoor power-related facility is not always performed from a predetermined photographing point. Therefore, the size and shape of the abnormal part included in the photographed image varies depending on the photographing angle and the photographing position. Even in such a case, it is necessary to be able to compare the temporal change of the abnormal portion between the past image and the current image.

  Then, the subject of this invention aims at providing the technique which determines the time-dependent change of equipment abnormalities, such as oil leak and rust, in an electric power related equipment.

The present invention performs image shape conversion on an image (collation image) including an abnormal part of the current equipment abnormality, and uses the same equipment shown in an image (reference image) serving as a reference for predetermined alignment. Align with respect to. This is because, generally, the collation image and the reference image have different shooting angles and shooting positions. This alignment makes it possible to compare the current abnormal part image with the past abnormal part image, and to determine the change with time. At this time, in the calculation of the image shape conversion parameter used for the image shape conversion, four first feature points are selected from an area including pixels with clear contrast in the collation image (first image), and the four second feature points are selected. Are selected from a region including pixels with clear contrast in the reference image (second image), and the selected first feature point is selected using the calculated image shape conversion parameter as the second feature point. By calculating the conversion points converted into the coordinates on the image and obtaining the second feature points in the vicinity of the conversion points, four sets of close ones are created, and the second feature is set in each set. The distance between the point and the conversion point is calculated, the sum of the distances for four sets is calculated, and the first feature point and the second feature point are again calculated until the sum of the distances is equal to or less than a predetermined threshold. Select the image shape conversion path again. Determining said image shape transformation parameter by repeatedly calculating the meter.
In addition, it is necessary to increase the S / N ratio of the abnormal portion image so that the abnormal portion image is not crushed by the image shape conversion. For this purpose, the irradiation light is pulsed, and the reflected light from the abnormal part is photographed and imaged at both timings immediately after emitting the pulsed light and when a predetermined time has elapsed. And the difference image which computed the difference of a pixel value in the image image | photographed at both timings is produced. Then, the S / N ratio is improved by accumulating the pixel values of the difference image for each pixel with respect to the image repeatedly photographed at both timings.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which determines the time-dependent change of equipment abnormalities, such as oil leak and rust, can be provided in an electric power related equipment.

It is a figure which shows the structural example of an equipment abnormality change analysis system. It is a figure which shows the outline | summary of the method of superimposing the change of an abnormal part. It is a figure which shows the classification of an abnormal condition, (a) represents new production | generation, (b) represents disappearance, (c) represents expansion, (d) represents a stop, (e) represents reduction. , (F) represents a change in position. It is a figure which shows the function example of an equipment abnormality time-dependent change determination apparatus. It is a figure which shows an example of the information (metadata) regarding the image of an abnormal part. It is a figure which shows an example of the method of superimposing the present and past specific wavelength collation image, (a) represents a specific wavelength collation image, (b) is a synthetic | combination image (visible light collation image + specific wavelength after scaling conversion) (C) represents the reference image, (d) represents the conversion result of the specific wavelength collation image, (e) represents the past specific wavelength collation image, and (f) represents the superimposition of the abnormal part. Represents an image. It is a figure which shows an example of the processing flow which determines the time-dependent change of an equipment abnormality. It is a figure which shows an example of the processing flow which determines the time-dependent change of an equipment abnormality. It is a figure which shows an example of the processing flow which determines the time-dependent change of an equipment abnormality. It is a figure which shows an example of the mask used for extraction of a positioning feature point, (a) represents the mask which emphasizes the pixel value change of the left-right direction, (b) represents the mask which emphasizes the pixel value change of the up-down direction, (C) represents a mask for emphasizing a pixel value change in the upper right and lower left direction, and (d) represents a mask for emphasizing a pixel value change in the upper left and lower right direction. It is a figure which shows the correspondence of the positioning feature point of each collation image and reference | standard image, (a) represents the feature point on a visible light collation image, (b) represents the feature point and coordinate transformation point on a reference | standard image. . It is a figure which shows an example of the acquisition flow of a visible light collation image and a specific wavelength collation image. It is a figure which shows an example of the method for improving the S / N ratio of a specific wavelength collation image, (a) represents the imaging | photography timing of an image, (b) represents the process of the integration | accumulation of a difference image. It is a figure which shows the outline | summary of the alignment of the collation image using a marker, and a reference | standard image, (a) represents a reference | standard image, (b) represents a collation image, (c) is the superimposition of a reference | standard image and a collation image. Represents the result. It is a figure which shows an example of the processing flow which determines the time-dependent change of the equipment abnormality at the time of using a marker. It is a figure which shows an example of the man-machine interface of the information regarding an abnormal part.

  Next, a mode for carrying out the present invention (hereinafter referred to as “the present embodiment”) will be described in detail with reference to the drawings as appropriate.

(Outline of equipment abnormality change analysis system)
FIG. 1 shows a configuration example of an equipment abnormality change analysis system 1 that analyzes changes in equipment abnormality over time.
The equipment abnormality change analysis system 1 includes an equipment abnormality aging change determination device 10, imaging devices (cameras) 20 a and 20 b, a lighting device 30, and an on-site processing device 40. The power-related equipment 50 is an inspection target and is not included in the equipment abnormality change analysis system 1.

  The equipment abnormality aging change determination device 10 analyzes the equipment abnormality image data acquired from the on-site processing device 40 and determines the aging change of the abnormal part. In this case, the equipment abnormality temporal change determination device 10 performs image shape conversion on an image including the current equipment abnormality acquired from the on-site processing device 40 (hereinafter referred to as a collation image), and is determined in advance. Registration is performed on an image serving as a reference for registration (hereinafter referred to as a reference image). Then, the equipment abnormality temporal change determination device 10 creates an image of the abnormal part. Next, the equipment abnormality temporal change determination device 10 compares the created image of the current abnormal part with the past abnormal part image, and determines the temporal change of the abnormal state. A detailed configuration example of the equipment abnormality aging change determination device 10 will be described later.

  The imaging devices 20a and 20b capture an image of an abnormal part of the equipment abnormality of the power-related equipment 50. Among these, the imaging device 20a captures an image of visible light. Further, the imaging device 20b captures an image in a specific wavelength range that reacts to insulating oil and rust in accordance with the brightness of the illumination in conjunction with the illumination 30 that emits pulsed light. In FIG. 1, each of the imaging devices 20a and 20b is described as physically separated, but may be integrated into one unit. The imaging device 20b may acquire a collation image in a specific wavelength range by applying an optical filter to the imaging device 20a that captures an image of visible light. In addition, the images captured by the imaging devices 20a and 20b may not necessarily have the same angle of view or focal length, and the size and the like of both images are adjusted using image shape conversion described later. .

  The illumination device 30 emits pulsed light using a halogen light source or the like so as to be interlocked with the photographing timing of the imaging device 20b.

  The on-site processing device 40 is a portable device having a personal computer function used by the facility inspector at the site, and stores image data taken by the imaging devices 20a and 20b. The on-site processing device 40 and the imaging devices 20a and 20b can communicate with each other by wire or wirelessly. The on-site processing device 40 transmits the stored image data to the equipment abnormality aging change determination device 10 by wire or wireless. Note that the image data stored by the on-site processing device 40 may be transferred to the equipment abnormality temporal change determination device 10 via a data storage device such as a USB (Universal Serial Bus).

  The power-related equipment 50 is a transformer or the like that is a target for investigating changes with time in equipment abnormality.

Next, a configuration example of the equipment abnormality temporal change determination device 10 will be described with reference to FIG.
The equipment abnormality temporal change determination device 10 includes a communication unit 11, a processing unit 12, a storage unit 13, an input unit 14, and an output unit 15.

  The communication unit 11 is an interface that receives image data transmitted from the on-site processing device 40 by wire or wirelessly. The communication unit 11 may be an interface connected to a data storage device such as a USB. The communication unit 11 passes the received image data to the processing unit 12.

  The processing unit 12 is composed of a CPU (Central Processing Unit) of the computer and a main memory. The application program stored in the storage unit 13 is expanded in the main memory, and the image shape conversion process of the verification image is performed. The superimposing process of the image of the part and the image of the abnormal part in the past, the determination process of the temporal change of the abnormal part, and the like are realized. A detailed function example and processing flow of the processing unit 12 will be described later. The processing unit 11 controls information transmission between the units 11 to 15.

The storage unit 13 stores a photographed image DB 131, an analysis image DB 132, and a time-change DB 133.
The captured image DB 131 stores image data captured by the imaging devices 20a and 20b.
The analysis image DB 132 stores the reference image, the abnormal part image created by the processing unit 12, the positional information of the abnormal part, and the captured time information in association with each other.
The temporal change DB 133 stores an abnormal portion image created by the processing unit 12 and a determination result of the temporal change in association with each other. Note that the abnormal part image stored in the time-change DB 133 and the abnormal part image stored in the analysis image DB 132 are managed by identification information or the like so that they can be associated with each other.

The input unit 14 is a mouse or a keyboard, and operates a cursor or the like on the screen displayed on the output unit 15 to input information to the processing of the processing unit 12.
The output unit 15 is a display, and displays a processing result of the processing unit 12, a message for prompting input, and the like.

(Outline of how to superimpose changes in abnormal parts)
Next, an outline of a method for superimposing a change in an abnormal part in the equipment abnormality aging change determination apparatus 10 will be described with reference to FIG. 2 (see FIG. 1 as appropriate).
The reference image 201 (second image) is used as a reference when aligning the collation image 202 and is stored in the analysis image DB 132. The reference image 201 is an image taken when the abnormal portion 211 is first discovered. Therefore, the reference image 201 need only be acquired once. However, since there are portions that cannot be seen by shooting from one place, a plurality of reference images 201 may be provided as a set. In FIG. 2, the abnormal part 211 corresponds to an image of a past abnormal part.

  The collation image 202 (first image) is an image obtained by photographing the current state of equipment abnormality. In the verification image 202, an abnormal portion 212 is shown. The collation image 202 is an image taken at a time different from the time when the reference image 201 was taken, and may be taken at a different shooting position and a different shooting angle of view from the reference image 201.

The superimposed image 203 is a result of superimposing the reference image 201 and an image obtained by converting the matching image 202 so as to align the position with the reference image 201. In this way, the abnormal part superposition 213 can be obtained. In the abnormal part superposition 213, the abnormal part 211 is an image of a past abnormal part, and the abnormal part 212b is an image of a current abnormal part after image shape conversion. Moreover, in the superimposed image 203, although the electric power related equipment 50 is shown with the dotted line, this represents the case where the display of the electric power related equipment 50 is abbreviate | omitted. That is, only the superimposition 213 of the abnormal part is displayed so that the change with time can be easily determined.
Moreover, in the superimposition 213 of the abnormal part, the shape of the abnormal part 212b is larger than the shape of the abnormal part 211, indicating that the abnormal state is expanding. As described above, by analyzing the state of the superposition 213 of the abnormal part, it is possible to determine the temporal change of the abnormal part.

  The alignment between the reference image 201 and the collation image 202 includes the rectangular areas 61, 62, 63, 64 including the feature points of the power-related equipment 50 of the reference image 201 and the feature points of the power-related equipment 50 of the collation image 202. The rectangular areas 65, 66, 67, and 68 are used. A feature point is a point where the contrast is clear in the vicinity of the abnormal part. The feature points are determined by the method described later in each of the rectangular areas 61 to 68. And the correspondence of each feature point is calculated | required between the reference | standard image 201 and the collation image 202. FIG. In FIG. 2, feature points in the rectangular area 61 and feature points in the rectangular area 65, feature points in the rectangular area 62 and feature points in the rectangular area 66, feature points in the rectangular area 63 and features in the rectangular area 67. A point, a feature point in the rectangular area 64, and a feature point in the rectangular area 68 correspond to each other. The positions of the rectangular areas 61 to 68 are manually specified using the input unit 14 such as a mouse. In this example, the feature points in the rectangular areas 61 to 68 after the specification are determined by the processing unit 12. Shall do. Further, the range in which alignment is performed is preferably a plane, and in the case of a curved surface, it can be handled by representing the curved surface with a small plane.

  The collation image 202 is collated with the reference image 201 after image shape conversion. However, when the collation image 202 is reduced by the image shape conversion, the abnormal portion 212 is crushed after the image shape conversion (the state where the boundary of the abnormal portion 212b is unclear or the abnormal portion 212 becomes small and is difficult to understand. It is also possible that Therefore, it is preferable to emphasize the abnormal portion 212 in comparison with other portions. In particular, when shooting outdoors, the intensity of reflected light from the abnormal portion 212 changes with the influence of background light. Further, when the size of the abnormal portion 212 is small, it may be buried in noise and cannot be discriminated. For this reason, the imaging device 20b takes an image at each of the times when the intensity of the reflected light from the abnormal portion 212 becomes maximum and minimum by intermittently irradiating the abnormal portion 212 with pulsed light from the illumination 30 using halogen light as a light source. Shoot. Such photographing is performed a plurality of times on the same abnormal portion 212, and a difference is calculated in each pixel of both the image at the maximum intensity and the image at the minimum intensity. Then, the abnormal portion 212 is emphasized by integrating the calculated difference for each pixel. The reflected light can be imaged using the imaging device 20b in which an optical filter that transmits only a specific wavelength is set. Thus, by integrating the difference between the image at the maximum intensity and the image at the minimum intensity, the influence of background light is canceled out in the abnormal portion 212. Details regarding emphasis of the abnormal portion 212 will be described later.

(Abnormal status type)
Next, the types of abnormal states will be described with reference to FIG.
In the power-related equipment 50, equipment abnormalities such as oil leakage and rust may occur with aging. Such an equipment abnormality may progress in an abnormal state, but may also stop spontaneously. In addition, the abnormal part may disappear by repairing or painting the rust part. Therefore, in order to determine the progress of the abnormal part, the past abnormal part range and the current abnormal part range are compared to classify the temporal change of the abnormal part. In the description of FIG. 3, the time t1 is assumed to be the past and the time t2 is assumed to be the present.
FIG. 3 (a) shows a new occurrence. An abnormal portion that has not occurred at the past time t1 (however, the time closest to the present in the past) indicates a state where it has newly occurred at the current time t2.
FIG. 3B shows disappearance. The abnormal part which occurred at the past time t1 shows a state where it disappears at the current time t2. For example, an abnormality such as oil leakage or rust has disappeared due to repair or painting of the abnormal part.
FIG. 3C shows the enlargement. A state where the range of the abnormal part at the current time t2 includes the range of the abnormal part at the past time t1 is shown.
FIG. 3D shows the stop. The range of the abnormal portion at the current time t2 and the range of the abnormal portion at the past time t1 substantially coincide with each other (within a predetermined threshold), and the expansion of the range of the abnormal portion is stopped. Show. Note that the predetermined threshold value is set in consideration of an error or the like due to image crushing that occurs due to image resolution reduction or image shape conversion.
FIG. 3E shows the reduction. A state where the range of the abnormal portion at the past time t1 includes the range of the abnormal portion at the current time t2. For example, it is a case where an abnormal part such as rust or oil leakage occurs again after repairing the equipment.
FIG. 3F shows a change in position. The range of the abnormal portion at the past time t1 indicates a state in which the position has moved compared to the range of the abnormal portion at the current time t2. For example, this occurs when the equipment is tilted due to an earthquake or the like in a state where oil leakage or the like occurs.

(Functional example of equipment abnormality secular change judging device)
Next, a function example of the equipment abnormality aging change determination device 10 will be described with reference to FIG. 4 (see FIG. 1 as appropriate).
The image acquisition unit 401 acquires a reference image and a collation image via the communication unit 11. In addition, the image acquisition unit 401 stores the acquired reference image and collation image in the captured image DB 131.

  The difference image generation integration unit 402 acquires the collation image captured by the imaging device 20b from the image acquisition unit 401 or the captured image DB 131. Then, as described above, the difference image generation integration unit 402 calculates the difference in each pixel of both the image at the maximum intensity and the image at the minimum intensity, which are repeatedly captured, as described above. Is emphasized for each pixel. Note that an image in which this abnormal portion is emphasized is referred to as a specific wavelength matching image.

  The specific wavelength verification image conversion unit 403 performs coordinate conversion for aligning the specific wavelength verification image with the visible light verification image captured by the imaging device 20a. When capturing a verification image, both the imaging device 20a and the imaging device 20b are used as shown in FIG. When the optical axes of the lenses of the imaging device 20a and the imaging device 20b are the same, only the angle of view is different, and therefore, reduction / enlargement conversion is performed as coordinate conversion. Further, when the optical axis of the lens is inconsistent or shifted in parallel, coordinate value nonlinear conversion is performed as coordinate conversion.

  The region extraction unit 404 selects only pixels having a pixel value within a predetermined threshold range determined in advance in the specific wavelength verification image coordinate-converted by the specific wavelength verification image conversion unit 403, and collects the selected pixels. Extract as an abnormal part.

  Candidate point coordinate input unit 405 is a rectangular region (reference numerals 61 to 68 in FIG. 2) representing a region including a feature point for alignment in each of the reference image and visible light collation image displayed on output unit 15. Candidate points serving as a basis for determining the position are instructed via the input unit 14, and the coordinates of those candidate points are acquired.

  The feature point determination unit 406 sets a rectangular area having a predetermined size so as to include the candidate point based on the coordinates of the candidate point acquired by the candidate point coordinate input unit 405, and selects a feature point from the rectangular area. Auto detect. Then, the feature point determination unit 406 obtains the correspondence between the feature points between the reference image and the visible light matching image, and determines the corresponding point between the reference image and the visible light matching image.

  The conversion parameter calculation unit 407 automatically calculates a parameter for image shape conversion (hereinafter referred to as an image shape conversion parameter) based on the coordinates of the corresponding points determined by the feature point determination unit 406.

  The image deforming unit 408 uses the image shape conversion parameter calculated by the conversion parameter calculating unit 407, and the reference of the current abnormal part image (specific wavelength matching image) extracted by the region extracting unit 404 read from the captured image DB 131. Image shape conversion processing is performed to match the position of the image. Further, the image transformation unit 408 stores the image of the abnormal part subjected to the image shape conversion process in the analysis image DB 132. In particular, by creating metadata including an image of an abnormal part, it is possible to facilitate management such as reading and storing of the image of the abnormal part. An example of the metadata is shown in FIG. As shown in FIG. 5, the metadata associates a number (management number), shooting date (time), equipment position coordinates, abnormal part position coordinates, name of a specific wavelength collation image, and reference image name. It is a thing.

  The change extraction unit 409 extracts from the analysis image DB 132 a past abnormal part image used for collation with the current abnormal part image (specific wavelength collation image) whose image shape has been converted by the image transformation unit 408.

  The change determination unit 410 executes a predetermined calculation (described later) based on the pixel values of the current and past abnormal portion images and the positions of the pixels, and determines the type of abnormal state (see FIG. 3). A numerical value to be used is calculated and transferred to an enlargement / reduction stop determination unit 411 and an occurrence / annihilation determination unit 413 which will be described later. In addition, the change determination unit 410 passes information on the pixel values of the current and past abnormal part images and the positions of the pixels to the position change determination unit 412 described later. Further, the change determination unit 410 determines the content of the temporal change based on the determination results acquired from the enlargement / reduction stop determination unit 411, the position change determination unit 412, and the occurrence / disappearance determination unit 413. Further, the change determination unit 410 controls the order of processing in the enlargement / reduction stop determination unit 411, the position change determination unit 412, and the occurrence / disappearance determination unit 413.

  The enlargement / reduction stop determination unit 411 acquires a numerical value from the change determination unit 410. Next, the enlargement / reduction stop determination unit 411 transmits the acquired numerical value to the threshold value testing unit 414, and acquires a verification result by comparing the acquired numerical value with the threshold value. The enlargement / reduction stop determination unit 411 determines whether the enlargement / reduction or stop is based on the verification result, and transmits the determination result to the change determination unit 410.

  The position change determination unit 412 performs a predetermined calculation (described later) on the pixel value of the abnormal portion acquired from the change determination unit 410 and the position of the pixel to determine whether the position of the abnormal portion has moved. Calculate the numerical value to be used. Next, the position change determination unit 412 transmits the calculated numerical value to the threshold value testing unit 414, and acquires a verification result obtained by comparing the calculated numerical values. The position change determination unit 412 determines whether the position of the abnormal part has moved based on the verification result, and transmits the determination result to the change determination unit 410.

  The occurrence / annihilation determination unit 413 acquires a numerical value from the change determination unit 410. Next, the occurrence / annihilation determination unit 413 transmits the acquired numerical value to the threshold value testing unit 414, and acquires a verification result by comparing the acquired numerical value with the threshold value. The occurrence / annihilation determination unit 413 determines whether a new occurrence or disappearance is made based on the verification result, and transmits the determination result to the change determination unit 410.

  The threshold value verification unit 414 receives the numerical value received from the enlargement / reduction stop determination unit 411, the position change determination unit 412, or the occurrence / disappearance determination unit 413, and a predetermined threshold value stored in advance in order to determine the type of abnormal state The comparison operation of large and small is executed with respect to etc. Then, the threshold value verification unit 414 returns the result of the comparison operation as a verification result to any of the enlargement / reduction stop determination unit 411, the position change determination unit 412, and the occurrence / annihilation determination unit 413 that have transmitted the numerical value.

  The report data creation unit 415 creates a material related to equipment abnormality and outputs the created material to the output unit 15.

  Note that the communication unit 11, the input unit 14, the captured image DB 131, the analysis image DB 132, and the time-change DB 133 in FIG. 4 are the same as those shown in FIG. Each unit 401 to 415 represents the function of the processing unit 12.

(Example of superimposing current and past specific wavelength matching images)
Here, an example of a method for superimposing current and past specific wavelength matching images will be described with reference to FIG. 6 (see FIG. 1 as appropriate).
As for the superimposing method, when the optical axes of the lenses of the imaging device 20a and the imaging device 20b are matched (case 1), and when the optical axes of the lenses are mismatched or deviated in parallel (case 2). This will be explained separately.

(Case 1)
Case 1 is a case where the imaging device 20b is realized by mounting an optical filter on a lens unit or a sensor element portion of the imaging device 20a, for example. However, when each of the imaging devices 20a and 20b is configured separately, since the lens unit and the sensor element unit are generally different, the focal length and the angle of view may be different.

FIG. 6A shows a specific wavelength collation image of the abnormal part 501. In the specific wavelength matching image, the shape of the power-related equipment 502 or the like may not appear clearly on the image due to the wavelength characteristics, and therefore, is represented by a broken line.
FIG. 6B shows a composite image (first image) obtained by combining the visible light collation image obtained by photographing the power-related facility 503 and the specific wavelength collation image after the enlargement / reduction conversion. In order to create a composite image, as described above, the visible light collation image shown in FIG. 6B and the specific wavelength collation image shown in FIG. 6A are generally different in focal length and angle of view. Therefore, enlargement / reduction conversion is required. That is, the abnormal portion 501a shown in FIG. 6B is obtained by enlarging and reducing the abnormal portion 501 shown in FIG. Note that the diagram shown in FIG. 6B corresponds to the same state as the collation image 202 shown in FIG.

  Next, a process of matching the composite image of FIG. 6B with the size and position of the reference image (second image) of FIG. 6C is performed. Specifically, the feature points in the rectangular areas 71 to 78 are extracted, the correspondence between the feature points is obtained between the composite image in FIG. 6B and the reference image in FIG. The corresponding point of the image is determined. Then, the planes surrounded by the corresponding points are matched. In this way, the image shape conversion parameter is obtained. Since the reference image shown in FIG. 6C is a visible light collation image, the abnormal part may not be clearly shown. 6C corresponds to a state similar to that of the reference image 201 illustrated in FIG.

In FIG. 6D, the abnormal portion 501a of the composite image in FIG. 6B is image-shaped converted using the image shape conversion parameter, and the abnormal portion 501b is obtained as the conversion result of the specific wavelength matching image.
Next, in FIG. 6F, an abnormal portion in which the specific wavelength verification image of the past abnormal portion 505 shown in FIG. 6E and the specific wavelength verification image of the abnormal portion 501b of FIG. Are obtained.
Then, by analyzing the state of the superimposed image, it is possible to determine the type of abnormal state (abnormal change with time) as shown in FIG.

(Processing flow for judging changes in equipment abnormality over time)
A processing flow for determining a change in equipment abnormality over time will be described with reference to FIGS. 7 to 9 (see FIG. 4 as appropriate).
In step S <b> 701, the image acquisition unit 401 acquires a reference image and a collation image via the communication unit 11.
In step S702, the difference image generation integration unit 402 generates a specific wavelength matching image.
In step S703, the specific wavelength matching image conversion unit 403 performs enlargement / reduction conversion for aligning the specific wavelength matching image with the visible light matching image, and changes the angle of view of the specific wavelength matching image to the angle of view of the visible light matching image. To match.

  In step S704, the candidate point coordinate input unit 405 includes a rectangular area (reference numeral 61 in FIG. 2) that represents an area including feature points for alignment in each of the reference image and the visible light collation image displayed on the output unit 15. The candidate point that is the basis for determining the position of (68) to 68) is designated via the input unit 14. The candidate point coordinate input unit 405 acquires the coordinates of the designated candidate point.

  In step S705, the feature point determination unit 406 extracts feature points in a rectangular area set based on the candidate points specified in the reference image and the visible light collation image. For this process, a mask as shown in FIG. 10 is used. 10A shows a mask that emphasizes a change in pixel value in the left-right direction, FIG. 10B shows a mask that emphasizes a change in pixel value in the vertical direction, and FIG. 10C shows a pixel value in the upper right and lower left direction. FIG. 10D shows a mask for emphasizing pixel value changes in the upper left and lower right directions.

First, the feature point determination unit 406 sets a rectangular area having a predetermined size so as to include the candidate point specified in step S704. If the resolution coordinate of a pixel is (x, y),
(X-k, Y-k), (X- (k + 1), Yk), ..., (Xk, Y) (X- (k-1), Y), ..., (X + (k- 1), Y + k), (X + k, Y + k)
Is equivalent to setting the image range. Then, a 3 × 3 mask as shown in FIGS. 10A to 10D is applied to the gray value of the image, and the average value of the gray values of the image is calculated. The mask is given values of 0 and 1. If the coordinates of the center of the mask are (x, y), (x−m, y−n) (m = −1, 0, 1, n = − Let the luminance of the pixel of 1, 0, 1) be VAL (x−m, y−n). VAL (x−m, y−n) is represented by the following equation (1) when represented by an RGB (Red, Green, Blue) model.
VAL (x-m, y-n)
= R (x-m, y-n) × 0.2989 + G (x-m, y-n) × 0.5866 + B (x-m, y-n) × 0.1145
... Formula (1)

At this time, for example, the mask value TotalVal in FIG. 10A is expressed by the following equation (2).
TotalVal = VAL (x, y-1) + VAL (x, y) + VAL (x, y + 1) (2)
The feature point determination unit 406 calculates the TotalVal for each pixel by shifting the masks shown in FIGS. 10A to 10D by one pixel within the rectangular area. Next, four TotalVal calculated using four types of masks shown in FIGS. 10A to 10D are added for each pixel. Then, in the rectangular area, the pixel position where the added TotalVal is the maximum is obtained. In such a mask process, the added TotalVal is maximized in a pixel corresponding to a bend of the image or the like (for example, a place where the contrast such as sides and corners of the outer shape of the power-related equipment 50 is clear). That is, it is possible to extract a planar feature point including an abnormal part. In general, since there are a plurality of pixels having the maximum TotalVal added in the same rectangular area, there may be a plurality of extracted feature points in the same rectangular area.

  Therefore, in step S706, the conversion parameter calculation unit 407 selects a feature point for each rectangular area set based on the candidate points of the visible light matching image. When there are a plurality of feature points in the same rectangular area, the feature points are selected at random.

  In step S707, the conversion parameter calculation unit 407 calculates an image shape conversion parameter based on the correspondence between the feature points of the reference image and the visible light matching image. First, an image shape conversion parameter P for performing image shape conversion between the reference image and the visible light matching image is obtained for the feature point of the reference image selected in step S706 and the feature point of the visible light matching image.

As an image shape conversion parameter, P is set as in the following formula (3).
P = (a ₀ , a ₁ , a ₂ , b ₀ , b ₁ , b ₂ , 1, c ₁ , c ₂ ) ^T Equation (3)
Here, T represents a transposed matrix.
Further, a transformation matrix H for coordinate transformation between feature points of the reference image and the visible light collation image is represented by the following equation (4).

ただし、上記式（４）において、本実施形態では特徴点の数は４つであるので、ｋ＝４である。また、基準画像の４つの特徴点の座標がそれぞれ(x₁,y₁)，(x₂,y₂)，(x₃,y₃)，(x₄,y₄)であり、可視光照合画像の４つの特徴点の座標がそれぞれ(X₁,Y₁)，(X₂,Y₂)，(X₃,Y₃)，(X₄,Y₄)であるものとする。 (Formula 1)

However, in the above formula (4), since the number of feature points is four in this embodiment, k = 4. The coordinates of the four feature points of the reference image are (x ₁ , y ₁ ), (x ₂ , y ₂ ), (x ₃ , y ₃ ), and (x ₄ , y ₄ ), respectively, and visible light verification Assume that the coordinates of the four feature points of the image are (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), (X ₃ , Y ₃ ), and (X ₄ , Y ₄ ), respectively.

In addition, P shown in Formula (3) is
HP = 0 Formula (5)
To be satisfied.

  Therefore, each parameter of P is obtained by the Gauss-Newton method. Specifically, the calculation is performed by putting the provisional value of the element of P in the equation (5), and an error from the element value of P calculated once before is obtained by repeated calculation. Then, an error is added to the new value of P, and equation (5) is calculated again. When the value on the left side shown in Expression (5) is smaller than a predetermined threshold value (a value close to 0) or when a predetermined number of repetitions is executed, the repeated calculation is terminated. Thereby, the element value of P, that is, the image shape conversion parameter can be acquired.

That is, the conversion from the coordinates of the visible light collation image to the coordinates of the reference image uses the image shape conversion parameter when the coordinates of the visible light collation image are (x, y) and the coordinates of the reference image are (x, y). Are expressed as in the following formulas (6) and (7).
X = (a ₀ + a ₁ x + a ₂ y) / (1 + c ₁ x + c ₂ y) (6)
Y = (b ₀ + b ₁ x + b ₂ y) / (1 + c ₁ x + c ₂ y) (7)

Next, in steps S <b> 801 to S <b> 803, the image transformation unit 408 determines a correspondence relationship between the feature points of the visible light collation image and the feature points of the reference image.
First, in step S801, the image transformation unit 408 uses the image shape conversion parameter acquired in step S707 (see FIG. 7) to set the feature point of the visible light matching image selected in step S706 as the coordinate on the reference image. A converted point is obtained, and a feature point of the reference image near the point is selected.

  In step S802, the image transformation unit 408 calculates the sum of the errors, using the distance between the point converted to the coordinates on the reference image in step S801 and the feature point of the selected reference image as an error.

In step S803, the image transformation unit 408 determines whether the total error is equal to or less than a predetermined threshold.
If it is determined that the total sum of errors is not less than or equal to the predetermined threshold (No in step S803), the process returns to B (step S706 in FIG. 7).
If it is determined that the total sum of errors is equal to or less than the predetermined threshold (Yes in step S803), the process proceeds to step S804.

Here, a specific example of the error calculation method will be described with reference to FIG. There are six feature points A, B, C, D, E, and F on the visible light collation image shown in FIG. Further, it is assumed that there are six feature points G, H, I, J, K, and L on the reference image shown in FIG. Now, it is assumed that four feature points A, B, C, and D are selected by the user on the visible light collating image shown in FIG. These feature points A, B, C, and D are converted into points Aa, Ba, Ca, Da on the reference image shown in FIG. 11B by using the equations (6) and (7) obtained in step S707. Is converted to Next, it is assumed that the characteristic points of the reference image in the vicinity of the points Aa, Ba, Ca, and Da on the reference image are G, H, I, and J, respectively. As described above, A-G, B-H, C-I, and D-J are combined, and the respective points are A (x ₁ , y ₁ ), G (X ₁ , Y ₁ ), B (x ₂ , y ₂ ), H (X ₂ , Y ₂ ), C (x ₃ , y ₃ ), I (X ₃ , Y ₃ ), D (x ₄ , y ₄ ), J (X ₄ , Y ₄ ) Equation (5) is calculated to calculate the image shape conversion parameter P.

  The remaining feature points E and F on the visible light collation image shown in FIG. 11A are also converted to coordinates on the reference image. The feature points E and F are converted into points Ea and Fa on the reference image shown in FIG. Then, feature points K and L on the reference image in the vicinity of the points Ea and Fa are selected. In general, the number of feature points of the visible light collation image and the number of feature points on the reference image often do not coincide with each other, and are ignored if there is no feature point within a predetermined threshold range. Then, A-G, B-H, C-I, D-J, E-K, and F-L are combined, and the distance between the points is calculated as an error to obtain the sum. If the sum of the errors is equal to or less than a predetermined threshold value, the calculation is terminated. However, if it is larger than the predetermined threshold, four new feature points of the visible light collation image shown in FIG. 11A are selected at random (step S706), and steps S707 and S801 to S803 are performed again. Run.

  As described above, in the method of processing in steps S706, S707, S801 to S803, first, the image shape conversion parameter P is calculated using the selected feature point, and the feature point on the visible light collation image is converted to a point on the reference image. The selection is performed in consideration of the correlation of selecting the feature point of the reference image near the converted point on the reference image. Thereby, there is an effect that the calculation processing time is shortened as compared with the case where the feature point is selected at random without considering the correlation and the image shape conversion parameter P is calculated.

  In step S804, the image deforming unit 408 performs image shape conversion on the specific wavelength matching image in order to align the specific wavelength matching image with the reference image. Specifically, the image transformation unit 408 converts the coordinate position of the specific wavelength collation image into a position on the coordinate of the reference image using the image shape conversion parameter. The position (x, y) on the coordinates of the specific wavelength collation image and the position (X, Y) on the coordinates of the reference image are converted using Expressions (6) and (7).

  In step S805, the image transformation unit 408 determines whether the conversion is successful. Specifically, when it is determined that the result of the image shape conversion is visually good, the image transformation unit 408 receives information on the success of conversion via the input unit 14 and determines that the conversion is successful. In addition, the image transformation unit 408 receives the information indicating that the result of the image shape conversion is visually unsatisfactory and receives the information indicating that the conversion is unsuccessful via the input unit 14 or the information indicating the successful conversion even after a predetermined time has elapsed. If it cannot be received, it is determined that the conversion has not succeeded.

When it is determined that the conversion is successful (Yes in step S805), the image transformation unit 408 stores the image shape conversion parameter in the storage unit 13. Also, the process proceeds to step S806.
If it is determined that the conversion has not succeeded (No in step S805), the process returns to C (step S704 in FIG. 7).

  In step S806, the change extraction unit 409 acquires from the analysis image DB 132 a past specific wavelength collation image for collation with the specific wavelength collation image whose image shape has been converted.

In step S807, the change determination unit 410 determines whether there is an abnormal part that has not been determined.
When it is determined that there is no abnormal portion that has not been determined (No in step S807), in step S809, the report data creation unit 415 determines the type of abnormal state determined in steps S901 to S913 (see FIG. 9). Along with the image of the abnormal part, it is stored in the time-change DB 133 as report data and displayed on the output unit 15. The report data format may be arbitrary. Also, in the report data, it is recommended to change the color of the text etc. to alert the manager of the power-related equipment about the oil leakage / rust abnormality that is expanding.

When it is determined that there is an abnormal portion that has not been determined (Yes in step S807), in step S808, the change determination unit 410 adjusts the pixels of the abnormal portion.
As shown in FIG. 3, there are six types of abnormal states: new occurrence, disappearance, enlargement, stop, reduction, and position change.
Therefore, the change determination unit 410 counts the number U of pixels of the abnormal portion superimposed from the positional relationship of the past and current specific wavelength matching images in order to classify these abnormal states. In addition, the number V of pixels that are not superimposed in the abnormal portion of the current specific wavelength matching image and the number W of pixels that are not superimposed in the abnormal portion of the past specific wavelength matching image are counted.
At this time, when the following expressions (8) and (9) are satisfied, the change determination unit 410 performs adjustment to incorporate pixels that are not overlapped into the overlapping range as being overlapped.
V <U × SD (8)
W <U × SD (9)
Here, SD is a value obtained by dividing the standard deviation of the number U of pixels in the overlapping portion by the number U of pixels in the overlapping portion.

  Note that the SD may be changed depending on an imaging angle difference between the reference image and the specific wavelength comparison image. That is, if the angle difference is α, SD may be SD [α]. If the shooting position is known by GPS (Global Positioning System) or the like, the angle difference can be obtained from the difference between the position when the reference image is captured and the position when the current image is captured using the equipment plan view. it can. In particular, when the image taken this time is taken from the horizontal direction and enlarged by image shape conversion, the angle difference is taken into consideration that the number of superimposed pixels is calculated to be larger. SD [α] may be estimated smaller as α is larger.

In step S <b> 901, the occurrence / annihilation determination unit 413 analyzes the new occurrence based on the pixel value of the abnormal part acquired from the change determination unit 410 and the position of the pixel. Specifically, the occurrence / annihilation determination unit 413 is a case where no abnormal portion exists in the collation image taken at the past time point closest to the present time, or a case where no past abnormal portion image exists. When an abnormal portion exists in the current collation image, it is determined that a new occurrence has occurred. Then, the occurrence / annihilation determination unit 413 transmits a new generation determination result to the change determination unit 410.
In step S902, the change determination unit 410 determines whether or not the type of abnormal state is a new occurrence.
If it is determined that a new occurrence has occurred (Yes in step S902), the process returns to G (step S807 in FIG. 8).
If it is determined that no new occurrence has occurred (No in step S902), the process proceeds to step S903.

In step S903, the occurrence / annihilation determination unit 413 analyzes the disappearance using the numerical value acquired from the change determination unit 410 (the numerical value calculated by equation (10) described later). Specifically, the occurrence / annihilation determination unit 413 calculates a numerical value obtained by calculating the ratio of the past abnormal part pixel number C _old and the current abnormal part pixel number C _new by the following equation (10) in the change determination unit 410. get.
Overlap _{= C} old _/ C _new ···· formula (10)
Then, the occurrence / annihilation determination unit 413 transmits the numerical value overlap calculated by Expression (10) to the threshold value verification unit 414. The threshold verification unit 414 verifies whether or not the following formula (11) is satisfied.
Overlap ≧ σ1 ・ ・ ・ ・ Formula (11)
However, σ1 (first threshold value) is a predetermined value.
The threshold value verification unit 414 transmits a verification result as to whether or not Expression (11) is satisfied to the occurrence / annihilation determination unit 413. The generation / annihilation determination unit 413 determines that it is extinguished when a verification result that satisfies Expression (11) is received. Then, the occurrence / annihilation determination unit 413 transmits the determination result of the disappearance to the change determination unit 410.

In step S904, the change determination unit 410 determines whether or not the abnormal state type is extinguished.
If it is determined that it is extinguished (Yes in step S904), the process returns to G (step S807 in FIG. 8).
If it is determined that it is not extinguished (No in step S904), the process proceeds to step S905.

In step S <b> 905, the enlargement / reduction stop determination unit 411 analyzes the expansion using the numerical overlap obtained from the equation (10) acquired from the change determination unit 410. Specifically, the enlargement / reduction stop determination unit 411 transmits a numerical overlap to the threshold verification unit 414 when determining that the current abnormal part includes the past abnormal part. The threshold verification unit 414 verifies whether or not the following formula (12) is satisfied.
Overlap <σ2 <1 Equation (12)
However, σ2 (second threshold value) is a predetermined value.
The threshold value verification unit 414 transmits the verification result as to whether or not Expression (12) is satisfied to the enlargement / reduction stop determination unit 411. The enlargement / reduction stop determination unit 411 determines that the enlargement occurs when the verification result that the expression (12) is satisfied is received. The enlargement / reduction stop determination unit 411 transmits the enlargement determination result to the change determination unit 410.

In step S906, the change determination unit 410 determines whether or not the type of abnormal state is enlargement.
If it is determined that the image is enlarged (Yes in step S906), the process returns to G (step S807 in FIG. 8).
If it is determined that the image is not enlarged (No in step S906), the process proceeds to step S907.

In step S <b> 907, the enlargement / reduction stop determination unit 411 analyzes the reduction using the numerical overlap obtained from the equation (10) acquired from the change determination unit 410. Specifically, when the enlargement / reduction stop determination unit 411 determines that the current abnormal portion is included in the past abnormal portion, the enlargement / reduction stop determination unit 411 transmits a numerical value overlap to the threshold verification unit 414. The threshold verification unit 414 verifies whether or not the following equation (13) is satisfied.
σ1>Overlap>σ3> 1 (13)
However, σ3 (third threshold value) is a predetermined value.
The threshold value verification unit 414 transmits a verification result as to whether or not Expression (13) is satisfied to the enlargement / reduction stop determination unit 411. When the enlargement / reduction stop determination unit 411 receives a verification result that satisfies the expression (13), the enlargement / reduction stop determination unit 411 determines that the image is reduced. Then, the enlargement / reduction stop determination unit 411 transmits the reduction determination result to the change determination unit 410.

In step S908, the change determination unit 410 determines whether or not the abnormal state type is reduction.
If it is determined to be reduced (Yes in step S908), the process returns to G (step S807 in FIG. 8).
If it is determined that the image is not reduced (No in step S908), the process proceeds to step S909.

In step S <b> 909, the enlargement / reduction stop determination unit 411 analyzes the stop using the numerical overlap obtained from Expression (10) acquired from the change determination unit 410. Specifically, the enlargement / reduction stop determination unit 411 transmits a numerical value overlap to the threshold verification unit 414. Then, the threshold verification unit 414 verifies whether or not the following formula (14) is satisfied.
σ4 <Overlap <σ5 ・ ・ Formula (14)
However, σ4 (fourth threshold value) and σ5 (fifth threshold value) are predetermined values, and 1−α = σ4 <σ5 = 1 + β, (0 <α <β <1).
The threshold value verification unit 414 transmits a verification result as to whether or not Expression (14) is satisfied to the enlargement / reduction stop determination unit 411. The enlargement / reduction stop determination unit 411 determines that it is stopped when it receives a verification result that satisfies Expression (14). Then, the enlargement / reduction stop determination unit 411 transmits a stop determination result to the change determination unit 410.

In step S910, the change determination unit 410 determines whether or not the type of abnormal state is stopped.
If it is determined that the operation is stopped (Yes in step S910), the process returns to G (step S807 in FIG. 8).
If it is determined not to stop (No in step S910), the process proceeds to step S911.

In step S911, the position change determination unit 412 analyzes the position change using the pixel value of the abnormal portion acquired from the change determination unit 410 and the position of the pixel. Specifically, the position change determination unit 412 determines the number U of pixels superimposed from the positional relationship between the past and current abnormal parts (third pixel number) and the number of pixels that are not superimposed in the current abnormal part. V (first number of pixels) and the number W (second number of pixels) of pixels in a portion that has not been superposed in the past abnormal portion are tabulated. Then, the position change determination unit 412 transmits the numbers U, V, and W of each pixel to the threshold verification unit 414. The threshold verification unit 414 verifies whether or not the following expressions (15) and (16) are satisfied.
V> U × SD Expression (15)
W> U × SD Expression (16)
The threshold value verification unit 414 transmits a verification result as to whether or not Expression (15) and Expression (16) are satisfied to the position change determination unit 412. The position change determination unit 412 determines that it is a position change when receiving the verification result that the expressions (15) and (16) are satisfied. Then, the position change determination unit 412 transmits the position change determination result to the change determination unit 410.

In step S912, the change determination unit 410 determines whether or not the type of the abnormal state is a position change.
If it is determined that the position is changed (Yes in step S912), the process returns to G (step S807 in FIG. 8).
If it is determined that the position is not changed (No in step S912), the process proceeds to step S913.

  In step S <b> 913, the change determination unit 410 determines that the abnormal state type cannot be classified by automatic processing, and accepts an input of the abnormal state type determined by the user via the input unit 14 (prompts input).

  In steps S903, S905, S907, and S909, by setting the threshold values in multiple stages, the size of erosion due to rust, the degree of oil leakage, etc. may be classified into a plurality of stages. Then, an image of an abnormal portion such as a portion where oil leakage or rust is remarkably progressing, a portion where progress is slow, or a portion where traces of oil leaked dry may be classified.

  As described above, the processing flows shown in FIGS. 7 to 9 relate to the case 1 that shows the case where the optical axes of the lenses of the imaging devices 20a and 20b are matched.

(Case 2)
Next, a case where the optical axes of the lenses of the imaging device 20a and the imaging device 20b are mismatched or shifted in parallel will be described.
In case 2, the focal lengths of the cameras of the imaging devices 20a and 20b, the distance between the camera lens centers, and the distance from the imaging plane to the target point are important. By a simple modification of the basic equation of camera geometry, the relationship between the visible light collation image and the special wavelength collation image is expressed by the following equations (17) and (18).
X = {f2 (xD−f1d)} / (f1D) (Equation 17)
Y = {f2 (yD−f1d)} / (f1D) (18)
Here, D >> d.
Each variable is as follows.
f1: Focal length of the imaging device 20a that captures the visible light collation image f2: Focal length of the imaging device 20b that captures the specific wavelength verification image D: Distance between the imaging plane and the object d: Between the optical axes of the two types of cameras Distance of
x: position of the object in the specific wavelength collation image (X coordinate on the imaging plane)
y: position of the object in the specific wavelength collation image (Y coordinate on the imaging plane)
X: the position of the object of the visible light collation image (X coordinate on the imaging plane)
Y: the position of the object of the visible light collation image (Y coordinate on the image plane)

  From equations (17) and (18), it can be seen which coordinate position (x, y) on the specific wavelength comparison image corresponds to the coordinate position (X, Y) of the visible light comparison image. Such coordinate transformation is non-linear transformation. Thus, when the coordinate position (X, Y) is specified, the shape of the pixel of the specific wavelength band image corresponding to the coordinate position (x, y) can be changed.

  Here, the lens surfaces of the two cameras are both set so that the optical axes are parallel, the focal lengths f1 and f2 are known, and the distance d between the optical axes can also be measured. To do. However, it is assumed that the distance D between the imaging plane and the object is not obtained.

  When the distance D is not obtained, the coordinate position (X, Y) and the coordinate position (x, y) are not associated with each other, as can be seen from the expressions (17) and (18). Therefore, there is a method for obtaining the distance D by laser ranging, but it can be obtained by the following method with some conditions. One point (x, y) of the specific wavelength matching image shown in FIG. 6A and one point (X, Y) of the visible light matching image shown in FIG. 6B clearly correspond to each other. If it is known, D can be obtained by substituting those coordinate values into the equations (17) and (18). That is, the image shape conversion parameter can be obtained by designating one candidate point for each image, executing the processes in steps S704 and S705, and obtaining the feature points. Thereby, the coordinates of the pixel of the specific wavelength image corresponding to each pixel of the visible light collation image can be obtained.

As described above, the shape conversion of the specific wavelength matching image in step S804 is replaced with two steps as follows.
First, the pixel of the specific wavelength collation image is selected, and its coordinates are substituted into the equations (17) and (18) to be converted into the coordinates of the visible light collation image.
Next, the position of the pixel converted into the coordinates of the visible light collation image is converted by the equations (6) and (7) and converted into the position of the reference image.

(Verification image acquisition flow)
Next, an example of an acquisition flow of the visible light collation image and the specific wavelength collation image will be described with reference to FIG. 12 (see FIGS. 1 and 4 as appropriate).
In step S1201, the on-site processing device 40 acquires a visible light region image (visible light collation image) captured by the imaging device 20a. This visible light collation image becomes a collation image for acquiring feature points.

  In step S <b> 1202, the on-site processing device 40 determines whether or not integration of difference images described later has been performed a predetermined number of times. If it is determined that the predetermined number of times has been executed, the process proceeds to step S1206. If it is determined that the predetermined number of times has not been executed, the process advances to step S1203.

The processing in steps S1203 to S1208 is executed to improve the S / N ratio of the specific wavelength matching image.
In step S1203, the processing apparatus 40 in the field sets a timer for designating timing for irradiating halogen light or the like as irradiation light from the illumination apparatus 30. Then, the on-site processing device 40 emits irradiation light at predetermined time intervals, determines the maximum and minimum times of the reflected light intensity in advance, and uses the image pickup device 20b at this time to generate an image. Shoot. The number of irradiations is determined in advance.

  In step S1204, the on-site processing device 40 emits irradiation light from the lighting device 30 based on a timer.

  In step S1205, the on-site processing device 40 acquires an image at the maximum time and the minimum time of the reflected light intensity determined in advance using the imaging device 20b. For example, FIG. 13A shows an example of the relationship between time and reflected light intensity. In FIG. 13A, an image having the maximum reflected light intensity (third image) is acquired at time t1 and time t3. In addition, at time t2 and time t4, an image having the minimum reflected light intensity (fourth image) is acquired. Then, the process returns to step S1202.

  In step S <b> 1206, the image acquisition unit 401 of the equipment abnormality temporal change determination device 10 stores the image acquired from the on-site processing device 40 in the captured image DB 131.

  In step S1207, the difference image generation integration unit 402 calculates an image difference between an image captured when the reflected light intensity is maximum and an image captured when the reflected light intensity is minimum. Specifically, the difference image generation integration unit 402 calculates a difference between pixel values (hue) for each pixel to generate a difference image.

  In step S1208, the difference image generation / accumulation unit 402 integrates the difference for each pixel using the plurality of generated difference images.

  For example, FIG. 13B shows a process of integrating difference images. In FIG. 13B, it is assumed that one cell represents one pixel, and each of the diagrams shown in (A) to (H) represents an image of 5 × 5 pixels. (A) is an initial value image for integrating difference images. As an example of the initial value image, all pixel values (hue) are all zero. (B) is an image at time t1, (C) is an image at time t2, and the pixel value of the image of (C) is subtracted from the pixel value of the image of (B) to obtain the difference image of (D). Generate. Next, the difference image of (D) and the initial value image of (A) are integrated to create an integrated image of (E). Further, (E) is an image at time t3, (F) is an image at time t4, and the difference between (G) is obtained by subtracting the pixel value of the image (F) from the pixel value of the image (E). Create an image. Then, the difference image of (G) and the integrated image of (E) are integrated to create an integrated image of (H).

For example, the pixel value (hue) at time t1 when the reflected light intensity is maximum is A, and the pixel value (hue) at time t2 when the reflected light intensity is minimum is B. A pixel that is not an abnormal part (for example, a pixel denoted by reference numeral 1301) does not react to oil leakage or rust, and therefore the reflected light intensity is substantially constant, so that A = B. When the calculation for subtracting B from A is executed, the difference becomes 0 or a value close to 0. That is, the influence of background light can be removed.
In addition, in a pixel in an abnormal part where an abnormality has occurred (for example, a pixel denoted by reference numeral 1302), a value (difference value) obtained by subtracting B from A is different from a pixel value of a pixel that is not an abnormal part. In this way, an abnormal part and a part that is not an abnormal part can be distinguished. Further, by integrating the difference values, the abnormal part is emphasized. As a result, even when the intensity of the background light is different between summer and winter, the S / N ratio, which is the ratio between the true value signal and the noise error, can be increased by accumulating the differential images acquired repeatedly.

  In FIG. 12, in steps S <b> 1202 to S <b> 1205, it has been described that the processing apparatus 40 in the field repeatedly acquires an image and then transmits the image to the equipment abnormality temporal change determination apparatus 10. However, the difference images may be integrated by performing the processing of steps S1206 to S1208 immediately after the processing of steps S1203 to S1205 and returning to step S1202.

(Case where feature points are set with markers)
In the above description, the feature point is a point where the contrast is clear in the vicinity of the abnormal part. However, abnormal parts may appear on a wide plane. In this case, there is a possibility that feature points cannot be extracted even using the mask shown in FIG. Therefore, feature points can be easily extracted by using markers. When the marker is different from the color of the equipment or has a characteristic shape, the position of the marker can be recognized by shape recognition. Note that five markers are placed (applied) in the vicinity of the abnormal part.

  An outline of the alignment between the collation image using the marker and the reference image will be described with reference to FIG. 14A shows a reference image, FIG. 14B shows a collation image, and FIG. 14C shows a result of superimposing the reference image and the collation image.

  In FIG. 14, the markers 1401 to 1404 (first marker) of the reference image shown in FIG. 14A are associated with the markers 1405 to 1408 (first marker) of the collation image shown in FIG. Shall be. As a result, the abnormal portions 1421a and 1422a of the collation image are collated with respect to the abnormal portions 1421 and 1422 of the reference image. That is, as shown in FIG. 14C, the abnormal portion 1421 of the past reference image and the abnormal portion 1421b of the current collation image after image shape conversion are superimposed, and the abnormal portion 1422 and the image shape of the past reference image are superimposed. The abnormal portion 1422b of the current collated image after conversion is superimposed. Thereby, it is possible to determine a change with time of the abnormal portion.

Next, a processing flow for determining a change in equipment abnormality over time when a marker is used will be described with reference to FIG. 15 in comparison with the processing flows shown in FIG. 7 to FIG. 9).
In step S1501, the processes in steps S701 to S703 shown in FIG. 7 are executed.

  Step S1502 is processing corresponding to step S705. The feature point determination unit 406 extracts markers (feature points) by automatic recognition in the reference image and the visible light collation image. When the color of the marker is different from the color of the equipment, the marker is recognized by color recognition. In this case, since the marker is different from the color of the other region, the pixel having the marker color can be searched and the center thereof can be used as the feature point. Furthermore, when the shape of the marker is a triangle having a specific size, the feature point is acquired by recognizing this shape. Five markers are installed, and one of these is used for position verification.

Since step S1503 is the same processing as step S706, description thereof is omitted.
Since step S1504 is the same process as step S707, description thereof is omitted.

  Step S1505 is processing corresponding to step S801. Using the image shape conversion parameter acquired in step S1504, the image transformation unit 408 obtains a point obtained by converting the feature point of the visible light collation image selected in step S1503 into the coordinates on the reference image, and near the point. A feature point of a reference image is selected. Here, if it is not necessary to rotate the image, an image shape conversion parameter is obtained from four points of the marker (first marker). The remaining one marker (second marker) is used for verifying whether or not the marker matches the corresponding marker in the reference image when the marker is transformed.

  Step S1506 is processing corresponding to step S802. The image deforming unit 408 calculates the sum of the errors using the distance between the point converted to the coordinates on the reference image in step S1505 and the feature point of the selected reference image as an error.

In step S1507, the image transformation unit 408 determines whether or not the total sum of errors is minimum. That is, when the sum of errors is minimized, it is determined that the feature points of the visible light collation image and the feature points of the reference image have been associated.
If it is determined that the total sum of errors is minimum (Yes in step S1507), the process proceeds to step S1508.
If it is determined that the total sum of errors is not the minimum (No in step S1507), the process returns to step S1503.

Since step S1508 is the same processing as step S804, description thereof will be omitted.
Since step S1509 is the same process as step S805, description thereof is omitted.

  In step S1510, the processes of steps S806 to S809 (FIG. 8) and S901 to S913 (FIG. 9) are executed.

(Man-machine interface for information on abnormal parts)
The method for superimposing an abnormal part described so far can be applied to power-related equipment, particularly substation equipment. At that time, it is necessary to manage in which equipment and where the abnormal part is located. Therefore, if the facility where the abnormal part exists is plotted in association with a plan view of a substation or the like, the position of the inspector or facility manager can be easily recognized.

  For example, when a GPS (Global Positioning System) function is added to the imaging device 20, the position coordinates acquired by the GPS are associated with the plan view. And when performing an inspection, when coming to a nearby place, the inspector may be notified by voice or warning sound.

Here, a man-machine interface for information on an abnormal part will be described with reference to FIG. 16 (see FIG. 1 as appropriate).
A plan view 1601 shown in FIG. 16 represents a state displayed on the output unit 15 and represents power-related equipment such as a substation. In the plan view 1601, a power-related facility 1611 such as a transformer is displayed. A cursor 1602 and an icon 1603 are displayed on the screen of the output unit 15.

  When the icon 1603 is selected (clicked) via the input unit 14, an image of the power related equipment 1611 is retrieved and displayed as shown in the middle right diagram 1612 of FIG. Further, when the icon 1613 in FIG. 1612 is selected (clicked), an image of the abnormal portion 1615 is displayed as shown in the lower right diagram 1614 of FIG. 16, and a past image of the abnormal portion 1615 is also displayed. That is, the image of the abnormal portion 1615 and its history are displayed with reference to the metadata shown in FIG.

  Specifically, when the processing unit 12 of the equipment abnormality aging change determination apparatus 10 detects selection (click) on the icon 1602 in the plan view 1601, the coordinate position of the icon 1602 is set to the position coordinate of the equipment in the metadata in FIG. And the metadata shown in FIG. 5 is searched. The processing unit 12 searches the column of the position coordinates of the equipment, and selects all the information corresponding to the coordinates within a predetermined range from the coordinate point of the clicked icon. Then, the processing unit 12 displays a reference image from the selected information. At this time, if the selected information is one, a reference image corresponding to the information is displayed. In addition, when there are a plurality of pieces of selected information, all the reference images corresponding to the information are displayed, and a message that prompts the user to select one of them is also displayed.

  A panoramic image including a plurality of facilities in one image may be taken. In such a case, a line segment icon 1621 (vertical bar) represented by a line segment shape is used instead of the shape of the icon 1603 (circular in FIG. 16). When this line segment icon 1621 is selected (clicked), a plurality of equipment images 1622 related to the line segment are displayed. An icon 1623 indicating an abnormal portion is displayed on the plurality of facilities displayed on the displayed image 1622. When detecting (clicking) the icon 1623, the processing unit 12 searches the metadata shown in FIG. 5, displays the abnormal image 1624, and displays the abnormal portion 1625 and its history.

  As described above, the equipment abnormality temporal change determination device 10 according to the present embodiment first acquires the specific wavelength reference image of the abnormal part 501 as shown in FIG. 6A in order to determine the temporal change of the abnormal part. Moreover, the equipment abnormality temporal change determination apparatus 10 creates a composite image shown in FIG. This composite image is a composite of the visible light collation image obtained by photographing the power-related equipment 503 and the specific wavelength collation image after the enlargement / reduction conversion. Next, the equipment abnormality temporal change determination device 10 executes a process of matching the composite image of FIG. 6B with the size and position of the reference image of FIG. Specifically, the equipment abnormality temporal change determination device 10 extracts feature points in the rectangular regions 71 to 78, and features points between the composite image in FIG. 6B and the reference image in FIG. The correspondence between the images is obtained, and the corresponding points of both images are determined. Then, the planes surrounded by the corresponding points are matched. In this way, the image shape conversion parameter is obtained. Further, as shown in FIG. 6 (d), the equipment abnormality temporal change determination device 10 performs image shape conversion on the abnormal portion 501a of the composite image in FIG. 6 (b) using the image shape conversion parameter, and performs specific wavelength verification. An abnormal portion 501b is created as a result of image conversion. Next, in FIG. 6 (f), the equipment abnormality aging change determining device 10 in FIG. 6 (f), the specific wavelength collation image of the past abnormal portion 505 shown in FIG. A superimposed image of the abnormal part is created by superimposing the verification image. Then, the equipment abnormality temporal change determination device 10 can determine the type of abnormal state as shown in FIG. 3 by analyzing the state of the superimposed image.

  In addition, the equipment abnormality temporal change determination device 10 causes the illumination device 30 to emit light intermittently, and captures an image with the imaging device 20b at each of the times when the intensity of reflected light from the abnormal portion is maximum and minimum. The imaging device 20b sets an optical filter that transmits only a specific wavelength. The equipment abnormality temporal change determination device 10 obtains an image at the maximum intensity and an image at the minimum intensity obtained by performing such imaging a plurality of times for the same abnormal part, and obtains both pixel values (hue) ) Is calculated for each pixel. Then, the equipment abnormality temporal change determination device 10 enhances the abnormal part by integrating the calculated difference for each pixel.

In the present embodiment, the reference image is a visible light image in which feature points can be easily detected. However, when the feature points are easier to detect when captured at a wavelength different from the wavelength at which the specific wavelength matching image is captured. May be an image in that wavelength region.
In FIG. 4, the functions of the change determination unit 410, the enlargement / reduction stop determination unit 411, the position change determination unit 412, the occurrence / annihilation determination unit 413, and the threshold verification unit 414 have been described separately. You may comprise as a function of the change determination part 410. FIG.

DESCRIPTION OF SYMBOLS 1 Equipment abnormality change analysis system 10 Equipment abnormality temporal change determination apparatus 11 Communication part 12 Processing part 13 Storage part 14 Input part 15 Output part 20, 20a, 20b Imaging device 30 Illuminating device 40 On-site processing device 50 Power-related equipment 71-78 Rectangular area 131 Captured image DB
132 Analysis image DB
133 DB with time
401 Image acquisition unit 402 Difference image generation integration unit 403 Specific wavelength matching image conversion unit 404 Region extraction unit 405 Candidate point coordinate input unit 406 Feature point determination unit 407 Conversion parameter calculation unit 408 Image transformation unit 409 Change extraction unit 410 Change determination unit 411 Enlargement / reduction stop determination unit 412 Position change determination unit 413 Occurrence / disappearance determination unit 414 Threshold verification unit 415 Report data creation unit 501, 505 Abnormal part 502 Specific wavelength comparison image 503 Composite image 504 Reference image 1301-1308 Marker 1601 Plan view 1602 Cursor 1604 Icon 1621 line icon

Claims (13)

The equipment abnormality aging change determination apparatus for determining the aging change of the equipment abnormality of the power-related equipment imaged by the imaging device,
An image acquisition unit for acquiring a first image including an abnormal part of the equipment abnormality and a second image serving as a reference for alignment of the first image;
The coordinates of the four points used for the alignment are acquired from the first image, each is determined as the coordinates of the first feature point, and the coordinates of the four points used for the alignment are acquired from the second image. A feature point determination unit that determines each as a coordinate of the second feature point;
A conversion parameter calculation unit that calculates an image shape conversion parameter for converting the coordinates for alignment based on the correspondence between the coordinates of the first feature point and the coordinates of the second feature point;
An image deformation unit that performs image shape conversion on the first image using the image shape conversion parameter;
A storage unit that stores an image of the abnormal portion included in the first image after the image shape conversion in association with a shooting time;
The image of the abnormal part of the first image after the image shape conversion and the image of the abnormal part stored in the storage unit taken at a photographing time different from the photographing time of the first image; And a change determination unit for determining a change over time during
The conversion parameter calculation unit selects the first feature point from an area including pixels with clear contrast in the first image, and the second feature point has clear contrast in the second image. A conversion point obtained by selecting each of the selected first feature points into coordinates on the second image using the calculated image shape conversion parameter selected from a region including a pixel, and the conversion points By obtaining the second feature points in the vicinity of the four, make four sets of close ones, calculate the distance between the second feature point and the conversion point in each set, Repeat the process of calculating the image shape conversion parameter by calculating the sum of the distances, selecting the first feature point and the second feature point again until the sum of the distances is equal to or less than a predetermined threshold. The image shape by Facilities abnormal aging determination device and determines a conversion parameter. The first wavelength is synthesized by synthesizing a specific wavelength comparison image obtained by photographing the abnormal portion indicating an oil leakage and rust region that is an equipment abnormality of the power-related equipment in a predetermined wavelength region and an image obtained by visible light. The equipment abnormal time-dependent change determination apparatus according to claim 1, further comprising a specific wavelength reference image conversion unit that generates an image. (A) The images captured at times when the intensity of the reflected light from the abnormal portion is maximized and minimized by changing the intensity of illumination applied to the abnormal portion are respectively the third image and the fourth image. Acquired as an image,
(B) calculating a difference between hue values of the third image and the fourth image;
(C) Repeat (A) and (B), add the calculated difference for each pixel to calculate an integrated value,
(D) The equipment abnormality according to claim 1, further comprising a region extraction unit that determines, as the abnormal part, a set of pixels of the integrated value within a predetermined threshold range after the process of (C). Time change determination device. The change determination unit
Superimposing the image of the abnormal part of the first image after the image shape conversion and the image of the abnormal part read from the storage unit, and determining a change with time based on an inclusion relationship in the superposition state. The equipment abnormality time-dependent change determination apparatus according to any one of claims 1 to 3, wherein the equipment abnormality change with time is determined. The change determination unit
The number of pixels of the abnormal portion stored in the storage unit, which is captured at a shooting time different from the shooting time of the first image, the number of pixels of the abnormal portion of the first image after the image shape conversion Calculate the value obtained by dividing by
(A) The abnormality is detected when an image of the abnormal part taken in the past closest to the time when the image of the abnormal part of the first image after the image shape conversion is taken is not stored in the storage unit. A process for determining that a new occurrence has occurred,
(B) There is an image of the abnormal part taken in the past closest to the time when the image of the abnormal part of the first image after the image shape conversion is taken, and the numerical value is larger than a first threshold value. A process for determining that the abnormal part has disappeared,
(C) The abnormal portion of the first image after the image shape conversion includes the abnormal portion stored in the storage unit that is captured at a photographing time different from the photographing time of the first image. And a process for determining that the abnormal part has expanded when the numerical value is less than a second threshold value less than 1.
(D) The abnormal portion of the first image after the image shape conversion is included in the abnormal portion stored in the storage unit that has been shot at a shooting time different from the shooting time of the first image. And, when the numerical value is less than the first threshold value and exceeds a third threshold value greater than 1, the process of determining that the abnormal portion has been reduced,
(E) a process of determining that the temporal change of the abnormal portion has stopped when the numerical value exceeds a fourth threshold value less than 1 and less than a fifth threshold value greater than 1;
(F) the abnormal part of the first image after the image shape conversion, the abnormal part stored in the storage unit taken at a photographing time different from the photographing time of the first image, and both The number of pixels of each overlapping portion is counted as the first pixel number, the second pixel number, and the third pixel number, and the pixel number obtained by subtracting the third pixel number from the first pixel number and the first pixel number A position indicating that the position of the abnormal portion has changed when each of the number of pixels obtained by subtracting the third number of pixels from the number of pixels of 2 is greater than a value obtained by adding a predetermined coefficient to the third number of pixels. 5. The equipment abnormality temporal change determination apparatus according to claim 4, wherein any one or a plurality of combinations of processes for determining a change is executed. The first image and the second image are photographed including the first marker installed at the four positions used for the alignment and the second marker installed at a position different from the four positions. If
The feature point determination unit acquires the coordinates of the four points of the first marker from the first image, determines the coordinates of the first feature point, and determines the coordinates of the first marker from the second image. Obtain the coordinates of the four points and determine the coordinates of the second feature point,
The conversion parameter calculation unit is configured to perform the alignment based on both the correspondence between the coordinates of the first feature point and the coordinates of the second feature point and the correspondence relationship of the second marker. The apparatus abnormality change determination apparatus according to claim 1, wherein an image shape conversion parameter for converting coordinates is calculated. A facility abnormality temporal change determination method used in an equipment abnormality temporal change determination device that determines temporal changes in equipment abnormalities of power-related equipment photographed by an imaging device,
The equipment abnormality aging change determination device,
An image acquisition step of acquiring a first image including an abnormal portion of the equipment abnormality and a second image serving as a reference for alignment of the first image;
The coordinates of the four points used for the alignment are acquired from the first image, each is determined as the coordinates of the first feature point, and the coordinates of the four points used for the alignment are acquired from the second image. A feature point determining step for determining each as a coordinate of the second feature point;
A conversion parameter calculation step for calculating an image shape conversion parameter for converting the coordinates for the alignment based on the correspondence between the coordinates of the first feature point and the coordinates of the second feature point;
An image transformation step of performing an image shape transformation on the first image using the image shape transformation parameter;
Storing the image of the abnormal portion included in the first image after the image shape conversion in a storage unit in association with the photographing time;
The image of the abnormal part of the first image after the image shape conversion and the image of the abnormal part stored in the storage unit taken at a photographing time different from the photographing time of the first image; And a change determination step for determining a change over time during
In the conversion parameter calculating step, the first feature point is further selected from a region including a pixel having a clear contrast in the first image, and the second feature point is selected as a contrast in the second image. Is selected from an area including clear pixels, and using the calculated image shape conversion parameter, a conversion point obtained by converting the selected first feature point into coordinates on a second image is calculated, and By obtaining the second feature points in the vicinity of the conversion points, four sets of close ones are created, and the distance between the second feature points and the conversion points is calculated for each set, and about the four sets Calculating the image shape conversion parameter by selecting the first feature point and the second feature point again until the total sum of the distances is equal to or less than a predetermined threshold value. To repeat Facilities abnormal aging determination method characterized by determining the image shape transformation parameter Ri. The equipment abnormality aging change determination device,
The first wavelength is synthesized by synthesizing a specific wavelength comparison image obtained by photographing the abnormal portion indicating an oil leakage and rust region that is an equipment abnormality of the power-related equipment in a predetermined wavelength region and an image obtained by visible light. 8. The equipment abnormality time-dependent change determination method according to claim 7, wherein a specific wavelength collation image conversion step of generating an image is executed. The equipment abnormality aging change determination device,
(A) The images captured at times when the intensity of the reflected light from the abnormal portion is maximized and minimized by changing the intensity of illumination applied to the abnormal portion are respectively the third image and the fourth image. Acquired as an image,
(B) calculating a difference between hue values of the third image and the fourth image;
(C) Repeat (A) and (B), add the calculated difference for each pixel to calculate an integrated value,
(D) The facility according to claim 7, wherein after the process of (C), an area extraction step is performed in which a set of pixels of the integrated value within a predetermined threshold range is determined as the abnormal part. Abnormal aging change determination method. The change determination step includes:
Superimposing the image of the abnormal part of the first image after the image shape conversion and the image of the abnormal part read from the storage unit, and determining a change with time based on an inclusion relationship in the superposition state. The equipment abnormality time-dependent change determination method according to any one of claims 7 to 9, wherein the equipment abnormality changes with time. The change determination step includes:
The number of pixels of the abnormal portion stored in the storage unit, which is captured at a shooting time different from the shooting time of the first image, the number of pixels of the abnormal portion of the first image after the image shape conversion Calculate the value obtained by dividing by
(A) The abnormality is detected when an image of the abnormal part taken in the past closest to the time when the image of the abnormal part of the first image after the image shape conversion is taken is not stored in the storage unit. A process for determining that a new occurrence has occurred,
(B) There is an image of the abnormal part taken in the past closest to the time when the image of the abnormal part of the first image after the image shape conversion is taken, and the numerical value is larger than a first threshold value. A process for determining that the abnormal part has disappeared,
(C) The abnormal portion of the first image after the image shape conversion includes the abnormal portion stored in the storage unit that is captured at a photographing time different from the photographing time of the first image. And a process for determining that the abnormal part has expanded when the numerical value is less than a second threshold value less than 1.
(D) The abnormal portion of the first image after the image shape conversion is included in the abnormal portion stored in the storage unit that has been shot at a shooting time different from the shooting time of the first image. And, when the numerical value is less than the first threshold value and exceeds a third threshold value greater than 1, the process of determining that the abnormal portion has been reduced,
(E) a process of determining that the temporal change of the abnormal portion has stopped when the numerical value exceeds a fourth threshold value less than 1 and less than a fifth threshold value greater than 1;
(F) the abnormal part of the first image after the image shape conversion, the abnormal part stored in the storage unit taken at a photographing time different from the photographing time of the first image, and both The number of pixels of each overlapping portion is counted as the first pixel number, the second pixel number, and the third pixel number, and the pixel number obtained by subtracting the third pixel number from the first pixel number and the first pixel number A position indicating that the position of the abnormal portion has changed when each of the number of pixels obtained by subtracting the third number of pixels from the number of pixels of 2 is greater than a value obtained by adding a predetermined coefficient to the third number of pixels. The facility abnormality aging change determination method according to claim 10, wherein any one or a plurality of combinations of the processing for determining a change is executed. The first image and the second image are photographed including the first marker installed at the four positions used for the alignment and the second marker installed at a position different from the four positions. If
The feature point determination step acquires the coordinates of the four points of the first marker from the first image, determines the coordinates of the first feature point, and determines the coordinates of the first marker from the second image. Obtain the coordinates of the four points and determine the coordinates of the second feature point,
The conversion parameter calculating step is performed for the alignment based on both the correspondence between the coordinates of the first feature point and the coordinates of the second feature point and the correspondence of the second marker. The method for determining abnormality in equipment over time according to claim 7, wherein an image shape conversion parameter for converting coordinates is calculated. A non-transitory computer-readable storage medium storing a program for causing the equipment abnormality temporal change determination apparatus according to any one of claims 7 to 12 to execute the equipment abnormality temporal change determination method as a computer.

Images