JP7629289B2 - Condition determination system and condition determination method. - Google Patents

Description

This invention relates to a system and method for determining the state of an object captured using infrared images taken from the air.

Traditionally, aerial photography using flying objects (unmanned aerial vehicles (drones, etc.) has been widely practiced. Such aerial images are not only used by individuals to appreciate the scenery, but are also used for business purposes such as disaster relief, agriculture, ecological surveys, news reports, infrastructure inspections, and inspections for abnormalities.

For example, as an example of infrastructure inspection, aerial images can be used to inspect solar power generation equipment (such as solar panels) at solar power plants. Solar power generation equipment generates electricity mainly through the photovoltaic effect, so power generation is hindered by dirt or damage on the surface that receives sunlight. Aerial images make it possible to ascertain dirt, damage, etc. on solar power generation equipment. Therefore, photographing and inspecting solar power generation equipment with aerial images is effective in improving power generation efficiency at solar power plants.

There is also an inspection method in which a solar panel is photographed with an infrared camera, the infrared image is viewed on a video monitor, and the image is saved or printed out to identify defective areas where abnormal heat is occurring, and to provide objective information and materials regarding repair estimates (see Patent Document 1).

However, there are photovoltaic power generation facilities that are configured by arranging photovoltaic panels (for example, arranged in a matrix). When inspecting such photovoltaic power generation facilities, depending on the scale, it may be difficult to check each panel one by one. However, even in such cases, a system has been proposed that obtains the temperature distribution on the front side of a group of photovoltaic panels from infrared images and determines abnormalities based on the temperature information (see Non-Patent Document 1).

JP 2015-146371 A

Guide and Technical Information for DC Electrical Safety of Solar Power Generation (2nd Edition) National Research and Development Agency, National Institute of Advanced Industrial Science and Technology, April 2019, pp. 207-209

The following is a specific example of a method for acquiring the temperature distribution on the surface side of a group of photovoltaic panels by taking infrared images from the air, and determining whether there is an abnormality based on the temperature information. First, the photovoltaic panel containing the hotspot is identified. Then, the temperature of the hotspot on that panel is identified. Next, the average temperature around the hotspot is found, and the difference from the temperature of the previous hotspot is found. With this method, an abnormality in the panel is determined by determining whether the difference found in this way exceeds a threshold value.

However, the difference between the average temperature around the hotspot and the temperature at the hotspot can vary greatly depending on various factors, including the season, weather conditions, and the progression of the disorder. For example, it is extremely difficult to classify the three patterns of progression of the disorder when it is winter and cloudy, when it is low under the same season and weather conditions, and when it is summer and sunny, simply by determining the threshold value of the temperature difference.

In other words, to extract all faulty panels using the above technology, it is desirable to include panels with a low probability of failure in the assessment and make the final judgment manually. However, this makes it difficult to reduce the burden of determining whether there is an abnormality using infrared images. In addition, there are various factors that can cause failures in solar panels, and unless the person in question is an experienced worker, there is a risk that the person in question will fail to determine whether there is an abnormality in the solar panel.

This invention was made to solve the above problems, and its purpose is to improve the efficiency of determining abnormalities in an imaged subject using infrared images.

A status determination system according to one embodiment includes an image receiving unit that receives an infrared image from an airborne vehicle, an identification unit that identifies a predetermined set area including an image capture target from the infrared image, a temperature information acquisition unit that calculates a distribution of temperature information based on infrared information for each identified set area, a high temperature information acquisition unit that calculates a first average value of temperature information included in a first set range including the highest temperature in the distribution, a mid-temperature range acquisition unit that calculates a second average value of temperature information included in a second set range including a temperature below the first set range and not overlapping with the first set range in the distribution, a reference value acquisition unit that calculates a reference value that is the difference between the first average value and the second average value, a determination target value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas, and calculates a determination target value for each group based on at least one of a value indicating the variation in the reference value for each set area and the average value, and a determination unit that identifies an image capture target that may be defective based on the determination target value.
Moreover, the type of failure may be determined by varying the setting range for the first average value.
Another embodiment of the status determination system includes an image receiving unit that receives an infrared image from the flying object, an identification unit that identifies a predetermined set area including the image capture target from the infrared image, a temperature information acquisition unit that calculates a distribution of temperature information based on infrared information for each identified set area, a mid-temperature range acquisition unit that calculates a second average value of the temperature information included in a second set range that is less than the maximum temperature within the distribution, a reference value acquisition unit that calculates a reference value that is the difference between the maximum temperature and the second average value, a judgment target value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas, and calculates a judgment target value for each group based on at least one of a value indicating the variation in the reference values of each set area and the average value, and a judgment unit that judges whether the image capture target is abnormal based on the judgment target value.
In addition, the judgment target value acquisition unit may be configured to calculate a standard deviation of the reference value as the variation in the reference value, and further calculate a deviation value for each set area, and the judgment unit may be configured to make an abnormality judgment by comparing the deviation value with a first threshold value.
In addition, when the judgment unit determines that the variation in the reference value or the average value is greater than the first threshold value, it may be configured to identify the suspected fault area causing this, calculate the reference value again so as not to include the suspected fault area, and identify the imaging target that may be faulty based on the reference value.
In addition, the determination unit may be configured to repeat the process of recalculating the reference value and identifying the suspected failure area until the suspected failure area is no longer identified or until the area or ratio of the suspected failure area falls below a predetermined value.
Another embodiment of the state determination system includes an image receiving unit that receives an infrared image from the flying object, a specifying unit that specifies a preset set area including the image capture target from the infrared image, a temperature information acquiring unit that obtains a distribution of temperature information based on infrared information for each specified set area, a high temperature information acquiring unit that obtains a first average value of temperature information included in a first set range including the highest temperature in the distribution, a medium temperature range acquiring unit that obtains a second average value of temperature information included in a second set range that is a temperature below the first set range and does not overlap with the first set range in the distribution, and a reference value that is the difference between the first average value and the second average value. the set area including the set area and other adjacent set areas, and for each group, obtains a judgment target value based on at least one of a value indicating the variation in the reference value of each set area and an average value; a suspect target identification unit that identifies a suspect target that is an imaged object that may have a malfunction based on the judgment target value and a first threshold value; and a judgment unit that classifies the malfunction of the imaged object based on a pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, re-identifies the suspect target based on the judgment target value of the suspect target and the second threshold value.
The fault classification of the judgment unit may include at least two of cluster faults, cracks, stains, and cell faults.
In addition, a second threshold value is used when the judgment unit judges the failure classification of the suspected object to be a cluster failure, and the second threshold value may be set so that fewer suspected objects are included in the set area compared to the first threshold value.
In addition, the judgment unit may perform the judgment by inferring the classification of the suspected fault based on deep learning technology.
The identification unit may also infer and identify the imaging target or a group of a predetermined number of imaging targets from the infrared image based on the characteristics of their arrangement.
The state determination system may further include a setting unit for setting characteristics of the arrangement of the imaging targets, and the identification unit may identify the imaging target or a group of a predetermined number of imaging targets by utilizing the characteristics of the arrangement set by the setting unit.
In addition, in the condition determination system, the identification unit may identify the imaging target or the group of imaging targets by inferring them based on deep learning technology.
The condition determination system may further include a first correction unit that determines a tilt of the infrared image based on the arrangement of the image capture target or the group of image capture targets identified by the identification unit, and performs tilt correction.
In addition, the status determination system may calculate the size of the identified imaging target or group of imaging targets, compare the variation in the calculated value with a determination value indicating an acceptable range of the variation, and determine an identification error if the variation is in a predetermined relationship with the determination value.
In addition, in the status determination system, it may be determined whether any of the imaging targets or imaging target groups are out of alignment based on the positional relationship of the imaging targets or imaging target groups identified by the identification unit, and an imaging target or imaging target group that is out of alignment may be determined to be an identification error.
Furthermore, to determine whether any of the imaging targets or groups of imaging targets are out of alignment, a variance value of the alignment positions may be calculated and compared with a judgment value in the Y-axis direction in the infrared image, and if the calculated variance value has a predetermined relationship to the judgment value, it may be determined that this is a specific error.
In addition, in the state determination system, the identification unit may sequentially identify the imaging objects in the arrangement direction, and determine an identification error depending on the degree of overlap between the imaging object or group of imaging objects to be determined and other identified objects.
Furthermore, the identification unit may sequentially perform identification processing in the X-axis direction, and may determine, as a degree of overlap, that an identification error has occurred when a predetermined position in the X-axis direction of the identified object is between one end and the other end of the X coordinate of an adjacent object in the X-axis direction.
In the condition determination system, the specified position in the X-axis direction of the imaging target that is identified may be the center position in the X-axis direction of the infrared image.
Furthermore, in the status determination system, for each row or column of the imaging targets or imaging target groups identified by the identification unit, the imaging targets or imaging target groups belonging to that row or column may be grouped into one cluster, and if the number of imaging targets or imaging target groups belonging to each cluster is equal to or less than a specified number, the imaging targets or imaging target groups belonging to that cluster may be determined to be an identification error.
In addition, in the condition determination system, the fourth determination unit may collect, as a cluster, imaging targets or groups of imaging targets having similar Y-axis positions or X-axis positions based on the Y-axis position of the imaging target or group of imaging targets identified by the identification unit.
In addition, in the state determination system, for each row or column of the imaging target or the imaging target group identified by the identification unit, the imaging target or the imaging target group belonging to that row or column are grouped into one cluster;
The fifth judgment unit may calculate the distance between each cluster and the nearest other cluster, compare this calculated value with a judgment value, and if the calculated value has a predetermined relationship with the judgment value, judge it to be a specific error.
The state determination system may further include a setting unit that allows a user to set at least one of the designated number in a cluster and the number of cluster columns.
Also,
The state determination system may have a sixth determination unit that, for each row or column of the imaging targets or group of imaging targets identified by the identification unit, groups the imaging targets or group of imaging targets belonging to that row or column into a single cluster so that they form a specified rectangle, and corrects the positions of the imaging targets or group of imaging targets belonging to each cluster that fall outside the rectangle so that they are included within the rectangular range.

According to the embodiment, it is possible to improve the efficiency of determining abnormalities in an imaged subject using infrared images.

1 is a schematic block diagram showing a state determination system according to a first embodiment; Schematic diagram of an infrared image. Schematic diagram of an infrared image. Schematic of infrared image 3 is a schematic flowchart of the first embodiment. FIG. 4 is a schematic block diagram showing a state determination system according to a modified example of the first embodiment. 11 is a schematic flowchart of a modified example of the first embodiment. FIG. 11 is a schematic block diagram showing a state determination system according to a second embodiment. 13A and 13B are schematic diagrams of infrared images illustrating the concept of processing according to a second embodiment. 10 is a schematic flowchart of a second embodiment. FIG. 11 is a schematic block diagram showing a state determination system according to a third embodiment. 13A and 13B are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. 13A and 13B are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. 13A and 13B are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. 13 is a schematic flowchart of a third embodiment. FIG. 13 is a schematic block diagram showing a determination object detection system according to a fourth embodiment. 13 is a schematic flowchart of a fourth embodiment. FIG. 13 is a schematic block diagram showing a state determination system according to a modified example of the fourth embodiment. 13 is a schematic flowchart of a modification of the fourth embodiment. FIG. 13 is a schematic block diagram showing a determination object detection system according to a fifth embodiment. FIG. 13 is a conceptual diagram showing an outline of a correction process according to a fifth embodiment. 13 is a schematic flowchart of a fifth embodiment. FIG. 13 is a schematic block diagram showing a determination object detection system according to a sixth embodiment. FIG. 23 is a conceptual diagram showing an outline of a determination process according to the sixth embodiment. 13 is a schematic flowchart of a sixth embodiment. FIG. 23 is a schematic block diagram showing a determination object detection system according to a seventh embodiment. FIG. 23 is a conceptual diagram showing an outline of a determination process according to the seventh embodiment. 13 is a schematic flowchart of a seventh embodiment. FIG. 23 is a schematic block diagram showing a determination object detection system according to an eighth embodiment. FIG. 23 is a conceptual diagram showing an overview of a determination process according to the eighth embodiment. 13 is a schematic flowchart of an eighth embodiment. FIG. 13 is a schematic block diagram showing a determination object detection system according to a ninth embodiment. FIG. 23 is a conceptual diagram showing an overview of a determination process according to the ninth embodiment. FIG. 23 is a conceptual diagram showing an overview of a determination process according to a modified example of the ninth embodiment. 13 is a schematic flowchart of a ninth embodiment. FIG. 23 is a schematic block diagram showing a determination object detection system according to a tenth embodiment. FIG. 23 is a conceptual diagram showing an overview of the process according to the tenth embodiment. FIG. 23 is a conceptual diagram showing an overview of the process according to the tenth embodiment. 23 is a schematic flowchart of a tenth embodiment. FIG. 13 is a conceptual diagram showing an outline of an image indicating the location of a suspected failure point. FIG. 13 is a conceptual diagram showing an outline of an image indicating the location of a suspected failure point.

The state determination systems according to the first to third embodiments and the determination target detection systems according to the fourth to tenth embodiments will be described with reference to Figures 1 to 40.

[First embodiment]
The overall configuration of a state determination system 100 according to a first embodiment will be described with reference to Figures 1 to 4. Note that the state determination system in Figure 1 is just an example, and does not exclude the inclusion of other configurations, and can be embodied in various forms.

(Overview of Aircraft D)
As shown in FIG. 1, the flying object D includes an image capturing unit D1 and a position acquiring unit D2. The image capturing unit D1 includes a visible light camera and an infrared camera, and can capture visible light images (FIGS. 39 and 40) and infrared images (FIGS. 2A and 2B) simultaneously or synchronously. The position receiving unit D2 receives navigation signals from positioning satellites such as GNSS (Global Navigation Satellite System), such as GPS, GLONASS, Galileo, and Quasi-Zenith Satellite System (QZSS), and measures the three-dimensional position of the flying object D and adds the position to the visible light image and the infrared image (for example, EXIF information). The position receiving unit D2 may also be equipped with, for example, a water pressure gauge and a barometer, and convert the results into depth and altitude. As another example, an ultrasonic sensor or the like may be equipped to measure the distance from the surface of the earth, the bottom of the sea, etc. to the flying object D. The received one-dimensional position information is added to the visible light image and the infrared image. In this way, the three-dimensional positions of the visible light image and the infrared image can be identified. These are transmitted as additional information to the condition determination system 100 together with the visible light image and the infrared image. Note that the imaged objects in Figures 39 and 40 and Figures 2A and 2B are merely examples, and the objects depicted in each image may be other types of imaged objects.

(Overview of state determination system 100)
The condition determination system 100 includes a position information receiving unit 110, an image receiving unit 120, an identification unit 130, a temperature information acquiring unit 140, a high temperature information acquiring unit 150, a medium temperature range acquiring unit 160, a reference value acquiring unit 170, a determination target value acquiring unit 180, a determination unit 190, a control unit C, and a storage unit S. The position information receiving unit 110 receives positioning information (such as a three-dimensional position) of the flying object D based on a navigation signal from the flying object D. For example, these may be received as information added to an image as EXIF information.

The image receiving unit 120 receives an infrared image from the photographing unit D1 of the flying object D. The identification unit 130 identifies a predetermined imaging target or group of imaging targets from the visible light image. This is hereinafter referred to as the "set area". The imaging target is, for example, a solar power generation panel and a group of multiple solar power generation panels. Other imaging targets can include a large number of automobiles, building structures (glass, etc.), a large number of various sensors, mass-produced products that are inspected, etc. In the following, a solar power generation panel will be used as an example of the set area. In addition, for the sake of convenience, the "solar power generation panel or a group of a predetermined number of solar power generation panels" as the "set area" will be simply referred to as the "solar power generation panel". However, in this embodiment and other embodiments and their modified examples described below, the term "set area" is merely an example and is not limited to the solar power generation panel.

(Identification unit 130)
The identification unit 130 will be further described. The identification unit 130 identifies a set area in a visible light image received from the imaging unit D1 of the flying object D. As an example, the identification is performed based on the shape of the outer edge of the imaging target. When the imaging target is a solar power generation panel or a group of solar power generation panels, the solar power generation panel is identified based on the shape of the panel. Identification of a solar power generation panel here means, as an example, identifying the coordinates of the outer edge of a solar power generation panel or the coordinates belonging to a solar power generation panel in an infrared image.
For example, the photovoltaic panel group may be a photovoltaic panel group with two vertical columns and two horizontal rows, or a photovoltaic panel group with two vertical columns and one horizontal row. This detection unit may be determined for each state determination system and not be changed, or may be set by a user's operation via an input unit (not shown). Below, a configuration for identifying "each photovoltaic panel" will be described, but this embodiment includes a modified example in which the identifying unit 130 detects "two or more photovoltaic panels as one unit", so the processing of the temperature information acquisition unit 140 below and each processing of the second embodiment and subsequent embodiments (including modified examples) also take into account the modified example in which such a photovoltaic panel group is detected.

Next, the identification unit 130 uses the coordinates of the identified area of the imaging target (set area) in the visible light image as they are in the coordinates of the infrared image. This is based on the premise that the angle of view, angle, scale, etc. are the same or substantially the same (within the allowable range of the difference in panel positions between the two images) in the visible light image and the infrared image.

Alternatively, if there is a difference in the angle of view, angle, scale, etc. between the visible light image and the infrared image, and it is not acceptable to simply replace the coordinates of one image with the coordinates of the other (the difference is large), the identification unit 130 may be configured to use some kind of landmark, marker, or morphological characteristics and positional characteristics of the imaged subject to perform scale and position alignment.

The above configuration in which the detection of the imaging target is performed in a visible light image and the coordinates of the identified area are applied to an infrared image can also be applied to the following modified examples and other embodiments.

(Temperature information acquisition unit 140)
The temperature information acquisition unit 140 acquires temperature information within a set area (e.g., each panel, a group of panels, etc.) by any known technique for each photovoltaic power generation panel in the infrared image identified by the identification unit 130. For example, the temperature information acquisition unit 140 obtains temperature information from the color temperature of each smallest unit constituting the image in the set area of the infrared image. The smallest unit may be a pixel or a predefined block of pixels. Coordinate information in the image may be associated with each of the temperature information of the smallest units. The temperature information acquisition unit 140 also obtains the temperature distribution for each set area from the temperature information.

(High temperature information acquisition unit 150)
The high temperature information acquisition unit 150 identifies temperature information included in a first set range based on temperature information for each minimum unit of the set area, for example, a temperature distribution. The first set range is a set range below the maximum temperature in the temperature distribution or a set range based on the maximum temperature. The high temperature information acquisition unit 150 also obtains an average value of each temperature based on the temperature information included in the first set range. This average value is set as the first average value. As a specific example, the high temperature acquisition unit 150 obtains temperature information of each pixel in the set area obtained by the temperature information acquisition unit 140. In this example, the high temperature acquisition unit 150 obtains an average value of each piece of temperature information of the top 20% including the maximum temperature in the set area. This average value does not have to be the average value of all temperatures in the set area. For example, it may be a sample average value such as an average of the maximum and minimum temperatures of the top 20% or a median.
In the case of a photovoltaic panel as an example of the imaging target, if there is a cluster fault, a high temperature area will appear in a cluster that spans multiple cells of the photovoltaic panel in a row or column. Therefore, in order to respond to the case where there is a cluster fault in each photovoltaic panel, high temperature information in the top 20% of the temperature distribution is used as the high temperature area in determining the suspected object. This high temperature information is used in comparison with the medium temperature area described below.
In contrast, in cases of faults where the area of the high temperature part tends to be small, such as dirt or cell faults, the high temperature information in the top 3% of the temperature distribution is used as the high temperature area in determining suspected objects. This high temperature information is used in comparison with the medium temperature area described below.

(Medium temperature range acquisition unit 160)
The mid-temperature range acquisition unit 160 identifies temperature information of a second set range that is less than the first set range based on temperature information for each minimum unit of the set range, for example, temperature distribution. The lower limit of the second set range is a temperature that does not reach the minimum temperature. In the case of an imaging target in which errors or defects appear in the high-temperature parts of an infrared image, such as a solar power generation panel, excluding low-temperature parts that are likely to become noise helps prevent false detections and improve the efficiency of abnormality determination of the imaging target. In other words, by setting a temperature higher than the minimum temperature and lower than the mid-temperature as the lower limit value of the second set range, a predetermined range that becomes a low-temperature part is excluded. In other words, a temperature zone between high and low temperatures is identified.
In addition, the mid-temperature range acquisition unit 160 obtains the average value of each temperature based on the temperature information included in the second set range. This average value is set as the second average value. As a specific example, the mid-temperature range acquisition unit 160 obtains the temperature information of each pixel in the set range obtained by the temperature information acquisition unit 140. In this example, the mid-temperature range acquisition unit 160 obtains the average value of each temperature information in the mid-temperature range, with any temperature exceeding the bottom 20% as the lower limit and any temperature below the top 20% as the upper limit. Alternatively, the mid-temperature range acquisition unit 160 obtains the average value of each temperature information in the top 40 to 80%. This average value does not have to be the average value of all temperatures in the set range. For example, it may be a sample average value such as the average of the maximum and minimum temperatures in the second set range, or a median value. Also, it may be a predetermined temperature (for example, a median value) instead of a range such as the second set range.

(Reference value acquisition unit 170)
The reference value acquiring unit 170 acquires a reference value _Xi which is the difference between the first average value acquired by the high temperature acquiring unit 150 and the second average value acquired by the medium temperature range acquiring unit 160 .

(Determination target value acquisition unit 180)
The determination target value acquisition unit 180 groups a plurality of set regions, including the set region and other adjacent set regions, into one group. This grouping is performed in a preset arrangement. When the imaging targets are arranged in a matrix, grouping is performed by the number of rows and columns.

The judgment target value acquisition unit 180 obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference value of each set region and an average value. For example, the judgment target value acquisition unit 180 obtains the difference between the reference value X _i (e.g., reference value = first average value - second average value) obtained by the reference value acquisition unit 170 and the average value μ _x of X _i . The judgment target value acquisition unit 180 also obtains a value obtained by dividing the difference "X _i - μ _x " by the standard deviation σ _x as a deviation value SS _i indicating the deviation amount of the temperature change difference of the set region.
The deviation value SS _i for the reference temperature (reference value) _Xi of each panel is defined by the following equation 1.

When the total number of panels is N, the average value μ _x of the reference temperature _Xi is defined by the following equation 2.

When the total number of panels is N, the standard deviation σ _x of the reference temperature _Xi is defined by the following equation 3.

In addition, when the reference temperatures _Xi of each set area are all equal,
Since the standard deviation σ _x =0 and SS _i cannot be acquired, SS _i =1 is set.

(Determination unit 190)
The determination unit 190 identifies an imaging target that may be faulty by comparing a determination target value such as the deviation value SS _i in the above example with a threshold value. Specifically, the determination target value is compared with a threshold value, a range of an appropriate range, or the like, and it is determined whether or not the imaging target includes an abnormality. For example, when the deviation value SS _i for the reference temperature _Xi in each photovoltaic power generation panel exceeds an allowable range or value (for example, when it exceeds a preset threshold value), it is identified as a suspected fault location. In addition, the determination unit 190 may store in advance the shape of a group (pattern) of pixels exhibiting an abnormal temperature, and identify a specific abnormality by comparing with the pattern.

Furthermore, if an abnormality is found in each photovoltaic power generation panel as a post-processing of the determination process, the determination unit 190 performs a process of at least identifying the position of that panel and presenting it to the user in a manner of their choice (see "DIFF" in the panel in the fourth row and sixth column of the upper block in Fig. 3, or Figs. 39 and 40, etc.). In addition to the position of the imaged subject, the determination unit 190 may also perform a process of creating and displaying the maximum temperature, average temperature, or number of abnormal pixels for each imaged subject as image data.

(Control Unit C Configuration)
For convenience of explanation, the processing of each part in the above description has been described as being executed under the control of the control unit C shown in FIG. 1. The control unit C may be configured to include a single or multiple circuits, such as a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC). The control unit C realizes its function by reading and executing, for example, an image position identification program stored in a memory. Instead of storing a program in a memory, the program may be directly incorporated in the circuit. In this case, the function is realized by reading and executing a program incorporated in the circuit as the control unit C. The control unit C is not limited to being configured as a single circuit, and may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the functions may be realized by integrating multiple components in the embodiment (e.g., at least two of the location information receiving unit 110, image receiving unit 120, identification unit 130, temperature information acquisition unit 140, high temperature information acquisition unit 150, medium temperature range acquisition unit 160, reference value acquisition unit 170, judgment target value acquisition unit 180, and judgment unit 190) into a single processor.

[Action]
4 is a diagram showing the flow of processing in this embodiment. Hereinafter, the flow of processing in the state determination system 100 will be described along with step numbers (S001 to S009) with reference to FIG.

(S001)
The position information receiving unit 110 in the status determination system 100 receives position information from the flying object D. Furthermore, the image receiving unit 120 receives visible light images and infrared images.

(S002)
The identification unit 130 identifies a set area (such as a solar power generation panel) in the visible light image. Next, the identification unit 130 uses the coordinates of the identified area of the imaging target (set area) in the visible light image as they are in the coordinates of the infrared image. Note that the identification unit 130 may identify the set area from the infrared image (see the fourth embodiment, etc.). In this case, acquisition of the visible light image and related processing may be omitted.

(S003)
The temperature information acquisition unit 140 acquires temperature information within each photovoltaic panel in the infrared image identified by the identification unit 130. This may be performed for each pixel, or a plurality of pixels may be treated as one unit and an average value for each pixel group may be used as the temperature information. The temperature information acquisition unit 140 also obtains the temperature distribution for each set region from the temperature information.

(S004)
The high-temperature information acquiring unit 150 identifies temperature information included in a first set range. The first set range is a set range below the maximum temperature in the temperature distribution or a set range based on the maximum temperature, for example, the top 20% including the maximum temperature in the set area.

(S005)
Moreover, the high temperature information acquiring unit 150 calculates a first average value which is an average value of each temperature based on the temperature information included in the first set range.

(S006)
The intermediate temperature range acquisition unit 160 identifies temperature information in the second set range. The second set range is, so to speak, a temperature zone between high and low temperatures.

(S007)
Furthermore, the mid-temperature range acquisition unit 160 obtains a second average value, which is the average value of each temperature based on the temperature information included in the second set range.

(S008)
The reference value acquiring unit 170 acquires a reference value that is the difference between the first average value acquired by the high temperature acquiring unit 150 and the second average value acquired by the medium temperature range acquiring unit 160 .

(S009)
The judgment target value acquisition unit 180 groups a plurality of set regions into one group. The judgment target value acquisition unit 180 also obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference values of the set regions and an average value. The judgment target value is, for example, a deviation value SS _i .

(S010)
The determination unit 190 compares the determination target value with the first threshold value to identify an imaging target that may be at risk of being impaired.
Furthermore, if an abnormality is found in any of the photovoltaic power generation panels as a post-processing step of the determination process, the determination unit 190 performs a process of at least identifying the position of that panel and notifying the user in a manner of his/her choice (see Figures 39 and 40).

[effect]
According to this embodiment, the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of the infrared image. In addition, a judgment target value is obtained based on at least one of the value indicating the variation of the reference value of each set area and the average value for each set area for each group of multiple set areas. Furthermore, an abnormality of the imaged object is judged based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation of the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

The above embodiment reduces misjudgments, making it easier for panels with a low probability of failure to be excluded from suspected failures. This makes it possible to improve the efficiency of determining abnormalities in the imaged subject using infrared images.

[Modification A]
A state determination system 100 according to a modified example of the first embodiment will be described with reference to Fig. 5 and Fig. 6. In the following description, descriptions that overlap with the first embodiment will be omitted.

(overview)
As in the first embodiment, the present modification also sets the imaging target in the infrared image as the set region. In other words, it is configured to exclude anything other than the imaging target from the judgment of suspected faults. That is, the photovoltaic power generation panels are first identified, and the abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, the abnormality judgment error due to the arrangement of the photovoltaic power generation panels is also reduced.

In the state determination system 100 of this modification, the high temperature information acquisition unit 150A identifies the maximum temperature based on temperature information for each minimum unit of the set region, for example, a temperature distribution. The medium temperature range acquisition unit 160 identifies temperature information for a second set range that is less than the maximum temperature and includes the medium temperature, based on temperature information for each minimum unit of the set region, for example, a temperature distribution.
The mid-temperature range acquisition unit 160 also obtains an average value of each temperature based on the temperature information included in the second set range, and sets this average value as a second average value.

The reference value acquisition unit 170 obtains a reference value that is the difference between the maximum temperature (high temperature information) identified by the high temperature acquisition unit 150 and the second average value obtained by the medium temperature range acquisition unit 160.

The determination object value acquisition unit 180 obtains a determination object value for each group based on at least one of a value indicating the variation of the reference value of each set area and an average value. The determination unit 190 identifies imaging objects that may be impaired by comparing the determination object value, such as the deviation value SS _i in the above example, with a threshold value.

[Action]
6 is a diagram showing the flow of processing in this modified example. Hereinafter, the flow of processing in the state determination system 100 will be described along with step numbers (S011 to S019) with reference to FIG.

(S011-S013)
These are similar to S001 to S003 in the state determination system 100 of the first embodiment.

(S014)
The high-temperature information acquiring unit 150A identifies the maximum temperature based on, for example, a temperature distribution.

(S015)
The mid-temperature range acquisition unit 160 identifies temperature information in the second set range.

(S016)
Furthermore, the mid-temperature range acquisition unit 160 obtains a second average value, which is the average value of each temperature based on the temperature information included in the second set range.

(S017)
The reference value acquiring unit 170 obtains a reference value which is the difference between the maximum temperature and the second average value.

(S018)
The judgment target value acquisition section 180 obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference values of the set regions and an average value. The judgment target value is, for example, a deviation value SS _i .

(S019)
The determination unit 190 identifies an imaging target that may be a failure by comparing the determination target value with the first threshold value. Furthermore, if an abnormality is found in each photovoltaic power generation panel as a post-processing of the determination process, the determination unit 190 performs a process of at least identifying the position of the panel and presenting it to the user in a method of his/her choice (see Figs. 39 and 40).

[effect]
According to this modification, the difference between the maximum temperature and the average value of the medium temperature range is obtained from the temperature distribution in the set area of the infrared image. In addition, a judgment target value is obtained for each group of multiple set areas based on at least one of the value indicating the variation in the reference value of each set area and the average value. Furthermore, a judgment is made about an abnormality of the imaged object based on the judgment target value. With this configuration, mistakes in judging an abnormality of the imaged object in each set area are reduced. In other words, since the variation in the difference between the maximum temperature and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

The above embodiment reduces misjudgments, making it easier for panels with a low probability of failure to be excluded from suspected failures. This makes it possible to improve the efficiency of determining abnormalities in the imaged subject using infrared images.

[Modification B]
A description will be given of a state determination system 100 according to the first embodiment and Modification B of Modification A thereof. In the following description, descriptions that overlap with the first embodiment and the like will be omitted.

(overview)
As in the first embodiment, the present modification also sets the imaging target in the infrared image as the set region. In other words, it is configured to exclude anything other than the imaging target from the judgment of suspected faults. That is, the photovoltaic power generation panels are first identified, and the abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, the abnormality judgment error due to the arrangement of the photovoltaic power generation panels is also reduced.

In addition, in the state determination system 100 in this modified example, the high temperature information acquisition unit 150A identifies temperature information or the maximum temperature included in the first set range based on temperature information for each smallest unit of the set area, for example, temperature distribution. In this modified example B, the high temperature information acquisition unit A acquires multiple types of high temperature information. An example of the multiple types is multiple ranges in which at least one of the upper and lower limits of the first set range is different. Also, it may be a predetermined temperature such as the maximum temperature, without being limited to a range. For example, the high temperature information acquisition unit 150A identifies three types: the top 20%, the top 3%, and the maximum temperature.

The mid-temperature range acquisition unit 160 identifies temperature information of a second set range that is less than the maximum temperature and includes the intermediate temperature, based on temperature information for each minimum unit of the set range, for example, a temperature distribution. This mid-temperature range may be configured to determine multiple set ranges in accordance with the high temperature range. The mid-temperature range acquisition unit 160 also determines the average value of each temperature based on the temperature information included in the second set range. This average value is set as the second average value. This average value does not have to be the average value of all temperatures in the set range. For example, it may be a sample average value such as the average of the maximum and minimum temperatures in the second set range, or a median. It may also be a predetermined temperature (for example, the median) rather than a range like the second set range.

The reference value acquisition unit 170 obtains a reference value that is the difference between the multiple types of high temperature information (such as the maximum temperature and the first average value) identified by the high temperature acquisition unit 150 and the second average value obtained by the mid-temperature range acquisition unit 160.

The determination object value acquisition unit 180 obtains a determination object value for each group based on at least one of a value indicating the variation of the reference value of each set region and an average value. The determination unit 190 identifies imaging objects that may be damaged by comparing the determination object value such as the deviation value SS _i in the above example with a threshold value. The threshold value may be set uniquely for at least one type of high temperature information depending on the type of high temperature information.

The determination unit 190 identifies imaging targets that may be impaired by comparing the determination target value with a threshold value. Specifically, the determination unit 190 identifies the presence or absence of an impairment and the type of impairment based on the results of the comparison. For example, if the determination target value for any of the above types of high temperature information is equal to or less than the threshold value, the determination unit 190 determines that there is no impairment. In contrast, if the threshold value is exceeded for any of the above types of high temperature information, the determination unit 190 identifies the type of impairment that corresponds to the type of high temperature information that was the basis for calculating the exceeded determination target value.

It is also possible to deal with cases where the determination unit 190 can identify two or more types of faults. As an example, it is assumed that a predicted pattern of the comparison result between the determination target value and a threshold value is stored in advance. In this case, the determination unit 190 determines which of the specified predicted patterns the comparison result with the threshold value is closest to, and identifies the presence or absence of a fault and the type of fault.

The above embodiment reduces misjudgment, making it easier for panels with a low probability of failure to be excluded from suspected failures. This makes it possible to improve the efficiency of determining abnormalities in the imaged subject using infrared images.

[Second embodiment]
A state determination system 200 according to the second embodiment will be described with reference to Figures 7 to 9. In the following description, descriptions that overlap with those of the first embodiment will be omitted.

As in the first embodiment, in the second embodiment, the imaged object in the infrared image is identified, and the reference value and the judgment object value are obtained. Also, as in the first embodiment and its modified example, the suspected fault object (panel, etc.) is identified.

In the second embodiment, if a location that may be a fault (hereinafter, in principle, referred to as a "suspected fault location") is detected, the status determination for that group is performed again without including that location. The more likely fault locations are detected within the same group during the determination, the more likely the calculated deviation amount will be slowed down. Therefore, in the second embodiment, the suspected fault locations are identified in detail by excluding the likely fault locations and performing a status determination again. Furthermore, in the second embodiment, the process is repeated until no suspected fault locations are identified for each group, or until the area or ratio of the suspected fault locations falls below a set value.

(Condition Determination System 200)
As shown in Figure 7, the condition determination system 200 of the second embodiment is composed of a location information receiving unit 210, an image receiving unit 220, an identification unit 230, a temperature information acquiring unit 240, a high temperature information acquiring unit 250, a medium temperature range acquiring unit 260, a reference value acquiring unit 270, a determination target value acquiring unit 280, a determination unit 290, as well as a control unit C and a memory unit S.

The detection of the photovoltaic power generation panel by the identification unit 230, the acquisition of high temperature information by the high temperature information acquisition unit 250, the calculation of the second average value by the medium temperature range acquisition unit 260, the calculation of the reference value by the reference value acquisition unit 270, the acquisition of the judgment target value by the judgment target value acquisition unit 280, and the identification of the imaging target that may be damaged by the judgment unit 290 are the same as in the first embodiment and its modified examples.

In the second embodiment, after the judgment by the judgment unit 290, if there is a suspected faulty area, the suspected faulty area is excluded. For example, the judgment unit 290 identifies the coordinates in the image of the suspected faulty area and excludes those coordinates. While keeping the initial grouping of the set areas, the temperature information acquisition unit 240 obtains the temperature distribution for each set area without including the temperature information at the excluded coordinates. The high temperature information acquisition unit 250 obtains high temperature information based on the temperature distribution after this exclusion.

The mid-temperature range acquisition unit 260 identifies temperature information for the second set range based on the temperature distribution after this exclusion. Alternatively, in the second embodiment, it is also possible to use the temperature information for the second set range in the first judgment. In this case, the identification process for the second set range from the second time onwards may be omitted. Even if the identification process is omitted in this way, the configuration may be such that the identification process for the second set range is performed again when the third judgment process is performed after the second judgment process.

The calculation of the reference value and the judgment target value thereafter is the same as in the first embodiment and its modified example. As in the first embodiment, the judgment unit 290 identifies a suspected fault location when, for example, the deviation value SS _i for each reference temperature _Xi of the photovoltaic power generation panel exceeds an allowable range or value (for example, when it exceeds a preset threshold value) (see FIG. 8). If a suspected location still exists after the identification process, the judgment unit 290 repeats the above process again. However, the second embodiment is not limited to this. For example, the judgment unit 290 may be configured to repeat the judgment process from the exclusion of the suspected location until the area or ratio of the suspected fault area falls below a predetermined value. Alternatively, instead of this configuration, the judgment unit 290 may be configured to repeat the judgment process from the exclusion of the suspected location while updating the threshold value.

[Action]
Fig. 9 is a diagram showing the flow of processing in the second embodiment. Below, with reference to Fig. 9, the flow of processing in the state determination system 200 will be explained along with step numbers (S021 to S030-1).

(S021-S028)
These are similar to S001 to S008 or S011 to S019 in the state determination system 100 of the first embodiment.

(S029)
The determination unit 290 compares the determination target value with the first threshold value to identify an imaging target that may be at risk of being impaired.

(S030)
The determining unit 290 determines whether or not there is a suspected fault location for each set area by comparing the determination target value with the first threshold value.

(S030-1)
As a result of the determination in S030, in the set area where it is determined that there is a suspected fault location (S030; Yes), the coordinates in the image of the suspected fault location are identified, and the coordinates are excluded from the set area.

After S030-1, the process returns to S023, and the temperature information acquisition unit 240 obtains the temperature distribution for each set area without including the temperature information at the excluded coordinates. These processes are repeated until the judgment in S030 does not determine that there is a suspected fault location (S030; No). Furthermore, as a post-processing step of the judgment process, if an abnormality is found in each solar power generation panel, the judgment unit 290 performs a process to at least identify the position of that panel and present it to the user in a manner of their choice (see Figures 39 and 40).

[effect]
According to the second embodiment, the difference between the maximum temperature or the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of the infrared image. In addition, a judgment target value is obtained based on at least one of the value indicating the variation of the reference value of each set area and the average value for each set area for each group of multiple set areas. Furthermore, an abnormality of the imaged object is judged based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation of the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

Furthermore, in the second embodiment, the suspected faulty part is excluded and the status judgment is performed again, thereby identifying the suspected faulty part in more detail. This makes it possible to reduce the omission of identifying the suspected faulty part even when many parts that may be faulty are detected within the same group during the judgment.

The above embodiment reduces misjudgment, making it easier for panels with a low probability of failure to be excluded from suspected failures. This makes it possible to improve the efficiency of determining abnormalities in the imaged subject using infrared images.

The determination object detection system understood from the second embodiment can have the following features.
[Feature A]
an image receiving unit for receiving an infrared image from the flying object;
An identification unit that identifies a preset region including an imaging target from an infrared image;
Identifying a suspected fault area for each of the identified setting areas,
a determination unit that again identifies other suspected fault areas so as not to include the suspected fault area.
The second embodiment can be configured independently instead of being combined with the first embodiment.

[Third embodiment]
A state determination system 300 according to the third embodiment will be described with reference to Fig. 10 to Fig. 14. In the following description, descriptions that overlap with the first and second embodiments will be omitted.

As in the first embodiment, in the third embodiment, the object to be imaged in the infrared image is identified, and the reference value and the judgment object value are obtained. Also, as in the first embodiment, the object suspected of being faulty (panel, etc.) is identified.

The third embodiment further classifies the type of failure for the set area that includes the suspected failure location. Depending on the classification of the failure, there may be cases where the amount of heat generated is low due to the influence of the external environment, such as weather conditions, or the progression of the failure. In other words, it is expected that the amount of deviation will be small when identifying the suspected failure location using the first embodiment or the like. Therefore, in order to deal with such situations, the third embodiment classifies the type of failure of the subject being photographed, such as a solar power generation panel, and is configured to identify more suspected failure locations in certain classifications than in other classifications, thereby reducing the failure to identify suspected failure locations in certain patterns, such as low heat generation, and also reducing excessive identification of suspected failure locations.

(State Determination System 300)
As shown in FIG. 10 , the status determination system 300 of the third embodiment is configured to include a location information receiving unit 310, an image receiving unit 320, an identification unit 330, a temperature information acquiring unit 340, a high temperature information acquiring unit 350, a medium temperature range acquiring unit 360, a reference value acquiring unit 370, a determination target value acquiring unit 380, a determination unit 390, a suspected fault identifying unit 391, as well as a control unit C and a memory unit S.

The detection of the solar power generation panel by the identification unit 330, the acquisition of high temperature information by the high temperature information acquisition unit 350, the calculation of the second average value by the medium temperature range acquisition unit 360, the calculation of the reference value by the reference value acquisition unit 370, and the acquisition of the judgment target value by the judgment target value acquisition unit 380 are the same as in the first embodiment, etc. In the third embodiment, the suspect target identification unit 391 identifies an imaged target that may be at risk of failure. The identification method is the same as in the first embodiment, etc.

(Determination unit 390)
In the third embodiment, after the suspected object identification unit 391 identifies a suspected fault location, if a suspected fault location is found, the determination unit 390 classifies the type of fault in the set area. As an example, the determination unit 390 classifies the type of fault in the set area in the image by deep learning.

This uses an inference model created by deep learning using so-called artificial intelligence (AI). The inference model contains a group of teacher data consisting of models of suspected areas that appear for each type of failure of the object being imaged in the infrared image (e.g., a solar power generation panel, etc.). The model of the suspected area includes information such as shape, size, and temperature. In other words, the shape can be said to be a feature of the failure pattern. In other words, the size can be said to be the input size.
The shapes that appear in an infrared image (e.g., a shape formed by a collection of relatively high temperature points (e.g., the "above threshold" part in FIG. 8)) are based on temperature information. Therefore, the temperature information as a model of a suspected area can be used as a supplement to the above shapes.

The artificial intelligence performs deep learning using the group of teacher data, and an inference model is generated. In the determination unit 390, inference is performed using the inference model for the set area including the suspected fault location, and the type of fault is classified. Note that in this embodiment, the classification method is not limited to deep learning. Other methods such as pattern matching can also be used.

Classification examples of the determination unit 390 will be described with reference to Figs. 11 to 13. Figs. 11 to 13 show infrared images in which the imaged subject is a solar power generation panel. Fig. 11 is an example of a cluster failure. Fig. 12 is an example of a crack. Fig. 13 is an example of dirt or cell failure.

<Classification of Disabilities>
In a solar panel, for example, if clusters each consisting of two rows of cells are arranged vertically in three columns, a cluster failure will occur in a horizontally elongated shape that roughly follows the shape of the cell cluster. In an infrared image, as shown in Fig. 11, an abnormal heat generation spot can be seen along the horizontally elongated cluster. Therefore, the determination unit 390 classifies the heat generation spot as a cluster failure based on the shape of the cluster on the infrared image that is similar to the cluster shape.

The determination unit 390 may classify a heat spot as a "fault other than a cluster fault" by determining that the shape of the heat spot on the infrared image is not similar to the cluster shape. Alternatively, if a heat spot is observed over a relatively wide area, such as a crack, but has a different form from the cluster model (see FIG. 12), the determination unit 390 may be configured to classify the heat spot as a "crack" fault.

The determination unit 390 may be configured to classify the fault as a type of dirt or cell fault as another classification. Dirt and cell faults are seen as heat generating areas that are relatively narrow compared to clusters and cracks. Therefore, if the fault is in a form different from the cluster or crack model (see FIG. 13), the determination unit 390 may be configured to classify the heat generating area as a "dirt or cell fault" fault.

<Re-identification process overview>
There are several classifications of faults, and the above is just one example, but by classifying the specific fault type by the determination unit 390 as described above, it becomes possible to execute the following re-identification process of suspect targets. As will be described later, this re-identification process can further reduce the failure to identify suspect targets. As a result, it becomes possible to more reliably determine the presence or absence of a fault in the set area according to the fault type.

In the above three-tiered vertical cluster arrangement, if a cluster fault occurs in one tier, abnormal heat generation tends to occur over the entire area according to the cluster arrangement shape, as shown in FIG. 11. The judgment target value used to identify a suspected fault is a value that indicates the variation of the reference value of each set area calculated for each group, so if a relatively wide-ranging fault occurs, such as a cluster fault, the judgment target value may not show a significant value or the variation may be small, which may cause the suspected fault location to be missed. In addition, in a cluster fault, it is expected that the deviation in heat generation may be small depending on the progression of the fault (reduced power generation, poor current, etc.). Weather conditions such as season, weather, and geographical conditions may also have a significant effect on the deviation in heat generation.
That is, in order to identify a cluster fault as a suspected fault location, it is necessary to identify it as a fault even if the temperature deviation is relatively smaller than other faults. When a suspected fault is identified due to a cluster fault, it is possible to prevent or reduce the oversight of identifying the suspected fault location by making a lenient fault determination, so to speak, compared to the case of cracks as in FIG. 12, dirt as in FIG. 13, and cell faults. However, since the deviation tends to be large in the case of cracks, dirt, and cell faults, the criteria for identifying a cluster fault may excessively identify (detect) non-faulty areas. Therefore, in order not to reduce the efficiency of anomaly determination, excessive identification of suspected infringement locations is reduced.
For this reason, in this embodiment, faults are classified as described above, and further, the suspected target identification unit 391 described below performs fault determination corresponding to the aspect (contour shape, etc.) of the suspected fault location for each fault classification.

<Re-identification process_suspect target identification unit 391>
In this embodiment, a re-identification process of a specific suspect target is performed in accordance with the classification of the failure mode. The re-identification process will be described using a cluster failure as an example. Among the setting areas, setting areas other than the setting area classified by the determination unit 390 as including a cluster failure (for example, a photovoltaic power generation panel or a group thereof) (hereinafter referred to as "setting areas of other classifications") are the targets of the re-identification process.

The suspected target identification unit 391 re-identifies suspected failure locations in other classification setting areas using a second threshold value different from the threshold value (first threshold value) for the setting area suspected of cluster failure or the setting area without suspected failure locations. The second threshold value is a value that suppresses excessive identification of suspected failure locations in this setting area. For example, when a suspected failure is identified using the deviation value SS _i in the above embodiment, the number of locations identified as suspected failure locations is further reduced by increasing the threshold value of the deviation value SS _i .

As described above, in the case of a cluster fault where the deviation in heat generation may be small due to weather conditions or the progression of the fault, excessive identification of suspected locations in other classification setting areas is reduced by the re-identification process using the second threshold value.

This embodiment is not limited to the above. For example, a threshold value corresponding to cracks may be set to distinguish them from other classifications. Also, a threshold value corresponding to dirt or cell failure may be set to distinguish them from other classifications. Also, it is sufficient that the classification of failures is divided into at least two or more classifications.

In the third embodiment, the exclusion of suspected fault locations and the identification process after exclusion in the second embodiment may be combined.

[Action]
Fig. 14 is a diagram showing the flow of processing in this embodiment. Below, with reference to Fig. 14, the flow of processing of the state determination system 300 will be explained along with the step numbers (S031 to S03). Note that the following processing is executed after the processing of identifying the suspected fault location in the first embodiment and the like. In addition, the flow of an example of a form in which re-identification processing is performed after classifying the cluster fault area will be explained.

(S031)
After the suspected object identifying unit 391 identifies a suspected fault location, if a suspected fault location is found, the determining unit 390 classifies the type of fault in the set area. As an example, the determining unit 390 classifies the type of fault in the set area in the image by deep learning.

(S032)
As a result of the classification, the determining unit 390 determines whether the set region includes a cluster failure.

(S033)
If the set area is classified as not including a cluster failure according to the classification result of the determination unit 390 (S032; No), the determination unit 390 determines whether it includes a suspected area classified as an area with other failures. If the set area does not include a failure location (S033; No), there is no cluster failure and no other classification of failures, so the area is determined to be normal.

(S034)
If the classification result of the determining unit 390 determines that a suspect area classified as a different failure is included, the suspected object identifying unit 391 performs re-identification processing.

(S035)
If it is determined that the set area includes a cluster failure (S033; Yes), or after the re-identification process (S034), a process is performed to at least identify the position of the set area in its entirety and to present this to the user by any method.

[effect]
According to the third embodiment, the difference between the maximum temperature or the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of the infrared image. In addition, a judgment target value is obtained based on at least one of the value indicating the variation of the reference value of each set area and the average value for each set area for each group of multiple set areas. Furthermore, an abnormality of the imaged object is judged based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation of the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

Furthermore, in the third embodiment, in the case of a specific fault classification, it is possible to reduce excessive identification of suspected fault locations by performing a re-identification process of the suspected fault location.

The above embodiment reduces misjudgment, making it easier for panels with a low probability of failure to be excluded from suspected failures. This makes it possible to improve the efficiency of determining abnormalities in the imaged subject using infrared images.

The determination object detection system understood from the second embodiment can have the following features.
[Feature B]
an image receiving unit for receiving an infrared image from the flying object;
a temperature information acquisition unit for obtaining a distribution of temperature information based on infrared information;
a suspect target identification unit that identifies a suspect target that is an imaging target that may be at risk of failure based on a distribution of temperature information and a determination target value that indicates a variation between a high temperature range and a medium temperature range and a first threshold value;
a determination unit that classifies the disorder of the imaging target based on a pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, identifies the suspect target again based on a determination target value of the suspect target and a second threshold value;
A state determination system.
The third embodiment can be configured independently instead of being combined with the first or second embodiment.

[Judgment target detection system]
Next, a description will be given of a determination object detection system that can be combined with the above-described embodiments and their modified examples, as well as the technical ideas shown in the corresponding claims. The determination object detection system will be described in the fourth to tenth embodiments.

[Fourth embodiment]
The object detection system 400 of this embodiment receives infrared images, positioning information, shooting direction information, etc. from the flying object D, and identifies the shooting object, as in the above embodiment. As shown in Fig. 15, the object detection system 400 includes a position information receiving unit 410, an image receiving unit 420, an identifying unit 430, a temperature information acquiring unit 440, a control unit C, and a storage unit S. The position information receiving unit 410 receives positioning information (such as a three-dimensional position) of the flying object D based on the navigation signal from the flying object D. For example, these may be received as information added to the image as EXIF information.

The image receiving unit 420 receives an infrared image from the imaging unit D1 of the flying object D. The identification unit 430 identifies a specific imaging target or group of imaging targets from the infrared image. The imaging target is, for example, a solar power generation panel. Other imaging targets may include a large number of automobiles, building structures (glass, etc.), a large number of various sensors, mass-produced products that are inspected, etc. In the following, solar power generation panels will be used as an example of an imaging target or group of imaging targets.

As an example, the identification unit 430 detects each solar power generation panel in the infrared image received by the image receiving unit 420 by inferring it using deep learning. This uses an inference model created by deep learning using so-called artificial intelligence (AI). The inference model has a group of teacher data consisting of multiple pieces of data that identify each solar power generation panel from the infrared image.

The artificial intelligence performs deep learning using the group of teacher data, and an inference model is generated. In the identification unit 430, inference is performed on the received image using the inference model, and the coordinates of, for example, the outer edge of each solar panel are identified from the infrared image (it may be the coordinates of the four corners of the solar panel instead of the outer edge).

In this embodiment, the method of detecting solar panels from an infrared image is not limited to deep learning. Other methods, such as pattern matching, can also be used. Furthermore, the detection of solar panels does not have to be for each solar panel, but may be for a group of solar panels in two vertical columns and two horizontal rows, or for a group of solar panels in two vertical columns and one horizontal row. This detection unit may be determined for each detection system to be judged and may not be changed, or may be set by a user's operation via an input unit (not shown).

In addition, the above configuration detects the imaging target in an infrared image. However, this embodiment is not limited to this. For example, it is also possible to identify the imaging target by the following modified example. That is, in this system, visible light images and infrared images are received from the flying object D, and both can be used. Furthermore, the visible light images and infrared images are taken at the same angle. Therefore, the position of the solar power generation panel can be identified not only in infrared images, but also in visible light images. In this case, the identification unit 430 identifies the position of the imaging target in the visible light image by using any of the above inference techniques.

Next, the identification unit 430 uses the coordinates of the identified area of the imaging target in the visible light image as they are in the coordinates of the infrared image. This is based on the premise that the angle of view, angle, scale, etc. are the same or substantially the same (within the allowable range of the difference in panel positions between the two images) in the visible light image and the infrared image.

Alternatively, if there is a difference in the angle of view, angle, scale, etc. between the visible light image and the infrared image, and it is not acceptable to simply replace the coordinates of one image with the coordinates of the other (the difference is large), the identification unit 430 may be configured to align the scale and position by using some kind of landmark, marker, or morphological characteristics and positional characteristics of the imaged subject. Furthermore, the alignment is not limited to this.

For example, the identification unit 430 acquires the signal transmission time from a navigation signal sent from a global navigation satellite system as additional information for the visible light image. It also acquires the horizontal movement speed from a speed sensor or the like of the flying object D. It also compares the time attached to the visible light image with the time attached to the infrared image to determine the time difference and the time difference between the images.

The identification unit 430 also determines the amount of positional shift between the two images (the distance between the respective shooting positions) based on the moving speed and the time difference, etc., and determines in what direction and by how much the two images are shifted based on the composite pitch angle θ corresponding to each of the visible light image and the infrared image and the amount of shift. Furthermore, when there is a difference in the angle of view, for example when the visible light image has a wider area, the identification unit 430 can reduce the processing load or improve the accuracy of the processing results when performing processing to identify the range indicated by the infrared image in the visible light image based on the difference information thus identified.

In addition, because it is possible to determine the positional correspondence between the visible light image and the infrared image, it is also possible to unify the angle of view of both images or change the ratio of the angle of view of both images to any desired ratio.

The above configuration in which the detection of the imaging target is performed in a visible light image and the coordinates of the identified area are applied to an infrared image can also be applied to the following modified examples and other embodiments.

The temperature information acquisition unit 440 acquires temperature information within the panel using any known technology for each solar power generation panel in the infrared image identified by the identification unit 430.

(Control Unit C Configuration)
For convenience of explanation, the processing of each unit in the above description has been described as being executed under the control of the control unit C shown in Fig. 1. The description of the control unit C is omitted because it overlaps with the description of the above embodiment. Note that the multiple components in this embodiment (for example, at least two of the location information receiving unit 410, the image receiving unit 420, the identifying unit 430, and the temperature information acquiring unit 440) may be integrated into one processor to realize the functions.

(Control Unit C: Determination Processing)
The control unit C also executes a judgment process in the judgment target detection system. For example, the control unit C can judge a faulty panel from an infrared image based on the temperature difference and pattern. As a specific example, the control unit C determines whether or not the panel contains an abnormality by calculating the average temperature of all pixels or the temperature distribution of all pixels based on the temperature information of each pixel contained therein, and comparing the average temperature of all pixels or the temperature distribution of all pixels with a threshold value or a range of an appropriate range based on the number of pixels showing a high temperature. The control unit C may also store in advance the shape of a group (pattern) of pixels exhibiting an abnormal temperature, and identify a specific abnormality by comparing with the pattern.

Furthermore, if an abnormality is found for each photovoltaic panel as a post-processing of the determination process, the control unit C performs a process of at least identifying the position of that panel and presenting it to the user in a manner of their choice (see "DIFF" for the panel in the fourth row and sixth column of the upper block in Fig. 3, or Figs. 39 and 40, etc.). In addition to the panel position, the control unit C may also perform a process of creating and displaying image data such as the maximum temperature, average temperature, or number of abnormal pixels for each photovoltaic panel. Note that the control unit C corresponds to an example of a "determination unit" in this process.

[Action]
16 is a diagram showing the flow of processing in this embodiment. The flow of processing in the determination target detection system 400 will be described below along with step numbers (S041 to S045) with reference to FIG.

(S041)
The position information receiving unit 410 in the determination object detection system 400 receives position information from the flying object D. Furthermore, the image receiving unit 420 receives an infrared image.

(S042)
The identification unit 430 identifies the photovoltaic power generation panel in the infrared image.

(S043)
The temperature information acquisition unit 440 acquires temperature information within each photovoltaic panel in the infrared image identified by the identification unit 430. This may be performed for each pixel, or a plurality of pixels may be treated as one unit, and an average value for each pixel group unit may be used as the temperature information.

(S044)
The control unit C detects obstructed panels from the infrared images based on temperature differences and patterns.

(S045)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error in the abnormality judgment of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for the abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error in the abnormality judgment.

[Modification]
A determination object detection system 400 according to a modified example of the fourth embodiment will be described with reference to Fig. 17 and Fig. 18. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
As in the fourth embodiment, in this modification, the imaging target in the infrared image is identified from other objects, and the temperature information of the imaging target is excluded from the information used for anomaly determination. That is, the photovoltaic power generation panels are first identified, and anomaly determination is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in anomaly determination due to the arrangement of the photovoltaic power generation panels are also reduced.

The judgment target detection system 400 in this modified example detects the position of a solar power generation panel, for example, by detecting the solar power generation panel using an AI (deep learning) inference model or pattern matching, and by using the array information of the solar power generation panel.

(Object Detection System 400)
17 , the determination target detection system 400 of this modification includes a location information receiving unit 410, an image receiving unit 420, an identifying unit 430, a temperature information acquiring unit 440, a control unit C, a storage unit S, and a setting unit 410a. The identifying unit 430 detects a solar power generation panel from an infrared image, similar to the fourth embodiment.

(Setting unit 410a)
As shown in FIG. 2A, FIG. 2B and FIG. 3, the imaging target such as a photovoltaic power generation panel may have a specific arrangement for each installation site. Alternatively, the imaging target may have a specific arrangement tendency. In either case, by setting in advance arrangement information indicating the mode or state of the arrangement, it is possible to reduce identification errors by using the arrangement information for the position of the imaging target detected from the infrared image. For example, it is possible to detect a photovoltaic power generation panel from an infrared image based on the arrangement information and the infrared information (or temperature information) indicated by the imaging target in the infrared image. In this example, as shown in FIG. 3, one or more blocks of photovoltaic power generation panels arranged two-dimensionally with four panel groups arranged vertically in seven columns horizontally and seven panel groups arranged vertically in four rows horizontally are included in the infrared image. When the user inputs the arrangement information of the infrared image using an input unit (not shown), the setting unit 410a stores the arrangement information in the storage unit S. In the case of a photovoltaic panel, the temperature information presented in the image differs between the panel (e.g., silicon) and the outer frame (e.g., aluminum), so an example of detection is to detect the frame using infrared information or the like and identify each panel. However, this modified example is not limited to a configuration in which the user inputs the array information. For example, it is also possible to dynamically derive the number of arrays. In this case, the derived number of arrays is used to reduce identification errors.

As another example, in deep learning, in addition to the above training data set, this sequence information can also be used in the form of multimodal learning.

The identification unit 430 detects the photovoltaic power generation panel from the infrared image as in the first embodiment, and further identifies the photovoltaic power generation panel by using the arrangement information in the storage unit S. A first example will be given. When the arrangement information is set to have a group of photovoltaic power generation panels in 4 rows and 7 columns as shown in FIG. 3, the photovoltaic power generation panel may detect the photovoltaic power generation panel in a manner different from the arrangement information. For example, when the setting is 4 rows vertically, the number of rows is less than 4, or the number of rows is more than 4, or when the setting is 7 columns horizontally, the number of rows is less than 7, or the number of columns is more than 7. In this case, the identification unit 430 may perform detection again and repeat the detection until there is no difference with the arrangement information. In addition, the identification unit 430 may be configured to notify the detection error in any manner, such as by outputting an error message to a display unit (not shown) or by outputting a warning sound to an audio output unit (not shown).

Next, a second example will be given. When the array information includes a matrix-shaped photovoltaic panel group as shown in FIG. 3, the outline shape of the outer edge of the panel group is, for example, a rectangle. Therefore, after detecting a photovoltaic panel, the identification unit 430 further identifies the outer edge of the photovoltaic panel group based on the array information. Furthermore, the identification unit 130 identifies a photovoltaic panel detected at a position outside the outer edge as a detection error. When a detection error occurs, the processing of the identification unit 430 is the same as in the first example.
Other examples of processes that may be performed by the determination object detection system 400 after a detection error has been determined will be described in detail after the explanation of the tenth embodiment (variation 1 and subsequent examples).

[Action]
18 is a diagram showing the process flow in this modified example. The process flow of the determination target detection system 400 will be described below along with step numbers (S041a to S045a) with reference to FIG.

(S041a)
The position information receiving unit 410 in the determination object detection system 400 receives position information from the flying object D. Furthermore, the image receiving unit 420 receives visible light images and infrared images.

(S042a)
The identification unit 430 identifies the photovoltaic power generation panel in the infrared image.

(S013a)
The identifying unit 430 determines whether there is a detection error for the multiple photovoltaic panels identified in S042a based on the arrangement information set in the setting unit 410a. If there is a detection error (S043a; Yes), the identifying unit 430 repeats the detection until the detection error is eliminated. Note that the present invention is not limited to this, and may be configured to notify the user of the detection error and terminate the inspection of the photovoltaic panels.

(S044a)
If there is no detection error or the detection error has disappeared as determined in S043a (S043a; No), the temperature information acquisition unit 140 acquires temperature information within the panel for each photovoltaic power generation panel in the infrared image identified by the identification unit 430. This may be performed for each pixel, or multiple pixels may be considered as one unit and an average value for each pixel group unit may be used as temperature information.

(S045a)
The control unit C detects obstructed panels from the infrared images based on temperature differences and patterns.

(S046a)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error in the abnormality judgment of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for the abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error in the abnormality judgment.

Furthermore, according to this embodiment, detection errors of photovoltaic panels are determined based on the array information, which can reduce the number of missed detections of faults in photovoltaic panels.

[Fifth embodiment]
A determination object detection system 500 according to the fifth embodiment will be described with reference to Fig. 19 to Fig. 21. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
As in the fourth embodiment, the fifth embodiment also performs a process of identifying the object being imaged in the infrared image from other objects, and excluding temperature information of objects other than the object being imaged from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of photovoltaic power generation panels are also reduced.

The fifth embodiment further performs image tilt correction in order to detect the imaged object such as the solar power generation panel in the fourth embodiment. In one example of the fourth embodiment, object detection is performed based on the physical characteristics of the imaged object using an inference model using deep learning technology, inference technology such as pattern matching, etc. In the case of a solar power generation panel, for example, the area of the imaged object is inferred based on its morphological characteristics (such as a rectangular outer edge). However, if the imaging direction is tilted with respect to the object to be photographed, even if a rectangular area is inferred as shown in the left diagram of FIG. 20, there is a risk that other adjacent imaged objects or other objects that are not the imaged object will be included in the detected rectangular area. Therefore, in the fifth embodiment, a predetermined position is identified for each of the multiple imaged objects, and the image tilt is calculated from the positional relationship of each position. Furthermore, the calculated tilt is corrected to reduce detection errors of the imaged object.

(Judgment Object Detection System 500)
19 , the determination target detection system 500 of the fifth embodiment includes a position information receiving unit 510, an image receiving unit 520, an identifying unit 530, a temperature information acquiring unit 540, a control unit C, a storage unit S, and a first correction unit 510a. The identifying unit 530 detects a solar power generation panel from an infrared image, similar to the first embodiment.

(First correction unit 510a)
As in the first embodiment, an imaging target, for example, a photovoltaic power generation panel, is identified from an infrared image. The first correction unit 510a further identifies predetermined positions of the identified multiple photovoltaic power generation panels. For example, as shown in the left diagram of FIG. 20, the position of the upper left corner of each identified imaging target area (coordinates in the image, hereinafter referred to as the "specific position") is identified. Next, the first correction unit 510a calculates a regression line that minimizes the distance to each specific position by the least squares method.

Next, the first correction unit 510a calculates the slope of this regression line for the infrared image. The first correction unit 510a also performs a correction to rotate the image so that this slope is eliminated. The identification unit 530 performs identification processing of the imaged object again after the rotation correction. Note that the first correction unit 510a may be configured not to perform the correction processing if the slope is within a predetermined range (e.g., ±5°) or less.

The first correction unit 510a may also determine that the image is an imaging error if the inclination is equal to or greater than a predetermined range (e.g., ±5°). As a result of the error determination, a process may be executed to exclude the image from the target of panel abnormality determination (obstructed panel determination). Note that even if the image is excluded from the target of abnormality determination, it is possible to deal with the situation in the following manner. For example, at least one of azimuth information and position information is associated with the infrared image. Therefore, it is possible to identify the imaging target group associated with the excluded image. In other words, it is possible to take an aerial photograph of the position again and acquire the image again. It is also possible for the user to directly inspect only that part.

In addition, other countermeasures when an image is excluded from the target of abnormality judgment are listed below. In aerial photography by an aircraft, there may be a deviation between the imaging direction to be controlled and the actual imaging direction due to the influence of the environment at that time (wind, air pressure, and other factors). To solve this problem, the aircraft D may be controlled to perform aerial photography of one imaging target multiple times, i.e., to perform overlapping photography. In the judgment target detection system 500, it is possible to configure it so that two or more images are acquired for one imaging target in response to the overlapping photography. With such a configuration, even if one image is excluded according to the degree of tilt, it is possible to complement it with other images. In other words, if the tilt in the other images is within a predetermined range, it is possible to perform an abnormality (obstructed panel) judgment in that image. If the tilt is large, it is assumed that other problems other than tilt, such as image blurring and crushing (reduced resolution, etc.), are occurring. Therefore, the judgment of tilt by the first correction unit 510a can also be performed to reduce abnormality judgment errors due to other factors.

In the other examples above, the degree of overlap in overlapping shooting may be anywhere between 10% and 95%, for example. Furthermore, if the time interval between aerial photography is short, the effects of the aerial photography environment (strong winds, etc.) will not change, and the image immediately following the image determined to be excluded may also exceed the allowable range of tilt. Furthermore, if the overlap range is too wide, this may result in an increase in the aerial photography time, as well as an increase in the processing time and processing load on the system. Taking these factors into consideration, the degree of overlap in overlapping shooting may be set to, for example, 90% or less.

On the other hand, if the overlap range is too narrow, there is a possibility that some images will not be included in the determination of defective panels when one image is excluded. Taking this into consideration, the degree of overlap may be set to 50% or more.

Alternatively, the system may be configured to use the wind speed above the shooting location to calculate an appropriate degree of overlap, and set each time according to the calculation result. Depending on the degree of overlap, the system may be configured to exclude images immediately following an image that has been excluded due to a large tilt without going through tilt determination.

[Action]
21 is a diagram showing the flow of processing in this embodiment. The flow of processing in the determination target detection system 500 will be described below along with step numbers (S051 to S056) with reference to FIG.

(S051)
The position information receiving unit 510 in the determination object detection system 500 receives position information from the flying object D. Furthermore, the image receiving unit 520 receives visible light images and infrared images.

(S052)
The identification unit 530 identifies the photovoltaic power generation panel in the infrared image.

(S053)
The first correction unit 510a identifies a predetermined position in each of the multiple image capture targets, and calculates a regression line based on the least squares method from the positional relationship of each position. The first correction unit 510a also calculates the inclination of the regression line with respect to the image. The first correction unit 510a further corrects the calculated inclination. The identification unit 530 again identifies the photovoltaic power generation panel.

(S054)
The temperature information acquisition unit 540 acquires temperature information within each photovoltaic panel in the infrared image identified by the identification unit 530. This may be performed for each pixel, or a plurality of pixels may be treated as one unit, and an average value for each pixel group unit may be used as the temperature information.

(S055)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) from the infrared images based on the temperature difference and pattern.

(S056)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is identified from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included in a large amount, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, there may be positions that are exposed to direct sunlight due to shadows and positions that are not, and for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array. In this respect, too, the present embodiment can reduce errors in abnormality judgment.

Furthermore, according to this embodiment, the tilt that may be the cause of a detection error of a solar power panel once it has been detected is corrected, thereby reducing the number of missed detections of faults in the solar power panel.

Sixth Embodiment
A determination object detection system 600 according to the sixth embodiment will be described with reference to Fig. 22 to Fig. 24. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
Similarly, in the sixth embodiment, the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from the information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of the photovoltaic power generation panels are also reduced.

The sixth embodiment of the object detection system 600 further detects the object to be imaged, such as a solar power generation panel, and then determines whether there is a deviation in the detection size of the object to be imaged, in order to reduce errors in the detection process of the object to be imaged, as in the fourth embodiment.

In one example of the fourth embodiment, object detection is performed based on the physical characteristics of the imaging target using an inference model using deep learning technology, pattern matching, and other inference technology. In the case of a solar power generation panel, for example, the area of the imaging target is inferred based on its morphological characteristics (such as a rectangular outer edge). However, the outer edge of the imaging target may not be clearly depicted in the infrared image. As a result, even if a rectangular area is inferred, there is a risk that other imaging targets adjacent to the detected rectangular area or other objects that are not imaging targets will be included, as shown in FIG. 23. Therefore, in the sixth embodiment, the size of each of the multiple imaging targets is calculated, and a detection error is determined based on the calculated size. This aims to reduce detection errors of imaging targets.

That is, depending on the type of imaging target, the size may be uniform (such as a solar power generation panel). Therefore, taking advantage of the uniform size, if there are imaging targets that are detected as large (e.g., L6 in FIG. 23) and small, the target detection system 600 calculates this as a variation in size, and as a result, determines whether there has been a false detection.

For example, in the case of solar power generation panels, within the same power plant, the areas and shapes of the light receiving surface, solar radiation surface, etc. are configured to be roughly equal for construction reasons and operational reasons. Note that construction reasons take into consideration land use and the surface area irradiated by sunlight, etc., while operational reasons take into consideration status monitoring and replacement in the event of a breakdown, etc.

Therefore, the panels are formed so that there is no significant difference in the vertical length, horizontal length, circumference, area, diagonal length, etc. According to this premise, if solar panels of clearly different sizes are detected from the infrared image, there is a possibility that this is a detection error. Since this detection error interferes with the abnormality determination results of the temperature information in the judgment object detection system, the judgment object detection system 600 has a configuration that can determine a detection error when there is an imaged object of a different size. Note that even if there is another imaged object, if one that deviates from the predetermined regularity specific to the imaged object is detected, it can be determined as a detection error, and thus even if the imaged object is not a solar power generation panel, this embodiment also includes an example of the imaged object.

(Object Detection System 600)
22 , the determination object detection system 600 of the sixth embodiment includes a position information receiving unit 610, an image receiving unit 620, an identifying unit 630, a temperature information acquiring unit 640, a control unit C, a storage unit S, and a first determination unit 610a. The identifying unit 630 detects a solar power generation panel from an infrared image, similar to the first embodiment.

(First Determination Unit 610a)
As in the first embodiment, an imaging target, for example, a photovoltaic power generation panel, is identified from the infrared image. The first determination unit 610a further identifies a predetermined position in the identified multiple photovoltaic power generation panels. For example, as shown in Fig. 23, any position on the upper edge of the region of each identified imaging target (Y coordinate in the image, hereinafter referred to as "specific position") is identified. Next, the first determination unit 610a calculates the length in the Y-axis direction at the coordinate on the image in the infrared image.

The length of the object in the Y-axis direction may be calculated at the center position in the X-axis direction, or at one or the other end in the X-axis direction. That is, the length of each object in the Y-axis direction at a predetermined position in the X-axis direction is calculated. For example, the first determination unit 610a calculates the difference between the coordinates of the top end (top end in the image) and the coordinates of the bottom end as the length in the Y-axis direction. Also, multiple (predetermined number) lengths in the Y-axis direction may be calculated for each object and the average may be taken. In this case, the predetermined number is a maximum that corresponds to the number of pixels in the image in the X-axis direction.

Next, the first determination unit 610a compares the calculated feature amount or a numerical value calculated based on the feature amount with the determination value. In the above example, the determination value is compared with a numerical value indicating the size of the imaging target itself or a numerical value calculated based on the size. An example is described below.

After calculating the length of each of the photovoltaic panels in the Y-axis direction as described above, the first determination unit 610a calculates a variance value of the lengths of the photovoltaic panels. The variance value is calculated, for example, by the following formula.
The first determination unit 610a subtracts the average value from the length value of each detection area to obtain the distance (deviation) between each individual length and the average value. The first determination unit 610a also obtains the variance by taking the mean square of the deviation. The larger the variance value, the greater the spread of the lengths of the detection areas from the average value.

The first determination unit 610a also compares this variance value with the determination value, and if they are in a predetermined relationship, determines that there is a detection error. For example, the variance value of the lengths of all the photovoltaic panels included in one infrared image is calculated, and this variance value is compared with the determination value (e.g., the difference is calculated). In this case, if the difference between the variance value and the determination value is not within a predetermined range, the first determination unit 610a may determine that there is a detection error. In other words, both the cases where a panel with the above-mentioned length being too long and the case where a panel with the above-mentioned length being too short are detected are determined to be an error.

It is also possible to use the length in the X-axis direction and the area of the detected imaged object for the judgment, instead of the length in the Y-axis direction. In photovoltaic power generation facilities where photovoltaic power generation panels are arranged, the photovoltaic power generation panels tend to be arranged facing south in consideration of their power generation efficiency. In this case, the power generation equipment (power controller, cable) is often not arranged on the north-south side of the photovoltaic power generation panel group. Similarly, it is assumed that plants such as trees adjacent in the north-south direction are also excluded as much as possible. In other words, consideration is given to not casting a shadow on the light receiving surface. Therefore, the variation in temperature is suppressed in the north-south direction, that is, in the Y-axis direction in the infrared image. From the above, in the infrared image showing the temperature information of the photovoltaic power generation panel, the edge of the imaged object in the Y-axis direction is easily detected with high accuracy. In this embodiment, the large variation in size is used to judge whether the imaged object is detected or not, so an example using the length in the Y-axis direction has been described. If the imaged object is not a rectangular photovoltaic power generation panel, the size of the imaged object to be used can be set according to the characteristics of the imaged object itself or the location where it is placed.

The first determination unit 610a may be configured to determine the variance from the population mean for the entire power plant, rather than the sample mean in the captured image. Alternatively, the first determination unit 610a may be configured to determine the standard deviation, rather than the variance. In that case, the determination value will be a determination value corresponding to the standard deviation. The first determination unit 610a may be configured to compare the average value with the determination value. In that case, the determination value will be a determination value corresponding to the average value. The first determination unit 610a may be configured to compare the size of each captured object (e.g., the length in the Y-axis direction) itself with the determination value to determine a detection error.

As an example, when the first determination unit 610a determines a detection error, it excludes the target image from the targets for temperature abnormality determination (targets for faulty panel determination). As another example, when the first determination unit 610a determines a detection error, it may be configured to exclude only imaging targets with large size variation and determine temperature abnormality. The magnitude of size variation can be determined, for example, by whether the difference between the calculated value of the length in the Y-axis direction of one imaging target and the average length of each imaging target is within a specified range. It is also possible to determine whether the magnitude of size variation of each imaging target is within an acceptable range by using standard deviation and variance.

[Action]
Fig. 24 is a diagram showing the flow of processing in this embodiment. The flow of processing in the determination target detection system 600 will be described below along with step numbers (S061 to S067-2) with reference to Fig. 24.

(S061)
A position information receiving unit 610 in the determination object detection system 600 receives position information from the flying object D. An image receiving unit 620 receives visible light images and infrared images.

(S062)
The identification unit 630 identifies the photovoltaic power generation panel in the infrared image.

(S063)
The first determination unit 610a determines a predetermined size (for example, the length in the Y-axis direction) of each of a plurality of imaging targets. The first determination unit 610a also determines the variation in size of the imaging targets.

(S064)
The first determination unit 610a determines the magnitude of the variation in size calculated in S063 based on the determination value. If it is determined that the variation is large, the process proceeds to S065. If it is determined that the variation is not large, the process proceeds to S065-2.

(S065)
If there is an image determined to have a large variation in size in S064 (S064; Yes), the first determination unit 610a determines that a detection error has occurred. Furthermore, in this case, the image including the object determined to have a large variation is excluded from the object of temperature abnormality determination. As another example, the first determination unit 610a may exclude only the imaged object with a large variation in size and include the image in the object of temperature abnormality determination in S066. The temperature information acquisition unit 640 acquires temperature information for each of the imaged objects that are not excluded.

(S066)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

(S067)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

(S065-2)
If it is determined in S064 that the size variation in all images is not large (for example, within a predetermined range) (S064; No), the first determination unit 610a determines that there is no detection error. In this case, all images that were the subject of the determination of the appropriateness of detection in S064 are included as subjects for the determination of temperature abnormality. The temperature information acquisition unit 640 acquires temperature information for each imaging target.

(S066-2)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on the temperature differences and patterns of all infrared images (all detected detection areas) that have been judged for temperature abnormalities by the first judgment unit 610a.

(S067-2)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

Furthermore, according to this embodiment, the detection error of the imaged object is judged based on the variation in size, and if the variation is large, the image or the imaged object judged to be an error is appropriately excluded from the object for temperature anomaly judgment. Therefore, it is possible to reduce the number of errors in identifying a faulty panel.

[Seventh embodiment]
A determination object detection system 700 according to the seventh embodiment will be described with reference to Fig. 25 to Fig. 27. In the following description, descriptions that overlap with the seventh embodiment will be omitted.

(overview)
As in the fourth embodiment, the seventh embodiment also performs a process of identifying the imaging target in the infrared image from other objects, and excluding temperature information of the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of the photovoltaic power generation panels are also reduced.

In the seventh embodiment, the object detection system 700 further detects the object to be imaged and determines whether there is a deviation in the alignment position of the object to be imaged, in order to reduce errors in the detection process of the object to be imaged, such as a solar power generation panel, in the fourth embodiment. For example, it is determined whether there is a false detection in the detection of the boundary part of the outer edge of the object to be imaged or the area of each object to be imaged. In one example of the fourth embodiment, object detection based on the physical characteristics of the object to be imaged is performed using an inference model using deep learning technology, inference technology such as pattern matching. In the case of a solar power generation panel, for example, the area of the object to be imaged is inferred based on its morphological characteristics (such as a rectangular outer edge). However, the outer edge of the object to be imaged may not be clearly drawn in the infrared image. For example, as shown in FIG. 26, even if a rectangular area is inferred, the detected rectangular area may deviate from the position of the actual object to be imaged, or the detected area may partially protrude from the range of the area of the actual object to be imaged (such as in the X-axis or Y-axis direction; see FIG. 23). Therefore, in the seventh embodiment, the regularity of a predetermined arrangement specific to the object to be imaged is used to determine whether there is any deviation from the regularity. For example, a specific position for each of the multiple imaging targets identified from the image is further identified, and the amount of deviation of the identified positions from the arrangement rule is calculated. If the amount of deviation is large, a detection error is determined. This aims to reduce detection errors of the imaging targets.

For example, in the case of solar panels, the areas and shapes (rectangles, etc.) of the light receiving surfaces and sunlight irradiated surfaces are generally the same, and they tend to be regularly arranged two-dimensionally in rows and columns. Therefore, it is assumed that the outer edges of each of the multiple solar panels are aligned in both the row and column directions.

Using this premise, it is possible to determine detection errors by determining from the infrared image whether characteristic positions such as both ends in the Y-axis direction, both ends in the X-axis direction, corners, and the center position of the light receiving surface are aligned or misaligned in a specified direction. Note that the specified direction is, for example, the Y-axis direction or the X-axis direction, or a direction inclined at a specified angle relative to the Y-axis direction and the X-axis direction.

Even if the object being imaged is another object, if it is arranged in a certain regularity specific to the object being imaged, for example in a concentric circle arrangement or a regular wave-like arrangement, and something that deviates from the arrangement shape is detected, it can be determined as a detection error. In this way, even if the object being imaged is not a solar power generation panel, this is also included as an example of this embodiment.

(Judgment Object Detection System 700)
25 , the determination object detection system 700 of the seventh embodiment includes a position information receiving unit 710, an image receiving unit 720, an identifying unit 730, a temperature information acquiring unit 740, a control unit C, a storage unit S, and a second determination unit 710a. The identifying unit 730 detects a photovoltaic power generation panel from an infrared image, similar to the fourth embodiment.

(Second Determination Unit 710a)
As in the first embodiment, an imaging target, for example, a photovoltaic power generation panel, is identified from the infrared image. The second determination unit 710a further identifies a predetermined position in the identified multiple photovoltaic power generation panels. For example, as shown in FIG. 26, the second determination unit 710a identifies any position of the upper edge of the area of each imaging target identified by the identification unit 730 (Y coordinate in the image, hereinafter referred to as "Ymin" or "Ymin _n "). Next, the second determination unit 710a identifies any position of the lower edge of the area (Y coordinate in the image, hereinafter referred to as "Ymax" or "Ymax _n ").

Next, the second determination unit 710a calculates the variation of Ymin _i and the variation of Ymax _i for each of the identified imaging targets. The sum of these is set as the alignment variance. The variance value is calculated, for example, by the following formula.

Without being limited to the above, the second determination unit 710a may obtain the variance of each imaging target Y'min _n in the next row after sorting in the specified row direction and the variance of Y'max _n (see FIG. 26 for Y'min _n and Y'max _n ). The variance value is calculated, for example, by the following formula.

The second determination unit 710a also compares the variance value with the determination value, and if they are in a predetermined relationship, determines that there is a detection error. For example, the variance value of the Y coordinates of all photovoltaic power generation panels included in a predetermined arrangement direction (row direction, etc.) is calculated, and the variance value is compared with the determination value (e.g., the difference is calculated). In this case, if the difference between the variance value and the determination value is not within a predetermined range, the second determination unit 710a may determine that there is a detection error. In other words, when a panel whose alignment position is outside the allowable range is detected among the above Ymin ₁ to Ymin _n (upper edge) and Ymax ₁ to Ymax _n (lower edge), it is determined that there is an error.

It is also possible to use the X coordinate instead of the Y coordinate for the determination. The reason why an example of the variation in the alignment position using the Y coordinate is described in this embodiment is that, as described in the sixth embodiment, in an infrared image showing the temperature information of a solar power generation panel, the edges (top/bottom ends, Y coordinate) of the imaged object in the north-south direction are easy to detect with high accuracy. However, if the imaged object is not a rectangular solar power generation panel, it is sufficient to set which position on the imaged object (characteristic position (position that is easy to identify)) to use depending on the characteristics of the imaged object itself or the location where it is placed.

The second determination unit 710a may also be configured to determine the standard deviation instead of the variance. In that case, the determination value will be a determination value that corresponds to the standard deviation.

As an example, when the second determination unit 710a determines a detection error, it excludes the target image from the targets for temperature abnormality determination. As another example, when the second determination unit 710a determines a detection error, it may be configured to exclude only imaging targets with a large variation in the amount of deviation from the arrangement rule and determine temperature abnormality. The magnitude of the variation in the amount of deviation from the arrangement rule can be determined, for example, by whether the difference between the Y coordinate value of one imaging target and the average value of the Y coordinates of each imaging target is within a predetermined range. It is also possible to determine whether the magnitude of the variation in the amount of deviation from the arrangement rule for each imaging target is within an acceptable range by using standard deviation and variance.

[Action]
Fig. 27 is a diagram showing the flow of processing in this embodiment. Below, the flow of processing in the determination target detection system 700 will be described along with step numbers (S071 to S077-2) with reference to Fig. 27.

(S071)
A position information receiving unit 710 in the determination target detection system 700 receives position information from the flying object D. An image receiving unit 720 receives visible light images and infrared images.

(S072)
The identification unit 730 identifies the photovoltaic power generation panel in the infrared image.

(S073)
The second determination unit 710a identifies the Y coordinates of the upper and lower edges of each of the multiple imaging targets, and also determines the variation in these Y coordinates of the multiple imaging targets.

(S074)
The second determination unit 710a determines the magnitude of the variation in the Y coordinate calculated in S073 based on the determination value. If it is determined that the variation is large, the process proceeds to S075. If it is determined that the variation is not large, the process proceeds to S075-2.

(S075)
If there is an image determined to have a large Y coordinate variation (e.g., alignment dispersion) in S074 (S074; Yes), the second determination unit 710a determines that a detection error has occurred. Furthermore, in this case, the image of the object determined to have a large variation is excluded from the object of temperature abnormality determination. As another example, the second determination unit 710a may exclude only the imaging object with a large Y coordinate variation and include the image in the object of temperature abnormality determination in S076. The temperature information acquisition unit 740 acquires temperature information for each imaging object that is not excluded.

(S076)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

(S077)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

(S075-2)
If it is determined in S074 that the variation in the Y coordinates is not large (for example, within a predetermined range) in all images (S074; No), the second determination unit 710a determines that there is no detection error. In this case, all images that were used in S074 to determine whether the detection was successful are also included as targets for determining whether there is a temperature abnormality. The temperature information acquisition unit 740 acquires temperature information for each image capture target.

(S076-2)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on the temperature differences and patterns of all infrared images (all detected detection areas) that have been judged for temperature abnormalities by the second judgment unit 710a.

(S077-2)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

Furthermore, according to this embodiment, the detection error of the imaged object is judged based on the amount of deviation of the imaged object from the regularity of a predetermined arrangement, and if the amount of deviation is large, the image or the imaged object judged to be an error is appropriately excluded from the object for temperature anomaly judgment. Therefore, it is possible to reduce the number of errors in identifying a faulty panel.

[Eighth embodiment]
A determination object detection system 800 according to the eighth embodiment will be described with reference to Fig. 28 to Fig. 30. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
As in the fourth embodiment, the eighth embodiment also performs a process of identifying the object being imaged in the infrared image from other objects, and excluding temperature information of objects other than the object being imaged from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of photovoltaic power generation panels are also reduced.

The eighth embodiment of the object detection system 800 further detects the object to be imaged, such as a solar power generation panel, in order to reduce errors in the detection process of the object to be imaged in the fourth embodiment, and then sequentially identifies the predetermined position of the object to be imaged in accordance with the arrangement direction of the detected object to be imaged. Furthermore, the object detection system 800 determines a detection error according to the degree of overlap between the object to be imaged or the group of objects to be imaged that is the object to be imaged and the object whose predetermined position was identified immediately before. For example, if the center position of the object to be imaged is located within the area of an adjacent object to be imaged, it is determined to be detection noise. Furthermore, the object to be imaged (detection area) determined to be detection noise is excluded from the obstruction panel determination.

(Judgment Object Detection System 800)
28 , the determination object detection system 800 of the eighth embodiment is configured to include a position information receiving unit 810, an image receiving unit 820, an identifying unit 830, a temperature information acquiring unit 840, a control unit C, a storage unit S, and a third determination unit 810a. The identifying unit 830 detects a solar power generation panel from an infrared image, similar to the first embodiment.

(Third Determination Unit 810a)
As in the fourth embodiment, an imaging target, for example, a solar power generation panel, is identified from the infrared image. The third determination unit 810a further identifies a predetermined position in the identified multiple solar power generation panels. For example, as shown in FIG. 29, the third determination unit 810a identifies any position (X coordinate in the image, hereinafter referred to as "Xmin") of the left edge of the area of each imaging target identified by the identification unit 830. Next, the third determination unit 810a identifies any position (X coordinate in the image, hereinafter referred to as "Xmax") of the right edge of the area. Furthermore, the third determination unit 810a identifies the center position (X coordinate in the image, hereinafter referred to as "CenterX") of the area of each imaging target.

Next, the third determination unit 810a determines whether the center position CenterX in the X-axis direction for each identified imaging target is included in the region of the adjacent imaging target. For example, as shown in FIG. 29, when CenterX is located between one end in the X-axis direction of the adjacent region (e.g., the left end position in the X-axis direction; "Prev Xmin") and the other end (e.g., the right end position in the X-axis direction; "Prev Xmax"), the third determination unit 810a may be configured to determine that the region of the determination target is included in the adjacent region.

In the description of this embodiment, the center position of the region of the imaging target to be judged is used because the solar power generation panel is taken as an example as described in the sixth embodiment. That is, in a solar power generation facility, the possibility that a shadow will be cast on the solar power generation panel at the end side in the X-axis direction is relatively high compared to all panels. Therefore, it can be said that the technical difficulty is relatively higher in detecting the edge in the X-axis direction than in the Y-axis direction. Therefore, if an error is judged to occur when the edge in the X-axis direction of the region of the imaging target to be judged is located between "Prev Xmin" and "Prev Xmax", the probability of error judgment will be high, and there is a possibility that error judgment will occur frequently depending on the infrared image. Therefore, frequent error judgment can be prevented by using, for example, the degree of overlap between the adjacent region and the judgment region being 30% or more as an index, or by using the center position as a reference as in this embodiment.
For this purpose, the present embodiment is not limited to a configuration in which the reference position of the region of the imaged object to be determined is the center position. For example, the reference position may be any position that is at least 30% of the total length of the region of the object in the X-axis direction from both ends of the region of the object.

When calculating the degree of overlap as a ratio or percentage as described above (e.g., "overlap of 30% or more"), one example is to compare the area belonging to both with the area of each judgment target region. Alternatively, it is also possible to determine the degree of overlap based on the ratio of the length from "Prev Xmin" to "Xmin" of the judgment target region to the length in the X-axis direction of the judgment target region.

It is also possible to use the Y-axis direction as the reference, rather than the degree of overlap in the X-axis direction. For example, when combining the sixth or seventh embodiment with this embodiment, if the sixth or seventh embodiment uses the displacement or protrusion of each imaging target area in the X-axis direction as the reference in the sixth or seventh embodiment, it is conceivable to use the Y-axis direction as the reference for positional displacement in this embodiment. By determining detection errors in both the X-axis and Y-axis directions, it is possible to more easily prevent misjudgment of obstructed panel determinations due to detection errors.

As an example, when the third determination unit 810a determines a detection error, it excludes the target image from the targets for temperature abnormality determination (targets for faulty panel determination). As another example, when the third determination unit 810a determines a detection error, it may be configured to exclude only the imaged target with a large positional deviation and perform a temperature abnormality determination. The criteria for the magnitude of positional deviation may be the same as the above determination criteria, or may be changed.

[Action]
Fig. 30 is a diagram showing the flow of processing in this embodiment. Below, the flow of processing in the determination target detection system 800 will be described along with step numbers (S081 to S087-2) with reference to Fig. 30.

(S081)
A position information receiving unit 810 in the determination target detection system 800 receives position information from the flying object D. An image receiving unit 820 receives visible light images and infrared images.

(S082)
The identification unit 830 identifies the solar power generation panel in the infrared image.

(S083)
The third determination unit 810a specifies the X coordinate of a predetermined position in each of the multiple imaging targets, and determines the degree of overlap with adjacent regions based on the specified X coordinate. For example, the third determination unit 810a specifies Xmin ₁ to Xmin _n , Xmax ₁ to Xmax _n , and CenterX ₁ to CenterX _n .

(S084)
Next, the third determination unit 810a determines whether CenterX of the determination target area is included in the area of the adjacent imaging target. For example, as shown in FIG. 17, it determines whether CenterX _n is located between "Prev Xmin" and "Prev Xmax" of the adjacent area.

(S085)
If it is determined in S084 that the CenterX of the region to be judged is included in the region of the adjacent imaging target (S084; Yes), the third judgment unit 810a judges the region as a detection error. Furthermore, in this case, the third judgment unit 810a excludes the image of the target determined to be included in the adjacent region from the temperature abnormality judgment target (obstructed panel judgment). As another example, the third judgment unit 810a may exclude only the corresponding imaging target region and include the image in the target of the temperature abnormality judgment (obstructed panel judgment) in S086. The temperature information acquisition unit 840 acquires temperature information for each imaging target that is not excluded.

(S086)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

(S087)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

(S085-2)
If it is determined in S084 that the CenterX of the determination area is not included in the adjacent area in all images (S084; No), the third determination unit 810a determines that there is no detection error. In this case, all images (all determined detection areas) that were used as the objects for determining whether or not the detection was successful in S084 are included as objects for determining whether or not there is a temperature abnormality. The temperature information acquisition unit 840 acquires temperature information for each of the imaging objects.

(S086-2)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) by temperature difference and pattern, based on all infrared images that have been subject to temperature abnormality judgment by the third judgment unit 810a.

(S087-2)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

Furthermore, according to this embodiment, the detection error of the imaged object is judged based on the degree of overlap between the imaged object area and the adjacent area, and if the degree of overlap is large, the image or the imaged object judged to be an error is appropriately excluded from the temperature anomaly judgment target. Therefore, it is possible to reduce the number of errors in identifying a faulty panel.

[Ninth embodiment]
A determination object detection system 900 according to the ninth embodiment will be described with reference to Fig. 31 to Fig. 34. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
As in the fourth embodiment, the ninth embodiment also performs a process of identifying the imaging target in the infrared image from other objects, and excluding temperature information of the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of the photovoltaic power generation panels are also reduced.

The ninth embodiment of the judgment target detection system 900 further detects the image capture target, such as a solar power generation panel, in order to reduce errors in the detection process of the image capture target in the fourth embodiment, and then sequentially identifies the predetermined position of the image capture target according to the arrangement direction of the detected image capture targets. Furthermore, the judgment target detection system 900 gathers together multiple image capture targets identified in the infrared image according to the regularity of the arrangement of the image capture targets (hereinafter, the collection is referred to as a "cluster"). Also, in this embodiment, a predetermined number of image capture targets that should be included in a cluster at a predetermined position is stored. The judgment target detection system 900 judges a detection error based on whether the number of detections in one cluster meets the predetermined number.

The judgment object detection system 900 may further be configured to perform an additional judgment of a detection error based on the distance between adjacent clusters (variant example described below). Furthermore, the judgment object detection system 900 excludes an imaged object (detection area) that is judged as a detection error from the judgment of an obstructed panel.

(Judgment Object Detection System 900)
31 , the determination object detection system 900 of the ninth embodiment includes a position information receiving unit 910, an image receiving unit 920, an identifying unit 930, a temperature information acquiring unit 940, a control unit C, a storage unit S, and a fourth determination unit 910a. The identifying unit 930 detects a solar power generation panel from an infrared image, similar to the first embodiment.

(Fourth Determination Unit 910a)
As in the first embodiment, an imaging target, for example, a photovoltaic power generation panel, is identified from the infrared image. The fourth determination unit 910a further identifies a predetermined position in the identified multiple photovoltaic power generation panels. For example, the imaging targets may be arranged two-dimensionally. In this case, it is possible to group imaging targets aligned in the X-axis direction in the infrared image as one cluster, as shown in FIG. 32 (see Cluster C to Cluster F).

As an example, the fourth determination unit 910a identifies the Ymin and Ymax of the imaging target, such as the upper and lower edges in the Y-axis direction, or the corners on the upper and lower edges. Furthermore, the fourth determination unit 910a searches for imaging targets whose Ymin and Ymax are close to each other among the multiple imaging targets detected, so that imaging targets aligned in the X-axis direction are included in one cluster. As an example, the fourth determination unit 910a uses the Ymin coordinate of one detection area as a reference and compares it with the Ymin of other detection areas to identify those other detection areas whose Y coordinates are within a predetermined distance.

The fourth determination unit 910a similarly compares the Ymax coordinate of one detection area with the Ymax of other detection areas, and identifies those other detection areas that are within a specified distance in Y coordinate. In this way, the fourth determination unit 910a associates the one detection area with the identified other detection areas as belonging to one cluster. In this way, the fourth determination unit 910a identifies clusters and detection areas within the clusters in the infrared image.

The formation of the above clusters involves determining which Y coordinates are close at Ymin and Ymax, but this is merely an example and the present embodiment is not limited to this. For example, the fourth determination unit 910a may form a cluster by determining which Y coordinates are close at least in one of Ymin and Ymax, or only in one of them. As another example, a cluster may be formed by determining which Y coordinates are close based on the center point of the imaging subject. Similarly, if the imaging subject is polygonal, one of the corners, or any number of the corners, may be used.

The fourth determination unit 910a then determines the number of detection areas contained in each cluster (see Cluster A to Cluster F in FIG. 32). It also compares the number of area detections in this cluster with a stored predetermined number. That is, the determination target detection system 900 compares the predetermined number that should be contained in one stored cluster with the number of detections of imaging targets that are actually contained in the cluster. This predetermined number is derived from the characteristics of the imaging targets and their placement locations, and the regularity of the arrangement, and may be set by the user, for example.

When the number of detections in a cluster does not reach a predetermined number, the fourth determination unit 910a determines that all detection areas belonging to the cluster are detection errors. In this case, as one example, the fourth determination unit 910a excludes the target image from the targets for temperature abnormality determination. As another example, the fourth determination unit 910a may be configured to determine temperature abnormality by excluding detection areas belonging to the cluster to be determined. Note that, without being limited to the above, the configuration may also be such that imaged targets arranged in the Y-axis direction are grouped into one cluster.

[Modification]
Next, a modified example of the ninth embodiment will be described. In this modified example, a process of grouping a plurality of imaging targets into one cluster is executed as in the ninth embodiment. However, in this modified example, the fourth determination unit 910a obtains the average value of Ymin and the average value of Ymax for each imaging target in a cluster, and sets these as the Ymin and Ymax of the cluster. The Ymin and Ymax of a cluster may be the median values of the Ymin and Ymax of each imaging target to which the cluster belongs. Furthermore, the fourth determination unit 910a obtains the variance value of the distance to the nearest cluster in the Y-axis direction. In addition, the obtained variance value is compared with the determination value, and if the two have a predetermined relationship, it is determined as a detection error. The variance value is calculated, for example, by the following formula.

The fourth determination unit 910a calculates the difference between the average distances from the cluster to the nearest cluster and the target cluster, and calculates the variance by taking the mean square of the deviations. The larger the variance, the greater the spread of each data item from the average value.

In addition, in this modified example, a judgment value for judging the variation in the interval between clusters is stored in a storage means (not shown). The fourth judgment unit 910a compares the above variance value and the judgment value, and judges it as a detection error if they have a predetermined relationship. For example, the variance value of the interval between Ymin of the cluster to be judged and Ymax of the cluster above is calculated, and the variance value is compared with the judgment value (e.g., the difference is calculated). Similarly, the variance value of the interval between Ymax of the cluster to be judged and Ymin of the cluster above is calculated, and the variance value is compared with the judgment value. In this case, if the difference between the variance value and the judgment value is not within a predetermined range, the fourth judgment unit 910a may judge it as a detection error. In the example of FIG. 33, the imaging targets are arranged two-dimensionally, and large intervals are not provided between the clusters in terms of the arrangement efficiency of the imaging targets. Therefore, if there is an interval longer than a predetermined interval on both the upper side and the lower side of the cluster to be judged, all detection areas belonging to that cluster may be considered to be detection noise.

The above judgment of the variation in the spacing between clusters was made with respect to the spacing between the Ymin of the object to be judged and the Ymax of the cluster above. However, this is merely an example, and the present embodiment is not limited to this. For example, the fourth judgment unit 910a may judge the variation in the spacing between the Ymin of the object to be judged and the Ymin of the cluster above (or below). Also, for example, the fourth judgment unit 910a may judge the variation in the spacing between the Ymax of the object to be judged and the Ymax of the cluster above (or below).

In the example of a solar panel, clusters aligned in the X-axis direction as described above tend to be arranged adjacently in the Y-axis direction to form an array configuration. In this case, the theoretical optimum value for the difference between Ymin of the cluster being evaluated and Ymax of the cluster located above is 0. In the above example of this embodiment, which compares variance, both upward and downward deviations of the reference average value are likely to be judged as detection noise. Therefore, it can be said that such detection noise is more likely to be detected if it is unidirectional.

In other words, cases where there is a negative deviation from the average value can be said to be in a state where clusters overlap, but this case can be removed using technology such as that of the seventh embodiment. In other words, if detection noise is concentrated in the upward deviation (when there is a gap between clusters), it becomes easier to reduce the occurrence of noise. Therefore, the explanation of this embodiment has been centered on the gap between Ymin of the object to be determined and Ymax of the upper cluster.

As an example, when the fourth determination unit 910a determines a detection error, it excludes the target image from the target for temperature abnormality determination. As another example, when the fourth determination unit 910a determines a detection error, it may be configured to exclude the detection area that belongs to the cluster in question and determine the temperature abnormality. Note that, without being limited to the above, it may also be configured to group the imaged targets arranged in the Y-axis direction into one cluster.

[Action]
34 is a diagram showing the process flow in this embodiment (determination based on the number of detections in a cluster). The process flow of the determination target detection system 900 will be described below along with step numbers (S091 to S097-2) with reference to FIG.

(S091)
A position information receiving unit 910 in the determination target detection system 900 receives position information from the flying object D. An image receiving unit 920 receives visible light images and infrared images.

(S092)
The identification unit 930 identifies the photovoltaic power generation panel in the infrared image.

(S093)
The fourth determination unit 910a, based on the arrangement rule of the imaging objects, groups together multiple imaging objects included in a part of the arrangement according to the rule into one cluster. When multiple imaging objects are arranged two-dimensionally, imaging objects aligned in the X-axis direction in the infrared image are grouped together into one cluster as shown in Figures 32 and 33. For example, a predetermined position (e.g., Ymin and Ymax) is specified for each imaging object in the infrared image. The fourth determination unit 910a also groups together multiple areas within a predetermined distance based on the specified predetermined position, for example, the Y coordinate, into one cluster.

(S094)
The fourth determination unit 910a obtains the number of detection areas included in each cluster (see FIG. 32; Cluster A to Cluster F) and compares it with a stored predetermined number. The fourth determination unit 910a determines whether the number of detection areas satisfies the predetermined number as a result of the comparison.

(S095)
As a result of the comparison in S94, if the number of detection areas in the cluster does not reach a predetermined number (S094; Yes), the fourth judgment unit 910a judges the area belonging to the cluster as a detection error. Furthermore, in this case, the fourth judgment unit 910a excludes the target image from the temperature abnormality judgment target. As another example, the fourth judgment unit 910a may exclude only the detection areas belonging to the judgment cluster as the temperature abnormality judgment target (obstructed panel judgment) and include the image in the temperature abnormality judgment target of S096. As another example, the fourth judgment unit 910a may exclude only the corresponding imaging target area and include the image in the temperature abnormality judgment target of S096. The temperature information acquisition unit 940 acquires temperature information for each imaging target that is not excluded.

(S096)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

(S097)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

(S095-2)
If it is determined in S094 that the number of detection regions in the cluster satisfies the predetermined number (S094; No), the fourth determination unit 910a determines that there is no detection error. In this case, all images (all determined detection regions) that were the subject of the determination of the appropriateness of detection in S094 are included as subjects for the determination of temperature abnormality. The temperature information acquisition unit 940 acquires temperature information for each imaging target.

(S096-2)
The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) by temperature difference and pattern, based on all infrared images that have been subject to temperature abnormality judgment by the fourth judgment unit 910a.

(S097-2)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[Operation of the Modified Example]
In the case of the above-described modification of this embodiment, in S094, the fourth determination unit 910a identifies predetermined positions (for example, Ymin and Ymax) in each of the multiple imaging targets, and thereby obtains the distance from the nearest other area.

In the case of the above modified example, if the comparison result of S94 in S095 shows that there is a gap longer than the predetermined gap on both the upper and lower sides of the cluster to be judged (S094; Yes), the fourth judgment unit 910a judges the detection area belonging to that cluster as a detection error. The processing after the error judgment is the same as in the ninth embodiment.

In the case of the above modified example, in S095-2, if the comparison in S094 shows that the distance is within a predetermined length on either the upper side or the lower side of the cluster being judged (S094; No), the fourth judgment unit 910a judges that there is no detection error. In this case, all images that were the subject of judgment on the suitability of detection in S094 (all judged detection areas) are included as subjects for judgment of temperature abnormality.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

Furthermore, according to this embodiment, the detection error of the imaged object is judged based on the presence or absence of deviation from the regularity of the imaged array, or the degree of deviation. Also, based on the judgment result, images judged to be erroneous or imaged objects judged to be erroneous are appropriately excluded from the targets for temperature anomaly judgment. Therefore, it is possible to reduce the number of errors in identifying obstructed panels.

[Tenth embodiment]
A determination object detection system 1000 according to a tenth embodiment will be described with reference to Fig. 35 to Fig. 38. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

(overview)
As in the fourth embodiment, the tenth embodiment also performs a process of identifying the imaging target in the infrared image from other objects, and excluding temperature information of the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, abnormality judgment errors due to the arrangement of the photovoltaic power generation panels are also reduced.

The target detection system 1000 of the tenth embodiment further detects the target and sequentially identifies the predetermined position of the target according to the arrangement direction of the detected target in order to reduce errors in the detection process of the target such as a photovoltaic power generation panel in the fourth embodiment. Furthermore, the target detection system 1000 groups the multiple targets identified in the infrared image into one cluster according to the regularity of the arrangement of the targets. In this embodiment, the target detection system 1000 corrects the targets included in a cluster that are displaced from the bottom edge position of the cluster based on the bottom edge position of the cluster at the bottom end position in one group of targets (cluster group). Note that the bottom edge position indicates the bottom side from the viewpoint of the figure as shown in FIG. 36, and does not indicate the vertically downward direction in real space. In a photovoltaic power generation facility, this bottom side tends to be the south side.

That is, as mentioned above, in solar power generation facilities, solar panels tend to be positioned facing south. In this case, the solar panels are mounted on a stand that is higher on the north side and lower on the south side so that the light-receiving surface faces south, i.e., slopes downward toward the south (see Figure 37).

In this configuration, as shown in FIG. 37, a shaded area and a sunny area are created to the south of each photovoltaic panel belonging to the bottommost cluster. For example, in a panel located at the bottom of a block, sunlight is blocked by the northern end (upper end in FIG. 36) of the panel adjacent to it to the south, creating a shaded area. However, as shown in FIG. 37, sunlight is not blocked by the northern end of the adjacent panel to the south, so that area (the area that is irradiated with direct sunlight) becomes a sunny area (see FIG. 36; symbol SP).

The temperature in sunny areas is relatively higher than in shaded areas, which can result in detection noise when determining whether a panel is obstructed. This embodiment makes it possible to reduce this detection noise by using the following configuration.

(Judgment Object Detection System 1000)
35, the determination object detection system 1000 of the tenth embodiment includes a position information receiving unit 1010, an image receiving unit 1020, an identifying unit 1030, a temperature information acquiring unit 1040, a control unit C, a storage unit S, a fifth determination unit 1010a, and a correction unit 1010b. The identifying unit 1030 detects a solar power generation panel from an infrared image in the same manner as in the fourth embodiment.

(Fifth Determination Unit 1010a)
As in the fourth embodiment, an imaging target, for example, a solar power generation panel, is identified from the infrared image. As in the sixth embodiment, the fifth determination unit 1010a can group imaging targets aligned in the X-axis direction in the infrared image into one cluster (see Cluster_num1 to Cluster_num4 in FIG. 36).

Furthermore, the fifth determination unit 1010a calculates the average value of Ymax for each imaging object in the cluster, and sets this as the Ymax of the cluster. Note that the Ymax of a cluster may be the median value of the Ymax of each imaging object to which it belongs.

The fifth determination unit 1010a also identifies the bottom row of the cluster (see Cluster_num4 in Figure 36) in one block of a two-dimensional array of multiple imaging targets. The method of identification is, for example, as follows. The control unit C displays the infrared image after the cluster has been identified on a display means (including a display means outside the system) not shown (see Figure 36). The control unit C also executes control to display an identification display for each cluster for each array of imaging targets in the infrared image. In the example of Figure 36, this identification display is a serial number displayed on the cluster. It may also be another identification display that allows the user to specify each cluster.

In this state, the user can visually confirm the identification markings for each cluster via the display means, and can further specify them using input means (not shown). In this configuration, when the user specifies any one of the identification marks (tier number, cluster number, etc.) among the group of identification marks on the screen using the input means, the control unit C accepts the specification operation and identifies the cluster in the infrared image.

Note that other methods may be used to identify the bottommost cluster in a two-dimensional array of multiple imaging targets. For example, it is possible to determine that a cluster whose distance to the nearest other region on the top side (Ymin) of the cluster is within a predetermined distance and whose distance to the nearest other region on the bottom side (Ymax) of the cluster exceeds a predetermined distance is the bottommost cluster. In this configuration, a cluster with no other region on the bottom side is also determined to be the bottommost cluster.

After identifying the bottom cluster, the fifth determination unit 1010a compares the Ymax coordinate of each detection area belonging to the cluster with the Ymax coordinate of the cluster. For example, the fifth determination unit 1010a finds the difference (distance) between the Ymax coordinate of each detection area and the Ymax coordinate of the cluster. The fifth determination unit 1010a further determines whether the difference is within a predetermined range. As an example, the fifth determination unit 1010a compares this difference with a determination value. If the difference exceeds the predetermined range, the fifth determination unit 1010a identifies the detection area.

(Correction unit 1010b)
The correction unit 1010b causes the bottom side of the detection area specified by the fifth determination unit 1010a to correspond to the bottom side of the cluster. For example, the correction unit 1010b corrects the Y coordinate of Ymax of the detection area so that it corresponds to the Y coordinate of Ymax of the cluster based on the calculated difference.

[Action]
Fig. 38 is a diagram showing the process flow in this embodiment. The process flow of the determination object detection system 1000 will be described below along with step numbers (S101 to S108-2) with reference to Fig. 38.

(S101)
A position information receiving unit 1010 in the determination object detection system 1000 receives position information from the flying object D. An image receiving unit 1020 receives a visible light image and an infrared image.

(S102)
The identification unit 1030 identifies the photovoltaic power generation panel in the infrared image.

(S103)
The fifth determination unit 1010a, based on the arrangement rule of the imaging objects, groups together multiple imaging objects included in a part of the arrangement according to the rule into one cluster. When multiple imaging objects are arranged two-dimensionally, imaging objects aligned in the X-axis direction in the infrared image as shown in FIG. 36 are grouped together into one cluster. For example, a predetermined position (e.g., Ymin and Ymax) is identified for each imaging object in the infrared image. The fifth determination unit 1010a also groups together multiple areas within a predetermined distance based on the identified predetermined position, for example, the Y coordinate, into one cluster.

(S104)
The fifth determination unit 1010a identifies the bottommost cluster among the clusters (see Cluster_num1 to Cluster_num4 in FIG. 36) included in one block of the two-dimensional array of the imaging target.

(S105)
The fifth determination unit 1010a compares the Ymax coordinate of each detection area belonging to the cluster with the Ymax coordinate of the cluster, and further determines whether the difference is within a predetermined range.

(S106)
If the comparison in S105 shows that any of the detection areas in the bottommost cluster has a bottom side separated from the bottom side of the cluster (S105; Yes), the correction unit 1010b performs a correction so that the bottom side of that detection area corresponds to the bottom side of the cluster.

(S107)
The temperature information acquisition unit 1040 acquires temperature information including the detection area in the bottommost cluster after the lower side of the detection area has been corrected, and detects faulty panels for each panel (for each group of a predetermined number of panels) based on the temperature difference and pattern.

(S108)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

(S106-2)
In S0105, if the difference between the bottom side of each of the detection areas in the cluster and the bottom side of the cluster is within a predetermined range (S104; No), temperature information is obtained for each panel (for each group of a predetermined number of panels) without performing correction processing by correction unit 1010b.

(S107-2)
The temperature information acquisition unit 1040 detects a faulty panel based on the temperature difference and pattern for each panel (each group of a predetermined number of panels).

(S108-2)
The control unit C indicates to the user any obstructed panels detected from the infrared image.

[effect]
According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

Furthermore, according to this embodiment, the deviation of the bottom edge position of each detection area from the bottom edge position of the bottommost cluster of each block including the imaging target group is determined, and if there is a detection area that deviates beyond the allowable range, a correction is made to align the bottom edge position of that area with the cluster. Also, in this embodiment, an obstruction panel determination is made after the correction. Therefore, it is possible to reduce errors in identifying obstruction panels.

[Modification 1]
Next, other examples of processing that the judgment object detection system can take after the judgment of a detection error in each of the above embodiments will be described. In the above embodiments (5 to 9), a detection error of the imaging object is judged, and if a detection error occurs, the detection area or an image including the detection area is excluded from the judgment of an obstructed panel. However, other configurations are also possible. For example, the judgment object detection system can be configured to notify the detection error in any manner, such as by outputting an error message to a display means (including a display means outside the system) not shown, or by outputting an alarm sound to an audio output unit not shown.

Furthermore, the control unit C of the judgment target detection system presents the user with an image including the area involved in the detection error, either after or without notifying the user. One example of this is to display the image on any display means (including display means outside the system) not shown. Furthermore, a message is displayed prompting the user to select whether to manually complete the portion of the image that does not match the sequence information.

As shown in Fig. 3, 39, 40, etc., the imaging target identified by the identification unit can be made recognizable to the user by the display means. In other words, the control unit C can make the area that the identification unit has identified as the imaging target in the infrared image visible to the user in the infrared image. For example, as shown in Fig. 3, 39, 40, etc., it is possible to draw the outer edge of the imaging target area so that it is distinguishable from others. In addition, it is possible to draw the entire area of each imaging target in any manner so that it is distinguishable from others (shaded, inverted, flashing).

Furthermore, the control unit C displays the area in the infrared image where a detection error occurred so that an operation can be performed to slide the area on the displayed image. For example, the control unit C highlights the outer edge of the area where an error was determined. Furthermore, when the user uses any input means such as a pointing device to superimpose a pointer or the like on the image on the displayed area, the control unit C transitions to a state in which it can accept an operation to slide the area on the image. Furthermore, when the user in this state moves the area to any position on the image via the input means, the display showing the area moves to the specified position on the image.

Furthermore, when the user executes an operation to complete the movement of the area via the input means, the control unit C rewrites the coordinate values of the area before the movement to the coordinate values after the movement. In this embodiment, the configuration allows the operation of correcting the area involved in the detection error by manual operation as described above. This embodiment also makes it possible to reduce mistakes in identifying obstructing panels.

[Modification 2]
In the above embodiment, the configuration is such that visible light images and infrared images are acquired in the aerial photography of the flying object D. However, depending on the imaging subject, the configuration may be such that only infrared images are acquired.

[Modification 3]
In the above embodiment, a visible light image and an infrared image are acquired in the aerial image of the flying object D. However, depending on the image-capturing target, instead of an aerial image, an image may be captured by an imaging means fixed above the image-capturing target (for example, fixed to the ceiling or the top of a structure (pole, etc.) protruding upward), and the condition of the image may be determined. For example, the condition may be determined from temperature information of a large number of image-capturing targets (stored items, manufactured items, etc.) arranged in a warehouse or factory.

The determination object detection system grasped from the above-mentioned modified example 3 can have the following features.
[Feature 1]
an image receiving unit that receives an infrared image acquired by photographing the imaging target from above the imaging target;
an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the infrared image;
a temperature information acquisition unit that acquires temperature information based on infrared information for each of the identified imaging targets or each of the imaging target groups;
a determination unit that performs an abnormality determination for each of the identified imaging targets or each of the imaging target groups based on the temperature information obtained by the temperature information acquisition unit;
A judgment object detection system comprising:

[Modification 4]
In the above embodiment, the state of the imaging target is determined from the temperature information based on the image taken from above the imaging target. However, depending on the imaging target, the state may be determined using an image taken from a horizontal direction at a predetermined distance away from the imaging target, rather than from above the imaging target. For example, the imaging means is fixed to a position (e.g., a central position) of a structure (pole, etc.) protruding upward at a position at a predetermined distance away from the imaging targets (window glass of a building) arranged two-dimensionally in the vertical direction. In this state, the imaging means may be configured to image the imaging target at the predetermined distance in the horizontal direction. The state may also be determined based on the acquired image.

[Alignment position detection system]
The technical ideas in the fifth to seventh embodiments are not limited to a determination object detection system that uses temperature information of an image pickup object, but can also be used as an alignment position detection system, which will be described below.

[Diversion of the fifth embodiment]
The fifth embodiment is for correcting the tilt of an image. This is not limited to temperature detection, but can be used for detecting the arrangement positions of a large number of imaging targets that are assumed to be aligned or arranged regularly.

As in the above embodiment, it is possible to detect individual imaging targets from an image captured of a group of imaging targets arranged in a plurality of arrays. In this case, imaging targets that should be arranged in a regular pattern, such as a two-dimensional array, may deviate from that pattern or may be missing. For example, when the same or identical products are packaged in a two-dimensional array, products may be missing or misaligned. Or, when a car is to be parked at a specified interval according to a specified arrangement rule, the parking position may be misaligned or missing.

By detecting each of the imaging targets using an image captured of a group of multiple imaging targets arranged in a matrix, it is possible to inspect the position of such an array.
In such a configuration, if the image is tilted, it may cause a problem in detecting the imaging target, and an inspection error for misalignment or omission may occur. In response to this, in this alignment position detection system, the image tilt is first detected as in the fifth embodiment. This image may be an infrared image or a visible light image. After the tilt is detected, tilt correction may be performed as in the fifth embodiment. Alternatively, a process may be performed to exclude the image from the inspection target for misalignment or omission as an imaging error as in the fifth embodiment.

In the alignment position detection system, similar to the identification unit of the fourth embodiment, the imaging targets are identified, and the arrangement of the imaging targets is further identified (e.g., 4 rows vertically and 7 columns horizontally). This may be done using an inference model created by deep learning using so-called artificial intelligence (AI), or other methods such as pattern matching may be used. Next, a predetermined control unit or the like compares the arrangement information set by the user with the arrangement of the imaging targets identified from the image, and performs an inspection process for misalignment and omissions.

The alignment position detection system realized by applying the fifth embodiment can have the following features.
[Feature 2]
an image receiving unit that receives an image of an imaging target;
an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image;
a correction unit that calculates a tilt of an imaging direction with respect to the imaging target and performs tilt correction based on the calculated tilt;
a seventh determination unit that determines whether or not there is an imaging object that deviates from a default arrangement rule or an imaging object that has been removed from a default arrangement rule in the corrected image;
An alignment position detection system comprising:

[Diversion of the sixth embodiment]
The sixth embodiment aims to reduce detection errors of the imaging target by utilizing the fact that the size of the imaging target (the size of the imaging target as viewed from at least one direction) is uniform. This is not limited to temperature detection, but can be used to detect the arrangement positions of a large number of imaging targets that are simply assumed to be aligned or arranged in a regular pattern.

As with the alignment position detection system adapted from the fifth embodiment, by detecting individual imaging targets from an image captured of a group of aligned imaging targets, it is possible to check for misalignment or missing alignment of aligned imaging targets.

In such a configuration, if an image target that should be uniform in size is detected as large or small due to a detection error, an error may occur in the inspection. In contrast, in the alignment position detection system that is a repurpose of the sixth embodiment, the size of the area of each detected image target is first calculated as in the sixth embodiment. The image may be an infrared image or a visible light image.

Next, as in the sixth embodiment, the size of the imaged object or a numerical value calculated based on that size is compared with the judgment value to determine whether a detection has been missed.

After determining that a detection error occurred, the user may be notified that a detection error occurred, and the captured object related to the detection error may be highlighted in the image. Alternatively, a process may be executed to exclude the image from inspection targets for misalignment and omission of placement as a detection error.

In addition, in the alignment position detection system, the arrangement of the imaging target is identified by an identification unit, similar to the alignment position detection system adapted from the fifth embodiment described above.

The alignment position detection system realized by applying the sixth embodiment can have the following features.
[Feature 3]
an image receiving unit that receives an image of an imaging target;
an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image;
an eighth determination unit that calculates the size of the imaging target or the group of imaging targets identified by the determination unit, compares the variation in the calculated value with a determination value indicating an acceptable range of the variation, and determines that there is an identification error if the variation is in a predetermined relationship with the determination value.
[Feature 4]
an image receiving unit that receives an image of an imaging target;
an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image;
a ninth determination unit that determines whether or not there is an imaging object that deviates from a default arrangement rule or an imaging object that has been removed from a default arrangement rule in the image;
An alignment position detection system comprising:

[Diversion of the seventh embodiment]
The seventh embodiment utilizes the arrangement rule of the imaging targets to reduce detection errors of the imaging targets. That is, after the imaging targets are detected, it is determined whether there is a deviation in the alignment position of the imaging targets. For example, it is determined whether there is a false detection in the detection of the boundary part of the outer edge of the imaging targets or the area of each imaging target. This is not limited to temperature detection, but can be used to detect the arrangement positions of a large number of imaging targets that are assumed to be aligned or arranged in a regular manner.

As with the alignment position detection system adapted from the fifth embodiment described above, by detecting individual imaging targets from an image captured of a group of multiple aligned imaging targets, it is possible to check for misalignment or missing alignment of aligned imaging targets.

In such a configuration, in an alignment position detection system that is a repurpose of the fourth embodiment, similar to the seventh embodiment, a predetermined arrangement regularity specific to the imaging target is utilized to determine whether there is any deviation from the regularity. For example, a predetermined position in each of multiple imaging targets identified from an image is further identified, and the amount of deviation of the identified position from the arrangement regularity is calculated. If the amount of deviation is large, a detection error is determined. This aims to reduce detection errors of the imaging target.

That is, as in the seventh embodiment, a predetermined position in the area of each detected imaging target is further specified. For example, the predetermined position is "Ymin" or "Ymin _n ". Note that the image may be an infrared image or a visible light image.

Next, as in the seventh embodiment, the variation in the specified position of each identified imaging target is obtained. The variation is judged to be within an acceptable range or an unacceptable range using a judgment value. As in the fourth embodiment, if the variation is within an unacceptable range, it is judged to be a detection error.

After determining that a detection error has occurred, the user may be notified that a detection error has occurred, and the captured object related to the detection error may be highlighted in the image. Alternatively, a process may be executed to exclude the image from inspection targets for misalignment and omission of placement as a detection error.

In addition, in this alignment position detection system, the arrangement of the imaging target is identified by an identification unit, similar to the alignment position detection system adapted from the fifth embodiment described above.

The alignment position detection system realized by applying the seventh embodiment can have the following features.
[Feature 5]
an image receiving unit that receives an image of an imaging target;
an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image;
a tenth determination unit that determines an amount of deviation from an arrangement in any of the imaging targets or the imaging target group based on a positional relationship of the imaging targets or the imaging target group identified by the determination unit, compares the amount of deviation with a determination value that indicates an allowable range of the amount of deviation, and determines an identification error when the amount of deviation is in a predetermined relationship with the determination value;
An alignment position detection system comprising:

The above three alignment position detection systems can be combined as appropriate.

Although the embodiments of the present invention have been described, the above embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, and are included in the scope of the invention and its equivalents as set forth in the claims.

100, 200, 300, State determination system 400, 500, 600, 700, 800, 900, 1000 Determination object detection system 110, 210, 310, 410, 510, 610, 710 Position information receiving unit 120, 220, 320, 420, 520, 620, 720,
820, 920, 1020 Image receiving unit 130, 230, 330, 430, 530, 630, 730,
830, 930, 1030 Specification part 140, 240, 340, 440, 540, 640, 740,
840, 940, 1040 Temperature information acquisition unit 150, 150A, 250, 350 High temperature information acquisition unit 160, 260, 360 Mid temperature range acquisition unit 170, 270, 370 Reference value acquisition unit 180, 280, 380 Judgment target value acquisition unit 190, 290, 390 Judgment unit 391 Suspect target identification unit 410a Setting unit 510a, 1010b Correction unit 610a, 710a, 810a, 910a, 1010a First to fifth judgment units D Aircraft D1 Photography unit D2 Position acquisition unit D3 Gyroscope D4 Gimbal D5 Geomagnetic sensor

Claims (13)

an image receiving unit for receiving an infrared image from the flying object;
An identification unit that identifies a preset region including an imaging target from an infrared image;
a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region;
a high temperature information acquiring unit that calculates a first average value of temperature information included in a first set range including the maximum temperature in the distribution;
a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is a temperature below the first set range and does not overlap with the first set range, from among the distribution;
a reference value acquiring unit that acquires a reference value that is a difference between the first average value and the second average value;
a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value;
A condition determination system comprising: a determination unit that identifies an imaging target that may be at risk of failure based on a determination target value. 2. The condition determination system according to claim 1, wherein the type of the fault can be determined by varying a set range for the first average value. an image receiving unit for receiving an infrared image from the flying object;
An identification unit that identifies a preset region including an imaging target from an infrared image;
a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region;
a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is less than the maximum temperature in the distribution;
a reference value acquisition unit that obtains a reference value that is a difference between the maximum temperature and the second average value;
a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value;
A condition determination system comprising: a determination unit that determines whether an image capture target is abnormal based on a determination target value. the determination target value acquisition unit obtains a standard deviation of the reference value as a variation in the reference value, and further obtains a deviation value for each set region;
4. The state determination system according to claim 1, wherein the determination unit performs abnormality determination by comparing the deviation value with a first threshold value. When the determination unit determines that the determination target value is greater than the first threshold value, the determination unit identifies a suspected fault area that caused the determination.
5. The state determination system according to claim 4, wherein the reference value is calculated again so as not to include the suspected fault area, and the imaging target that may be at fault is identified based on the reference value. The status determination system of claim 5, characterized in that the determination unit repeats the process of recalculating the reference value and identifying the suspected failure area until the suspected failure area is no longer identified or until the area or ratio of the suspected failure area falls below a predetermined value. an image receiving unit for receiving an infrared image from the flying object;
An identification unit that identifies a preset region including an imaging target from an infrared image;
a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region;
a high temperature information acquiring unit that calculates a first average value of temperature information included in a first set range including the maximum temperature in the distribution;
a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is a temperature lower than the first set range and does not overlap with the first set range, from among the distribution;
a reference value acquiring unit that acquires a reference value that is a difference between the first average value and the second average value;
a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value;
a suspect target identification unit that identifies a suspect target that is an imaging target that may be impaired based on a judgment target value and a first threshold value;
a determination unit that classifies the disorder of the imaging target based on a pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, identifies the suspect target again based on a determination target value of the suspect target and a second threshold value;
A state determination system. 8. The state determination system according to claim 7, wherein the fault classification of the determination unit includes at least two of cluster fault, crack, dirt, and cell fault. A second threshold is used when the determination unit determines that the failure classification of the suspected target is other than a cluster failure,
9. The state determination system according to claim 7, wherein the second threshold is set so that the number of suspected targets included in a set area is reduced compared to the first threshold. The state determination system according to any one of claims 7 to 9, characterized in that the determination unit infers a classification of a suspected fault based on deep learning technology. receiving an infrared image from the air vehicle;
Identifying a preset region including an imaging target from the infrared image;
determining a distribution of temperature information based on infrared information for each specified set region;
determining a first average value of temperature information included in a first set range including the maximum temperature in the distribution;
determining a second average value of temperature information included in a second set range that is less than the first set range and does not overlap with the first set range from the distribution;
determining a reference value which is a difference between the first average value and the second average value;
a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value;
and identifying an imaging target that may be at risk of failure based on the judgment target value. receiving an infrared image from the air vehicle;
A process section for identifying a preset region including an imaging target from an infrared image;
determining a distribution of temperature information based on infrared information for each specified set region;
determining a second average value of the temperature information included in a second set range of the distribution that is less than the maximum temperature;
determining a reference value which is a difference between the maximum temperature and the second average value;
a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating the variation in the reference value of each set area within the group and an average value;
and determining whether or not there is an abnormality in the image capture target based on the determination target value. receiving an infrared image from the air vehicle;
Identifying a preset region including an imaging target from the infrared image;
determining a distribution of temperature information based on infrared information for each specified set region;
determining a first average value of temperature information included in a first set range including the maximum temperature in the distribution;
determining a second average value of temperature information included in a second set range that is less than the first set range and does not overlap with the first set range;
determining a reference value which is a difference between the first average value and the second average value;
a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating the variation in the reference value of each set area within the group and an average value;
A step of identifying a suspected object that is an imaging object that may be impaired based on a judgment object value and a first threshold value;
A condition determination method comprising a step of classifying a disorder of the imaged object based on a pattern of high temperature areas in the identified suspect object, and in the case of a specific classification, re-identifying the suspect object based on the judgment target value of the suspect object and a second threshold value.