JP7535484B2 - Anomaly detection device and anomaly detection method - Google Patents

Description

Article 30, Paragraph 2 of the Patent Act applies Date of publication: August 20, 2021 Address of publication: http://gijutsu12/Giken/index. html

The present invention relates to an anomaly detection device and an anomaly detection method.

In conventional automated conveyor manufacturing lines, imaging devices are installed at key locations, and parts are visually inspected or image-based judgments are made on the captured images based on a rule base. As a technology for image judgments on images captured by imaging devices in such manufacturing lines, an inspection system has been disclosed that includes an acquisition unit that acquires an image of the surface of an object, and a judgment unit that performs a color judgment process that judges the color of the surface of the object based on multiple reflection states in which the ratios of specular reflection components and diffuse reflection components differ on the surface of the object, which are obtained from the image of the surface of the object acquired by the acquisition unit (see, for example, Patent Document 1).

International Publication No. 2019/112040

However, with conventional manufacturing line technology, there is an issue with parts such as automotive parts, which have many distinctive curved areas with unevenness, in that it is difficult to detect shape abnormalities using image processing based on simple color judgment processing, etc.

The present invention has been made in consideration of the above-mentioned problems, and aims to provide an anomaly detection device and an anomaly detection method that can accurately detect shape abnormalities in an object.

In order to solve the above-mentioned problems and achieve the object, the anomaly detection device of the present invention comprises an acquisition unit that acquires an image of a specified target object in a part transported on a transport path, the image being captured by an imaging device; an extraction unit that extracts an image portion of the target object from the image acquired by the acquisition unit as an extracted image; a division unit that switches between dividing the extracted image into multiple images or not depending on a setting or on the data size or shape of the extracted image; a determination unit that takes the multiple images as input if the image has been divided by the division unit, and takes the extracted image as input if the image has not been divided by the division unit, and performs an anomaly determination process using a predetermined learning model to obtain as output a determination result indicating the presence or absence of an abnormality and a reliability indicating the degree of reliability of the determination result; and a display control unit that causes a display device to superimpose the determination result indicating an abnormality and the reliability of the determination result on the image acquired by the acquisition unit and display them.

The present invention makes it possible to detect shape abnormalities in an object with high accuracy.

FIG. 1 is a diagram illustrating an example of an overall configuration of an anomaly detection system according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of the anomaly detection device according to the embodiment. FIG. 3 is a diagram illustrating an example of a functional block configuration of the anomaly detection device according to the embodiment. FIG. 4 is a diagram showing an example of a captured image of the roof member and an extracted image including the flange portion. FIG. 5 is a diagram showing an example of a normal image of the flange portion. FIG. 6 is a diagram showing an example of an image of an abnormality in a flange portion. FIG. 7 is a diagram for explaining a state in which the extracted image of the flange portion is divided. FIG. 8 is a diagram showing an example of a state in which the determination result regarding the shape of the flange portion is superimposed on the captured image. FIG. 9 is a flowchart illustrating an example of the flow of an anomaly detection process in the anomaly detection system according to the embodiment.

Below, with reference to Figures 1 to 9, embodiments of an anomaly detection device and an anomaly detection method according to the present invention will be described in detail. Furthermore, the present invention is not limited to the following embodiments, and the components in the following embodiments include those that a person skilled in the art would easily come up with, those that are substantially the same, and those that are within the scope of what is called equivalent. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the following embodiments.

(Overall configuration of anomaly detection system)
1 is a diagram illustrating an example of the overall configuration of an anomaly detection system according to an embodiment. The overall configuration of an anomaly detection system 1 according to the present embodiment will be described with reference to FIG.

The abnormality detection system 1 shown in FIG. 1 is a system for detecting abnormalities in the shape of a specific object in a part shown in an image captured by an imaging device on a conveyor line 3 (conveyor path). As shown in FIG. 1, the abnormality detection system 1 includes a PLC (Programmable Logic Controller) 11, an imaging device 12, an abnormality detection device 10, a display device 13, and a server device 20.

PLC11 is an industrial control device that controls the installation, assembly, and transport operations of parts 30 such as vehicle bodies on the transport line 3. When the parts 30 are transported to a predetermined position on the transport line 3 suitable for imaging by the imaging device 12, the PLC11 transmits a transport signal indicating that the parts have been transported to the predetermined position to the abnormality detection device 10. The transport signal from the PLC11 may be transmitted to the abnormality detection device 10 directly or via a relay component or the like. For example, the PLC11 transmits a transport signal to the abnormality detection device 10 when the parts 30 are placed at a predetermined position suitable for imaging by the imaging device 12 under the control of a multi-joint robot or the like, or when the transport of the parts 30 to the predetermined position is detected by a limiter switch, a proximity sensor, a photoelectric sensor, or the like. The abnormality detection device 10 may receive a transport signal indicating that the parts 30 have been transported to the predetermined position from a limiter switch, a proximity sensor, a photoelectric sensor, or the like installed on the transport line 3.

The imaging device 12 is an imaging device that captures an image of an area including a flange portion of a roof member, which is a specified object in the part 30. The imaging device 12 transmits the captured image to the abnormality detection device 10. In this embodiment, the part 30 is described as a vehicle body, and the specified object is described as a flange portion of a roof member of the vehicle body. Note that the part 30 and the specified object are not limited to these.

The images captured by the imaging device 12 may be still image data or video data.

The abnormality detection device 10 is a device for detecting shape abnormalities in the flange portion of the roof member of the part 30 included in the captured image captured by the imaging device 12. The abnormality detection device 10 is realized, for example, by a single-board computer. Note that the abnormality detection device 10 is not limited to being a single-board computer, and may be an information processing device such as a normal PC (Personal Computer) or a workstation. The abnormality detection device 10 includes an image processing program 101, a pre-processing program 102, and an abnormality determination program 103.

The image processing program 101 is a program that performs image processing such as extracting an image of the flange portion of the roof member from the captured image captured by the imaging device 12. The extraction method in the image processing program 101 may be a method based on any of the following: a cutout process of a predetermined area of the captured image, an extraction process based on a predetermined algorithm using template matching or the like, or an existing extraction process using YOLO (You Only Look Once) or the like that uses a predetermined learning model generated by supervised learning or the like.

The pre-processing program 102 is a program for executing a predetermined pre-processing for the subsequent anomaly determination program 103 on the extracted image extracted by the image processing program 101. Specifically, as the predetermined pre-processing, the pre-processing program 102 divides the extracted image into multiple images in order to reduce the load of the anomaly determination process by the anomaly determination program 103.

The abnormality determination program 103 is a program for performing an abnormality determination process for each divided image divided by the pre-processing program 102 using a predetermined learning model 104. Specifically, the abnormality determination program 103 receives each divided image as an input, and obtains an output of a determination result of normality or abnormality by a discrimination process using the learning model 104. The learning model 104 is a learning model generated by, for example, a convolutional neural network (CNN), which is an example of supervised learning in machine learning, using teacher data that combines normality or abnormality information as a label with an image of the flange portion of the roof member. In addition, in the case of CNN, the divided images divided from the extracted image by the pre-processing program 102 must be divided into images (square images) with the same number of pixels vertically and horizontally. However, it is not necessarily necessary to directly divide the extracted image into square images, and the image size may be changed so that the divided image becomes a square within a range in which the accuracy of the abnormality determination by the learning model 104 can be ensured. The learning algorithm for generating the learning model 104 is not limited to CNN, and may be generated by other learning algorithms. In this embodiment, the description will be given assuming that, as a result of the abnormality determination process using the learning model 104 from an image of the flange portion, a determination result as to whether the shape of the flange portion is normal or abnormal, and a reliability indicating the degree of reliability of the determination result are obtained.

The display device 13 is a display device such as an LCD (Liquid Crystal Display) or an OELD (Organic Electro-Luminescent Display) that displays the results of the abnormality determination program 103.

The server device 20 is a server device that accumulates the judgment results etc. by the abnormality judgment program 103.

(Hardware configuration of anomaly detection device)
2 is a diagram illustrating an example of a hardware configuration of the anomaly detection device 10 according to the embodiment. The hardware configuration of the anomaly detection device 10 according to the embodiment will be described with reference to FIG.

As shown in FIG. 2, the anomaly detection device 10 includes a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, an auxiliary storage device 204, a network I/F 205, an input/output I/F 206, a display output I/F 207, and an imaging I/F 208. These components are connected to each other via a bus line 210 so that they can communicate data with each other.

The CPU 201 is a computing device that controls the operation of the entire anomaly detection device 10. The RAM 202 is a volatile storage device used as a work area for the CPU 201. The ROM 203 is a non-volatile storage device that stores a program that the CPU 201 executes initially, such as an IPL (Initial Program Loader). The CPU 201, RAM 202, and ROM 203 may be configured as a SoC (System on a Chip) mounted on a single board, for example.

The auxiliary storage device 204 is an auxiliary storage device such as a HDD (Hard Disk Drive), SSD (Solid State Drive), eMMC (embedded Multi Media Card) or microSD card that stores the image processing program 101, preprocessing program 102, anomaly determination program 103 and learning model 104 described above, as well as various data used in these programs.

The network I/F 205 is an interface for connecting to a network or the like and communicating data. In the example shown in FIG. 2, the network I/F 205 is connected to the server device 20, and transmits the above-mentioned judgment results in the anomaly detection device 10 to the server device 20. The network I/F 205 is, for example, an interface conforming to the Ethernet (registered trademark) standard that enables communication using the TCP (Transmission Control Protocol)/IP (Internet Protocol) protocol. The network I/F 205 is not limited to wired communication, and may be an interface for wireless communication based on Wi-Fi (registered trademark) or the like.

The input/output I/F 206 is an interface such as a general purpose input/output (GPIO) for communicating with the PLC 11. For example, the input/output I/F 206 receives the above-mentioned conveyance signal from the PLC 11, and transmits a signal indicating the result of the determination made by the abnormality determination program 103 (a signal indicating normality or abnormality) to the PLC 11.

The display output I/F 207 is an interface such as HDMI (registered trademark) (High-Definition Multimedia Interface) for transmitting display data such as the results of the abnormality determination program 103 to the display device 13.

The imaging I/F 208 is an interface such as a Universal Serial Bus (USB) for receiving captured images from the imaging device 12.

The hardware configuration of the anomaly detection device 10 shown in FIG. 1 is an example, and is not limited to this configuration. For example, the device may be equipped with a GPU (Graphics Processing Unit) specialized for graphic processing.

(Functional block configuration of anomaly detection device)
FIG. 3 is a diagram showing an example of a functional block configuration of the anomaly detection device according to the embodiment. FIG. 4 is a diagram showing an example of a captured image of the roof member and an extracted image including the flange portion. FIG. 5 is a diagram showing an example of a normal image of the flange portion. FIG. 6 is a diagram showing an example of an abnormal image of the flange portion. FIG. 7 is a diagram explaining a state in which the extracted image of the flange portion is divided. FIG. 8 is a diagram showing an example of a state in which a determination result regarding the shape of the flange portion of the captured image is superimposed and displayed. The functional block configuration and operation of the anomaly detection device 10 according to the present embodiment will be described with reference to FIGS. 3 to 8.

As shown in FIG. 3, the anomaly detection device 10 has an acquisition unit 301, a process communication unit 302, an extraction unit 303, a preprocessing unit 304 (division unit), an anomaly determination unit 305 (determination unit), a display control unit 306, a generation unit 307, and a memory unit 308.

The acquisition unit 301 is a functional unit that acquires an image captured by the imaging device 12, including a flange portion of a roof member of the part 30, which is a vehicle body, via the imaging I/F 208. Here, FIG. 4(a) shows an image IMG1 captured by the imaging device 12, including a flange portion 31a of the roof member 31 of the part 30, as an example of an image. In addition, when the acquisition unit 301 receives a notification of the receipt of the above-mentioned conveyance signal for the part 30 from the process communication unit 302 described later, the acquisition unit 301 starts acquiring the image captured by the imaging device 12. Note that the imaging operation by the imaging device 12 and the acquisition operation of the image captured by the acquisition unit 301 may be started at the timing when the conveyance signal is received by the process communication unit 302. The acquisition unit 301 outputs the acquired image to the extraction unit 303. The acquisition unit 301 is realized, for example, by the CPU 201 shown in FIG. 2 executing the image processing program 101.

The process communication unit 302 is a functional unit that communicates via the input/output I/F 206. For example, the process communication unit 302 receives a transport signal from the PLC 11 at the timing when the part 30 is transported to a predetermined position on the transport line 3 suitable for imaging by the imaging device 12, and outputs to the acquisition unit 301 a notification that the transport signal has been received. The process communication unit 302 also transmits a signal indicating the result of the determination by the abnormality determination unit 305 (a signal indicating normality or abnormality) to the PLC 11. The process communication unit 302 is realized, for example, by the CPU 201 shown in FIG. 2 executing a predetermined program.

The extraction unit 303 is a functional unit that extracts an image portion of a predetermined object from the captured image acquired by the acquisition unit 301. Specifically, the extraction unit 303 extracts an image of the flange portion of the roof member of the part 30 from the captured image acquired by the acquisition unit 301. For example, FIG. 4B shows an example in which the extraction unit 303 extracts an extracted image EIMG1 as an image including the flange portion 31a of the roof member 31 of the part 30 from the captured image IMG1 shown in FIG. 4A, which is an image received from the acquisition unit 301. The extracted image EIMG1 shown in FIG. 5 shows an example of an extracted image including the flange portion 31a having a normal shape, extracted from the captured image by the extraction unit 303. On the other hand, the extracted image EIMG2 shown in FIG. 6 shows an example of an extracted image including the flange portion 31a having an abnormal shape, extracted from the captured image by the extraction unit 303. FIG. 6 shows an example in which the flange portion 31a shown in the extracted image EIMG2 includes three abnormal portions 31b as abnormal shaped portions.

The extraction method used by the extraction unit 303 may be a method based on any of the following: cutting out a predetermined area of the captured image, extraction processing based on a predetermined algorithm using template matching or the like, or existing extraction processing using YOLO or the like that uses a predetermined learning model generated by supervised learning or the like. The extraction unit 303 outputs the extracted image of the flange portion 31a to the pre-processing unit 304. The extraction unit 303 is realized, for example, by the CPU 201 shown in FIG. 2 executing the image processing program 101.

The pre-processing unit 304 is a functional unit that performs a predetermined pre-processing for the abnormality determination processing by the subsequent abnormality determination unit 305 on the extracted image of the flange portion 31a extracted by the extraction unit 303. Specifically, the pre-processing unit 304 divides the extracted image into multiple images as the predetermined pre-processing, in order to reduce the load of the abnormality determination processing by the abnormality determination unit 305. In the example shown in FIG. 7, the pre-processing unit 304 divides the extracted image EIMG1 extracted by the extraction unit 303 into five divided images SIMG. Note that, when the learning model 104 used by the abnormality determination unit 305 is based on CNN, as described above, the divided images divided from the extracted image need to be divided into images (square images) with the same number of pixels vertically and horizontally. However, it is not necessarily necessary to directly divide the extracted image into square images, and the image size may be changed so that the divided images are square within a range in which the accuracy of the abnormality determination by the learning model 104 can be ensured. The pre-processing unit 304 outputs the divided images to the anomaly determination unit 305.

When the extracted image extracted by the extraction unit 303 is a high-resolution image with a large data size, a large load is placed on the abnormality determination process by the abnormality determination unit 305. However, as described above, the load of the abnormality determination process can be reduced by dividing the extracted image into multiple divided images by the pre-processing unit 304. In order to reduce the load of the abnormality determination process by the abnormality determination unit 305, it is possible to reduce the data size by shrinking the extracted image, but this is not preferable because it reduces the resolution of the shape of the flange portion 31a included in the extracted image and reduces the accuracy of the abnormality determination process. In the example of FIG. 7, the flange portion 31a is horizontally elongated as the predetermined object, so it is divided horizontally, but this is not limited to this, and it may be divided vertically, or horizontally and vertically, depending on the shape and size of the predetermined object included in the extracted image. In addition, it is not necessary to divide the extracted image by the pre-processing unit 304, and the abnormality determination process may be performed directly on the extracted image by the abnormality determination unit 305 depending on the data size and shape of the extracted image. In this case, whether or not the preprocessing unit 304 divides the extracted image may be switched by a setting, or may be switched automatically depending on the data size or shape of the extracted image.

The preprocessing unit 304 is realized, for example, by the CPU 201 shown in FIG. 2 executing the preprocessing program 102.

The abnormality determination unit 305 is a functional unit that executes an abnormality determination process for each divided image divided by the preprocessing unit 304 using the learning model 104 stored in the storage unit 308. Specifically, the abnormality determination unit 305 uses each divided image as an input, and outputs a determination result of whether it is normal or abnormal (a determination result indicating the presence or absence of an abnormality) through a discrimination process using the learning model 104. In this case, the abnormality determination unit 305 performs an abnormality determination process for each divided image, so it is possible to distinguish and output which divided image has an abnormality. In addition, the abnormality determination unit 305 outputs, for each divided image, a determination result of whether the shape of the flange portion 31a is normal or abnormal, and a reliability indicating the degree of reliability of the determination result. The abnormality determination unit 305 outputs the determination result and reliability for each divided image to the display control unit 306 and the server device 20, and outputs the determination result for each divided image to the process communication unit 302. The process communication unit 302 then transmits the judgment result for each divided image to the PLC 11, and if the judgment result for any divided image indicates an abnormality, the PLC 11 executes a predetermined process (such as stopping the line) for the conveying line 3. The abnormality judgment unit 305 is realized, for example, by the CPU 201 shown in FIG. 2 executing the abnormality judgment program 103.

The abnormality determination process by the abnormality determination unit 305 utilizes discrimination processing using the learning model 104, but is not limited to this, and the abnormality determination process may be based on a predetermined algorithm that does not use a learning model, such as template matching. Even in this case, the load of the abnormality determination process can be reduced by dividing the extracted image into multiple divided images by the pre-processing unit 304.

The display control unit 306 is a functional unit that displays the judgment result received from the abnormality judgment unit 305 on the display device 13. For example, the display control unit 306 displays the judgment result by superimposing it on the captured image acquired by the acquisition unit 301. In addition to the judgment result, the reliability of the judgment result may be superimposed on the captured image. In the example shown in FIG. 8, the display control unit 306 displays "anomaly" as a judgment result indicating an abnormality, and superimposed display units SPNa and SPNb including the reliability of the abnormality judgment process, on the abnormal part in the flange part 31a determined by the abnormality judgment process by the abnormality judgment unit 305 in the captured image IMG2 acquired by the acquisition unit 301. In addition, the display control unit 306 may display the display color, etc. of the superimposed display unit in a different manner depending on the numerical value of the reliability, for example.

In this way, by displaying the judgment result on the display device 13, the worker on the conveying line 3 can visually recognize the judgment result. Furthermore, by displaying the judgment result superimposed on the abnormal part of the flange portion 31a in the captured image, the worker can easily confirm the abnormal part of the shape of the flange portion 31a that was determined by the learning model 104. Furthermore, by displaying the reliability of the identification process using the learning model 104 superimposed on the abnormal part of the flange portion 31a in the captured image, the worker can confirm the reliability of the abnormality judgment process based on the learning model 104.

The display control unit 306 is realized, for example, by the CPU 201 shown in FIG. 2 executing a program such as the abnormality determination program 103.

The generating unit 307 is a functional unit that generates the learning model 104 in advance, for example, by CNN, which is an example of supervised learning in machine learning, using teacher data that combines images of the flange portion of the roof member with normal or abnormal information as labels. The generating unit 307 stores the generated learning model 104 in the memory unit 308. The generating unit 307 is realized, for example, by the CPU 201 shown in FIG. 2 executing a learning program that realizes the above-mentioned supervised learning.

The storage unit 308 is a functional unit that stores the image processing program 101, the preprocessing program 102, the abnormality determination program 103, and the learning model 104, as well as various data used in these programs. As shown in FIG. 1 above, the image processing program 101, the preprocessing program 102, and the abnormality determination program 103 using the learning model 104 are separate programs, but this is not limited to the above and may be configured as a single program, for example. The storage unit 308 is realized by the auxiliary storage device 204 shown in FIG. 2.

Note that each functional unit of the anomaly detection device 10 shown in FIG. 3 is a conceptual representation of a function, and is not limited to this configuration. For example, multiple functional units illustrated as independent functional units in FIG. 3 may be configured as a single functional unit. On the other hand, the function of one functional unit in FIG. 3 may be divided into multiple units and configured as multiple functional units.

(Flow of anomaly detection processing in the anomaly detection system)
9 is a flowchart illustrating an example of the flow of an anomaly detection process of the anomaly detection system 1 according to the embodiment. The flow of an anomaly detection process of the anomaly detection system 1 according to the embodiment will be described with reference to FIG.

<Step S11>
First, when the part 30 is conveyed to a predetermined position on the conveying line 3 suitable for imaging by the imaging device 12, and a conveying signal indicating that the part has been conveyed to the predetermined position is received by the process communication unit 302 from the PLC 11 (step S11: Yes), the abnormality detection device 10 returns from the standby mode to the normal operation mode, and the process communication unit 302 outputs to the acquisition unit 301 a notification that the conveying signal has been received, and the process proceeds to step S12. If the part 30 has not been conveyed to the predetermined position (step S11: No), the abnormality detection device 10 waits until the part 30 is conveyed to the predetermined position. Here, the standby mode is, for example, an operation mode in a power saving state.

<Step S12>
When the acquisition unit 301 receives a notification that the transport signal has been received from the process communication unit 302, the acquisition unit 301 starts acquiring the captured image captured by the imaging device 12, and outputs the acquired captured image to the extraction unit 303. Then, the process proceeds to step S13.

<Step S13>
The extraction unit 303 extracts an image portion of the flange portion 31a of the roof member 31 of the part 30, which is a predetermined object, from the captured image acquired by the acquisition unit 301. The extraction unit 303 outputs the extracted image of the flange portion 31a to the pre-processing unit 304. Then, the process proceeds to step S14.

<Step S14>
The pre-processing unit 304 divides the extracted image of the flange portion 31a extracted by the extraction unit 303 into a plurality of images as pre-processing for the abnormality determination process by the subsequent abnormality determination unit 305. The pre-processing unit 304 outputs the divided images to the abnormality determination unit 305. Then, the process proceeds to step S15.

<Step S15>
The abnormality determination unit 305 executes an abnormality determination process for each divided image divided by the preprocessing unit 304, using the learning model 104 stored in the storage unit 308. Specifically, the abnormality determination unit 305 receives each divided image as an input, and outputs a determination result of whether the image is normal or abnormal through a discrimination process using the learning model 104. The abnormality determination unit 305 also outputs, for each divided image, a determination result of whether the shape of the flange portion 31a is normal or abnormal, and a reliability indicating the degree of reliability of the determination result. The abnormality determination unit 305 outputs the determination result and the reliability for each divided image to the display control unit 306 and the server device 20, and outputs the determination result for each divided image to the process communication unit 302. Then, the process proceeds to step S16.

<Step S16>
The display control unit 306 superimposes and displays the determination result on the captured image acquired by the acquisition unit 301. Then, the process proceeds to step S17.

<Step S17>
The process communication unit 302 transmits the judgment result for each divided image to the PLC 11. If the judgment result for any divided image indicates an abnormality, the PLC 11 executes a predetermined process (such as stopping the line) for the conveying line 3. Then, the abnormality detection device 10 transitions to a standby mode. Then, the abnormality detection process ends.

As described above, in the anomaly detection device 10 according to this embodiment, the acquisition unit 301 acquires an image of the flange portion 31a of the roof member 31 in the part 30 transported on the transport line 3, captured by the imaging device 12, the pre-processing unit 304 divides the image portion of the flange portion 31a in the captured image acquired by the acquisition unit 301 into multiple images, and the anomaly determination unit 305 receives the multiple divided images divided by the pre-processing unit 304 as input, and performs an anomaly determination process using the learning model 104 to obtain as output a determination result indicating the presence or absence of an anomaly. This makes it possible to accurately detect shape anomalies in the flange portion 31a as a specified object. In addition, because the image portion of the flange portion 31a in the captured image is divided into multiple divided images, the load of the anomaly determination process performed by the anomaly determination unit 305 can be reduced.

In addition, in the anomaly detection device 10, the extraction unit 303 extracts an image portion of the flange portion 31a from the captured image acquired by the acquisition unit 301 as an extracted image, and the pre-processing unit 304 divides the extracted image extracted by the extraction unit 303 into multiple images. As a result, the image portion including the flange portion 31a as the specified object is extracted and then the anomaly determination process is performed, thereby improving the accuracy of anomaly detection for the flange portion 31a.

In addition, in the abnormality detection device 10, the process communication unit 302 receives a transport signal indicating that the part 30 has been transported to a predetermined position on the transport line 3, and the acquisition unit 301 starts acquiring the captured image captured by the imaging device 12 when the transport signal is received by the process communication unit 302. As a result, the image acquisition process starts from the point in time when the transport signal is received, and a series of processes up to the abnormality determination process are executed, so that the processing load of the abnormality detection device 10 can be reduced.

In addition, in the abnormality detection device 10, when a conveyance signal is received by the process communication unit 302, the device is returned from standby mode, and when the abnormality determination process by the abnormality determination unit 305 is completed, the device is switched to standby mode. This makes it possible to employ a single-board computer or the like as the abnormality detection device 10, thereby reducing the processing load.

1 Anomaly detection system 3 Conveyor line 10 Anomaly detection device 11 PLC
REFERENCE SIGNS LIST 12 Imaging device 13 Display device 20 Server device 30 Part 31 Roof member 31a Flange portion 31b Abnormal portion 101 Image processing program 102 Pre-processing program 103 Abnormality determination program 104 Learning model 301 Acquisition unit 302 Process communication unit 303 Extraction unit 304 Pre-processing unit 305 Abnormality determination unit 306 Display control unit 307 Generation unit 308 Storage unit

Claims (5)

an acquisition unit that acquires an image of a predetermined object in the part being transported on the transport path, the image being captured by an imaging device;
an extraction unit that extracts an image portion of the object from the captured image acquired by the acquisition unit as an extracted image;
a division unit that switches between dividing the extracted image into a plurality of images and executing the division in accordance with a setting or a data size or shape of the extracted image ;
a determination unit that receives the plurality of images as an input when the images have been divided by the division unit, and receives the extracted image as an input when the images have not been divided by the division unit, and performs an anomaly determination process using a predetermined learning model to obtain, as an output, a determination result indicating the presence or absence of an anomaly and a reliability indicating the degree of reliability of the determination result;
a display control unit that causes a display device to display the determination result indicating an abnormality and the reliability of the determination result superimposed on the captured image acquired by the acquisition unit; and
An anomaly detection device equipped with the above. a receiving unit for receiving a signal indicating that the part has been conveyed to a predetermined position on the conveying path,
The anomaly detection device according to claim 1 , wherein the acquisition unit starts acquiring the image captured by the imaging device when the signal is received by the receiving unit. The anomaly detection device according to claim 2 , wherein the device is caused to return from a standby mode when the signal is received by the receiving unit, and is caused to transition to the standby mode when the anomaly determination process by the determining unit is completed. The anomaly detection device according to any one of claims 1 to 3 , further comprising a generation unit that generates the learning model through supervised learning using teacher data that combines an image of a predetermined object in the part and a label indicating whether the object is normal or abnormal. An acquisition step of acquiring an image of a predetermined object in the part transported on the transport path, the image being captured by an imaging device;
an extraction step of extracting an image portion of the object from the acquired captured image as an extracted image;
a dividing step of switching between dividing the extracted image into a plurality of images and executing the dividing step according to a setting or according to a data size or shape of the extracted image ;
a determination step of obtaining , as an output, a determination result indicating the presence or absence of an abnormality and a reliability indicating the degree of reliability of the determination result by performing an abnormality determination process using a predetermined learning model, with the plurality of images as an input if the images are divided, or with the extracted image as an input if the images are not divided ;
a display control step of displaying, on a display device, the determination result indicating an abnormality and the reliability of the determination result in a superimposed manner on the acquired captured image;
The anomaly detection method includes the steps of:

Images