JP7734604B2 - Appearance inspection support device, appearance inspection support method, and appearance inspection support program - Google Patents

Description

This disclosure relates to an appearance inspection support device, an appearance inspection support method, and an appearance inspection support program.

Technologies for supporting visual inspections to check for product defects are known. For example, Patent Document 1 discloses a method for supporting visual inspections by using a visual inspection device that irradiates light onto a part to be inspected and checks for defects based on the light reflected from the part. In the technology disclosed in Patent Document 1, a specular reflection light illumination device and an imaging device are installed so that when light is irradiated onto a normal part of the part to be inspected, specular reflection light enters the imaging device, and a diffuse reflection light illumination device is installed so that when light is irradiated onto a defective part of the part to be inspected, diffuse reflection light enters the imaging device.

Japanese Patent Application Laid-Open No. 2007-271510

When imaging and lighting devices are installed around the production line, the installation space taken up increases the length of the entire production line. Visual inspection remains important as a form of visual inspection. However, the quality of visual inspection is highly dependent on the individual, and inexperienced workers are prone to poor inspection quality.

This disclosure was made in consideration of the above issues, and aims to enable high-quality inspection while preventing the overall length of the production line from increasing.

To achieve the above objective, the visual inspection support device disclosed herein supports visual inspections performed on products under inspection. The visual inspection support device includes an input unit, an analysis unit, an output unit, and a first flying object. The input unit inputs defect location data indicating defect locations of defective products among the products under inspection. The analysis unit analyzes the defect location data for multiple defective products and sets pre-set areas where the number of defects or the defect rate is above a standard as specific areas where defects are likely to occur. The output unit outputs information indicating the areas where the number of defects or the defect rate is above a standard. The first flying object has a projector to which information indicating the areas analyzed as having the number of defects or the defect rate above a standard is input from the output unit. The projector projects an image indicating the pre-set area onto the product under inspection based on the information input from the output unit.

According to the present disclosure, defective location data is analyzed, and areas where the number or rate of defects is above a certain standard, i.e., areas where defects are likely to occur, are identified. An image showing this area is then projected onto the product being inspected. Inspectors can refer to this image while inspecting the product being inspected. This reduces the dependency of inspection quality on individuals, enabling high-quality inspections. Furthermore, by placing the projector on the flying vehicle, there is no need to reserve an area for the projector at the inspection location, which helps prevent the overall production line, including the visual inspection equipment, from becoming too long.

FIG. 1 is a diagram showing an overview of a visual inspection support device according to a first embodiment; FIG. 1 is a diagram showing an example of a specific region according to the first embodiment; FIG. 10 is a diagram showing an example of a warning image displayed on an inspected product according to the first embodiment; FIG. 1 is a block diagram showing a configuration of a visual inspection support device according to a first embodiment. A block diagram showing the functions of a control device according to a first embodiment. FIG. 1 is a diagram showing an example of a production schedule storage unit according to the first embodiment; FIG. 1 is a diagram illustrating an example of a device master storage unit according to the first embodiment. FIG. 1 is a diagram showing an example of a non-defective product image related data storage unit according to the first embodiment; FIG. 1 is a diagram showing an example of a defective product image related data storage unit according to the first embodiment; FIG. 1 is a diagram showing an example of an output preparation data storage unit according to the first embodiment; 1 is a flowchart showing input processing according to the first embodiment; FIG. 1 is a diagram showing an example of a defective product image according to the first embodiment; FIG. 1 is a diagram showing an example of a non-defective product image according to the first embodiment; Flow diagram showing analysis processing according to the first embodiment Flow diagram showing output processing according to the first embodiment FIG. 4 is a schematic diagram showing an example of a projection angle. FIG. 4 is a schematic diagram showing an example of a projection angle. FIG. 4 is a schematic diagram showing an example of a projection angle. FIG. 10 is a schematic diagram showing the arrangement of drones according to a modified example of the first embodiment. Flow diagram showing analysis processing according to the second embodiment FIG. 10 is a diagram showing an example of an output preparation data storage unit according to the second embodiment; A block diagram showing the functions of a control device according to a third embodiment. FIG. 10 is a diagram showing an example of a schedule storage unit according to the third embodiment. FIG. 13 is a diagram illustrating an example of a worker master storage unit according to the third embodiment. Flow diagram showing input processing according to the third embodiment FIG. 10 is a diagram showing an example of a defective product image related data storage unit according to the third embodiment; Flow diagram showing output processing according to the third embodiment FIG. 10 is a diagram showing an example of an output preparation data storage unit according to the third embodiment; FIG. 10 is a diagram showing an example of an attention-calling image according to the third embodiment; FIG. 10 is a diagram showing another example of an attention-calling image according to the third embodiment; Flow diagram showing input processing according to the fourth embodiment FIG. 13 is a diagram showing an example of a defective product image related data storage unit according to the fourth embodiment; FIG. 13 is a diagram showing an example of an output preparation data storage unit according to the fourth embodiment. Flow diagram showing output processing according to the fourth embodiment A block diagram showing the functions of a control device according to a fifth embodiment. Flow diagram showing learning processing according to the fifth embodiment Flow diagram showing inference processing according to the fifth embodiment FIG. 13 is a diagram showing an example of an improvement measure data storage unit according to the fifth embodiment. FIG. 13 is a diagram showing another example of the improvement measure data storage unit according to the fifth embodiment.

Embodiments of the visual inspection support device, visual inspection support method, and visual inspection support program according to the present disclosure will be described below with reference to the drawings.

[First Embodiment]
An overview of a visual inspection support device 100 according to this embodiment will be described with reference to FIG. 1. As shown in FIG. 1, the visual inspection support device 100 supports visual inspections of products PR produced in a factory. During the visual inspection, a worker P visually determines whether the product PR has defects such as scratches or uneven color. During the visual inspection, a projector 2 mounted on a first drone DR1 projects an image indicating areas of the product PR that are likely to have defects, based on information indicating areas of the product PR that are likely to have defects. Hereinafter, the product PR that is the subject of the visual inspection will be referred to as the "inspected product W," and an inspected product W in which a defect is found during the visual inspection will be referred to as the "defective product D." The product PR may be, for example, an electrical appliance such as a dehumidifier, rice cooker, or vacuum cleaner.

The visual inspection support device 100 statistically analyzes areas containing defective parts to identify areas where the probability of defects occurring is above a certain standard, and presents these identified areas to the worker P. Hereinafter, the location of a defective part on a defective product D will be referred to as the "defective position," and the area containing the defective part will be referred to as the "defective area." Additionally, an area that is highly prone to defects occurring, i.e., an area identified as an area with a high probability of defects occurring, may also be referred to as the "specific area."

The visual inspection support device 100 includes a first drone DR1 having a projector 2, a second drone DR2 having a camera 1, and a control device 10 that identifies areas where defects are likely to occur. The first drone DR1 is an example of a first aerial vehicle, and the second drone DR2 is an example of a second aerial vehicle. The first drone DR1 is, for example, a battery-powered drone equipped with a projector 2. The second drone DR2 is, for example, a battery-powered drone equipped with a camera 1. Hereinafter, the first drone DR1 and the second drone DR2 will be simply referred to as "drone DR1" and "drone DR2," respectively.

Drone DR1 is positioned above the visual inspection area SV, at a location where projector 2 can project an image onto the inspected product W. Note that "drone DR1 is positioned" means "drone DR1 is hovering at the adjusted position."

Based on information from the control device 10 identifying areas where defects are likely to occur, the projector 2 projects an image showing areas where defects are likely to occur onto the inspected product W. The worker P focuses on the areas indicated by the projected image and inspects the inspected product W for defects. In the following, the external shape of the inspected product W is assumed to be a rectangular parallelepiped. Furthermore, each of the six sides of the inspected product W is assumed to be assigned to a different appearance inspection site SV and to be handled by a different worker P. One projector 2 is placed at each appearance inspection site SV. In other words, the number of drones DR1 installed corresponds to the number of appearance inspection sites SV, six in this embodiment.

Drone DR2 is positioned above the defective product storage area SD, in a location where camera 1 can photograph defective product D. Note that "drone DR2 is positioned" means "drone DR2 is hovering at the adjusted position."

When worker P determines that the inspected product W is a defective product D, he moves the defective product D from the visual inspection area SV where the visual inspection is carried out to the defective product storage area SD. As mentioned above, in this example, the inspected product W is a rectangular parallelepiped, and a different visual inspection area SV is installed for each of the six sides of the inspected product W, with a defective product storage area SD installed at each visual inspection area SV. One camera 1 is placed at each defective product storage area SD. Therefore, six drones DR2 are installed, corresponding to the number of defective product storage areas SD.

Every time worker P moves defective product D to the defective product storage area SD, he or she has camera 1 of drone DR2 take a photo of defective product D. Hereinafter, an image showing defective product D will be referred to as a "defective product image."

Camera 1 captures an image of defective product D and outputs the image data to control device 10. Note that the data on the defective product image is an example of "defect location data."

The control device 10 may be configured, for example, as a personal computer, a tablet terminal, or a PLC (Programmable Logic Controller).

The control device 10 analyzes the location of defective areas based on the image data output from the camera 1 and identifies areas where defects tend to occur, i.e., areas with a high probability of occurring.

Areas where defects are likely to occur are areas where the number or percentage of defects occurring in a period preset by the user is equal to or greater than the first defect threshold T1. The first defect threshold T1 is preset by the user of the visual inspection support device 100 as a value such as "3," "5," "10," "3%," or "10%." Users include production managers, production engineers, and defect analysts. Note that the first defect threshold T1 is an example of a standard.

When the control device 10 identifies an area that is highly prone to defects as a specific area, it generates image data indicating the specific area and outputs it to the projector 2 of the drone DR1.

As mentioned above, the number of drones DR1 installed corresponds to the number of visual inspection sites SV; in this embodiment, six drones DR1 are installed. Based on the input image data, the projector 2 projects an image indicating areas where defects are likely to occur toward the inspected product W, thereby displaying the image on the surface of the inspected product W. Hereinafter, the act of the projector 2 projecting an image toward the inspected product W and displaying an image on the surface of the inspected product W may be simply referred to as "displaying an image." The image displayed on the inspected product W is also referred to as a "warning image." The data of the warning image is an example of "information indicating areas where defects are likely to occur."

The product W to be inspected is placed in a predetermined orientation at a predetermined position on the inspection table at each visual inspection area SV. More specifically, the product W to be inspected is placed in a predetermined orientation at a predetermined position on the inspection table at the visual inspection area SV in a form that allows the surface of the inspection target to display a warning image. Drone DR2 is adjusted and placed in a position that allows the warning image to be displayed on the surface of the product W to be inspected using projector 2.

The attention drawing image GC displayed on the inspected product W will be described with reference to Figures 2A and 2B. Figure 2A is a diagram showing an example of a specific area R1 identified by the control device 10 as an area with a high tendency for defects to occur, and Figure 2B is a diagram showing an example of an attention drawing image GC displayed on the inspected product W. Note that the X-axis and Y-axis shown in Figures 2A and 2B are coordinate axes on the inspection table on which the inspected product W is placed. The X-axis is the left-right direction of the inspected product W as seen by the operator P, and the Y-axis is the front-to-back direction of the inspected product W as seen by the operator P, and is perpendicular to the X-axis.

As shown in Figure 2A, when specific region R1 is near a corner on the +X side and the -Y side of product PR, projector 2, described with reference to Figure 1, projects image GC1 that surrounds the corner on the +X side and the -Y side of the inspected product W, as shown in Figure 2B. As a result, an attention drawing image GC including image GC1 pointing to specific region R1 is displayed on the inspected product W. Note that if there are multiple specific regions R1, the attention drawing image GC includes images surrounding multiple specific regions R1.

Next, the configuration of the control device 10 shown in Figure 1 will be described in detail with reference to Figures 3 and 4.

As shown in FIG. 3, the control device 10 further includes an operation unit 11 that accepts user operations, a display 12 that displays images, an input unit 13 that receives image data from each camera 1 of multiple drones DR2, an output unit 14 that outputs data for a warning image GC, a processor 15 that executes various processes, a main memory device 16a used as a work area for the processor 15, and an auxiliary memory device 16b that stores various data such as programs. The main memory device 16a includes RAM (Random Access Memory). The auxiliary memory device 16b includes non-volatile memory such as flash memory, HDD (Hard Disk Drive), and SSD (Solid State Drive).

When the operation unit 11 receives an operation from the user, it outputs a signal corresponding to that operation to the processor 15. An example of an operation by the user is an operation to adjust the position of the drone DR1 to a position where an image indicating an area with a high tendency for defects is projected from the projector 2 onto the inspected product W. Note that an operation by the user is not limited to an operation to adjust the position, but also includes any other type of operation by the user. The operation unit 11 is, for example, a touch sensor, a keyboard, a mouse, etc.

The visual inspection support device 100 is equipped with a transmission unit (not shown). The transmission unit transmits control signals to the drone DR1 that instruct it to operate in accordance with operations on the operation unit 11. The drone DR1 operates in accordance with the control signals from the transmission unit.

The display 12 displays an image based on a signal output from the processor 15. The display 12 may be, for example, a liquid crystal display or an organic electroluminescence (EL) display.

The input unit 13 is connected to each camera 1 of the multiple drones DR2, and receives the image data output from each camera 1. The input unit 13 may be any device capable of receiving image data, such as a video card or graphics card. The input unit 13 may also be a communications device capable of communicating in accordance with communications standards such as Wi-Fi (registered trademark), Ethernet (registered trademark), or USB (Universal Serial Bus).

The output unit 14 connects to each of the projectors 2 of the multiple drones DR1 and outputs data of the attention warning image GC to each projector 2. The output unit 14 may be any device capable of outputting data of the attention warning image GC to the projector 2, such as a video card or graphics card. The output unit 14 may also be a communications device that communicates in accordance with communications standards such as Wi-Fi (registered trademark), Ethernet (registered trademark), or USB.

The processor 15 is an MPU (Micro Processing Unit), CPU (Central Processing Unit), or the like. The processor 15 reads the appearance inspection support program stored in the auxiliary storage device 16b into the main storage device 16a and executes it.

As shown in FIG. 4, the auxiliary storage device 16b includes a data storage unit 16. The data storage unit 16 stores data on multiple defective products D. The data on multiple defective products D is used when analyzing trends in areas where defects occur.

The data storage unit 16 includes an image data storage unit 160 that stores data on defective product images.

The image data storage unit 160 also stores data on non-defective product images previously captured by the user. The non-defective product image data is used when identifying defective areas. After capturing a non-defective product image of the inspected product W, the user operates the operation unit 11 to input the non-defective product image data into the control device 10 via the input unit 13, and stores the data in the image data storage unit 160. Note that the non-defective product image includes an image of the entire surface of the inspected product W for each model.

The data storage unit 16 also includes a production schedule storage unit 161 that stores data indicating the factory's production schedule, an equipment master storage unit 162 that stores data related to the equipment equipped in the visual inspection support device 100, a good product image related data storage unit 163 that stores data related to good product images, a defective product image related data storage unit 164 that stores data related to defective product images captured by the camera 1, and an output preparation data storage unit 165 that stores output preparation data required for output to the projector 2 shown in Figure 1.

As shown in FIG. 5, the production schedule storage unit 161 associates and stores model data 161a indicating the model of product PR produced in the factory with production schedule data 161b indicating the schedule for producing that model of product PR. The production schedule data 161b includes production start date and time data 161c indicating the date and time when production starts and production end date and time data 161d indicating the date and time when production ends. The data stored in the production schedule storage unit 161 is registered in advance by the user.

As shown in FIG. 6, the equipment master memory unit 162 shown in FIG. 4 associates and stores equipment name data 162a indicating the equipment name of the equipment equipped in the visual inspection support device 100, equipment identification name data 162b indicating the identification name of the equipment, installation location data 162c indicating the location where the equipment is installed, and inspection content data 162d indicating the content of the visual inspection corresponding to the location where the equipment is installed. The equipment identification name data 162b is identification data such as the equipment's MAC address (Media Access Control address) or IP address (Internet Protocol address). The installation location data 162c indicates the installation location according to the function of the equipment. Examples of installation locations include the defective product storage area SD and the visual inspection area SV. The inspection content data 162d indicates the surface that is the target of the visual inspection for which the equipment is used at the installation location. The data stored in the device master storage unit 162 is registered in advance by the user.

As shown in FIG. 7, the good-quality image related data storage unit 163 shown in FIG. 4 stores, in association with one another, good-quality image identification name data 163a that identifies a good-quality image, model data 163b that indicates the model of the good-quality product indicated by the good-quality image, and photographed portion data 163c that indicates the photographed portion of the good-quality image. The data related to good-quality images stored in the good-quality image related data storage unit 163 is registered in advance in the good-quality image related data storage unit 163 by the user.

As shown in FIG. 8, the defective product image related data storage unit 164 shown in FIG. 4 associates and stores defective product image identification name data 164a that identifies the defective product image, model data 164b that indicates the model of the defective product D indicated by the defective product image, photographed portion data 164c that indicates the photographed portion of the defective product D, discovery date and time data 164d that indicates the date and time the defect was discovered, and defective area data 164e that indicates the defective area. The defective product image related data is stored in the defective product image related data storage unit 164 by the input processing unit 151 shown in FIG. 4 executing input processing.

As shown in FIG. 9, the output preparation data storage unit 165 shown in FIG. 4 stores, in association with each other, model data 165a indicating the model of the inspected product W, photographed portion data 165b indicating the portion of the inspected product W where the attention alert image GC is displayed, and specific area data 165c indicating the specific area R1 indicated by the attention alert image GC.

As shown in Figure 4, the processor 15 shown in Figure 3 functions as an input processing unit 151 that performs input processing, an analysis unit 152 that performs analysis processing, and an output processing unit 153 that performs output processing.

When image data is input from camera 1 of drone DR2 via input unit 13, input processing unit 151 acquires the data necessary to analyze the defective area and performs input processing to store it in data storage unit 16.

The analysis unit 152 receives image data from camera 1 of drone DR2 and stores data for analyzing the tendency of defective areas in the image data storage unit 160. Based on the data stored in the image data storage unit 160, it analyzes the tendency of defective areas and performs an analysis process to identify areas that are highly prone to defects.

The output processing unit 153 generates data for a warning image GC that indicates areas identified by the analysis unit 152 that are likely to have defects, and performs output processing to output the data for the warning image GC to the projector 2.

The process of projecting the warning image GC onto the inspected product W is realized by i) an input process of inputting and accumulating defect data of defective products D, ii) an analysis process of analyzing the accumulated defect data and identifying specific areas where defects tend to occur above a standard, iii) a process of outputting information indicating the specific areas to the projector 2 based on the analysis, and iv) a process of the projector 2 projecting the warning image GC onto the inspected product W. Note that the input process is an example of an input step, the analysis process is an example of an analysis step, and the output process is an example of an output step.
The input process, analysis process, and output process will be described below in order with reference to FIGS.

(Input processing)
When the product W to be inspected is transported to the appearance inspection site SV that the operator P is in charge of, the operator P places the product W on the inspection table. At this time, the operator P places the product W to be inspected in a predetermined position on the inspection table. More specifically, the operator P places the product W to be inspected in a predetermined position on the inspection table in a predetermined orientation. The operator P places the product W to be inspected on the inspection table with the inspection portion at the appearance inspection site SV, in embodiment 1, one of the six faces, facing upward.

Next, worker P visually inspects the surface of the inspected product W. At this time, projector 2 of drone DR1 located at the visual inspection site SV projects a warning image GC toward the inspected product W. This indicates areas on the surface of the inspected product W that are likely to have defects, drawing the worker P's attention.

When a worker P finds a defect, he or she moves the inspected product W to the defective product storage area SD as a defective product D. In this embodiment, the worker P places the inspected product W on the workbench in the defective product storage area SD with the side of the inspected product W containing the defect facing up.

Next, worker P operates operation unit 11 to input an instruction to control device 10 to have camera 1 photograph defective product D. In response to the signal from operation unit 11, processor 15 causes camera 1 to photograph the surface of defective product D that contains the defective part. Camera 1 generates imaging data and property data and outputs them to control device 10.

The processor 15, functioning as the input processing unit 151, starts the input processing shown in FIG. 10. The input processing unit 151 acquires the supplied defective product image data via the input unit 13 and stores the acquired defective product image data in the image data storage unit 160 shown in FIG. 4 (step S102).

Next, the input processing unit 151 acquires data identifying the defective product image (step S104). The data identifying the defective product image is included in the image data input from the camera 1, and is data indicating, for example, the file name and save location of the defective product image.

Next, the input processing unit 151 identifies the model of defective product D based on the image capture time included in the image data (step S106). More specifically, the input processing unit 151 uses the image capture time data when the defective product image was captured as a search condition to search the production schedule storage unit 161 shown in FIG. 5 for records whose image capture time data is after the date and time indicated by the production start date and time data 161c and before the date and time indicated by the production end date and time data 161d. When the input processing unit 151 finds a record that matches the search condition, it acquires the model data 161a of that record to identify the model of defective product D.

Next, the input processing unit 151 identifies the photographed surface of defective product D (step S108 in Figure 10). Hereinafter, the photographed surface will be referred to as the "photographed portion."

In more detail, the input processing unit 151 uses data indicating the identification name of camera 1 contained in the image data as a search condition to search the equipment master storage unit 162 shown in FIG. 6 for records in which the data indicating the identification name of camera 1 matches the equipment identification name data 162b. When the input processing unit 151 finds a record that matches the search condition, it acquires the inspection content data 162d of that record to identify the photographed portion of defective product D.

Next, the input processing unit 151 identifies the defective area indicated by the defective product image (step S110). More specifically, the input processing unit 151 searches the data stored in the non-defective product image related data storage unit 163 shown in FIG. 7 for a record whose model data 163b matches the model data 161a acquired in step S106 and whose captured portion data 163c matches the inspection content data 162d acquired in step S108. When the input processing unit 151 finds a record that matches the search criteria, it uses the non-defective product image identification name data 163a of the found record as a search condition to acquire non-defective product image data from the data stored in the image data storage unit 160. When the input processing unit 151 acquires the non-defective product image data, it identifies the defective area by comparing the non-defective product image with the non-defective product image.

The method for identifying defective areas will now be described with reference to Figures 11A and 11B. Figure 11A shows an example of a defective product image GD, and Figure 11B shows an example of a non-defective product image GR.

As shown in Figures 11A and 11B, the input processing unit 151 divides each of the defective product image GD and the non-defective product image GR into nine sections SC. In Figures 11A and 11B, for ease of understanding, the boundaries of each section SC are indicated by two-dot chain lines. Note that the number of sections SC to be divided is not limited to nine and can be set in advance by the user.

In the following, the section SC on the -X side and +Y side will be referred to as "SECTION 1," and the sections from "SECTION 1" to the +X side will be referred to as "SECTION 2" and "SECTION 3." The section SC adjacent to "SECTION 1" on the -Y side will be referred to as "SECTION 4," and the sections from "SECTION 4" to the +X side will be referred to as "SECTION 5" and "SECTION 6." The section SC adjacent to "SECTION 4" on the -Y side will be referred to as "SECTION 7," and the sections from "SECTION 7" to the +X side will be referred to as "SECTION 8" and "SECTION 9."

For example, when the number of sections SC to be divided is set, the data indicating the position of each section SC is calculated by the processor 15 and stored in the auxiliary storage device 16b as coordinate data corresponding to each section SC.

The input processing unit 151 matches the defective product image GD with the good product image GR for each section SC, and identifies sections SC where the defective product image GD and the good product image GR differ, i.e., areas containing defective parts, as defective areas.

In the example shown in FIG. 11A, "SECTION 9" of the defective product image GD includes an image GS showing a scratch. Therefore, the input processing unit 151 determines that the image of "SECTION 9" of the defective product image GD does not match the image of "SECTION 9" of the non-defective product image GR, and identifies "SECTION 9" as a defective area. The input processing unit 151 generates data indicating the identified defective area.

When the input processing unit 151 identifies a defective area, it associates the data acquired in steps S104 to S110 of FIG. 10 and stores the data in the defective product image related data storage unit 164 shown in FIG. 8 (step S112). Specifically, the input processing unit 151 stores the data acquired in steps S104 to S110 in the defective product image related data storage unit 164 as "defective product image identification name data 164a," "model data 164b," "photographed portion data 164c," and "defective area data 164e," respectively. The input processing unit 151 also stores the photography time data indicating the time the defective product image GD was photographed in the defective product image related data storage unit 164 as "discovery date and time data 164d."

The input processing unit 151 then completes the input processing.

Next, the analysis process will be explained with reference to Figure 12. The analysis process is executed, for example, every time the factory closes. The timing at which the analysis process is executed is set in advance by the user.

When the preset start time arrives, the processor 15, as the analysis unit 152, starts the analysis process shown in Figure 12.

The analysis unit 152 tallyes data related to images of multiple defective products D (defective product images GD) stored in the defective product image-related data storage unit 164 shown in FIG. 8 by model, photographed part, and defective area, and obtains the number of tallied records as the number of defects (step S202). The number of defects is the number of cases in which defects occurred. The analysis unit 152 tallyes the number of defects based on data related to defective product images GD stored over a preset period, such as one day or one week.

The analysis unit 152 identifies areas where defects are likely to occur based on the number of defects (step S204). The analysis unit 152 identifies defect areas where the number of defects, which is the result of the aggregation in step S202, is equal to or exceeds the first defect threshold T1 as areas where defects are likely to occur, i.e., specific areas R1. Note that the first defect threshold T1 is set in advance by the user as a value according to the period over which the number of defects is aggregated. Note that when identifying areas where defects are likely to occur, any type of criterion may be used, not limited to the number of defects. For example, the defect occurrence rate may be used.

For example, if the period set by the user is one day and the first defect threshold T1 is set to "3," when the data in the defective product image-related data storage unit 164 shown in FIG. 8 is aggregated by model, photographed part, and defective area, the number of records for the model "Rice Cooker," photographed part "Top Surface," and defective area "SECTION 9" for the one day of "August 26, 2020" is seven, which is more than three. Therefore, the analysis unit 152 identifies the model "Rice Cooker" and the defective area "SECTION 9" of the photographed part "Top Surface" as areas with a high tendency for defects to occur, i.e., specific area R1.

Once the analysis unit 152 has identified the specific region R1, it stores the data necessary for output by the output processing unit 153 in the output preparation data storage unit 165 shown in FIG. 9 (step S206). In more detail, the analysis unit 152 stores the model data 164b, photographed portion data 164c, and defective region data 164e of the record extracted from the defective product image related data storage unit 164 in the output preparation data storage unit 165 as "model data 165a," "photographed portion data 165b," and "specific region data 165c."

The analysis unit 152 then completes the analysis process.

The processor 15 performs output processing as the output processing unit 153, for example, periodically, during the period in which the visual appearance inspection is being performed.

As shown in FIG. 13, the output processing unit 153 identifies the model of the production target produced in the last n hours in order to identify the model of the inspected product W (step S302). More specifically, the output processing unit 153 acquires data indicating the current time from the timer. The output processing unit 153 searches the data stored in the production schedule storage unit 161 shown in FIG. 5 using the acquired current time data as a search condition. When the output processing unit 153 finds a record that matches the search condition, it identifies the model indicated by the model data 161a of that record as the model that was produced in the factory in the last n hours and is the target of visual inspection.

Next, the output processing unit 153 specifies, for each projector 2, the content of the appearance inspection to be performed at the appearance inspection site SV where the projector 2 is installed (step S304). Specifically, the output processing unit 153 searches the device master storage unit 162 shown in FIG. 6 using data indicating the identification name of each projector 2 as a search condition. When the output processing unit 153 finds a record that matches the search condition, it specifies the inspection content indicated by the inspection content data 162d of that record as the appearance inspection to be performed at the appearance inspection site SV where the projector 2 is installed. Note that the data indicating the identification name of the projector 2 is input by the user via the operation unit 11, for example.

Next, the output processing unit 153 searches the data stored in the output preparation data storage unit 165 shown in FIG. 9 for a record in which the model data 161a acquired in step S302 matches the model data 165a and the inspection content data 162d acquired in step S304 matches the captured portion data 165b. When the output processing unit 153 finds a record that matches the search criteria, it acquires the specific area data 165c of that record. The output processing unit 153 generates data for an attention alert image GC that indicates the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the data for the attention alert image GC to the projector 2 of the corresponding drone DR1 via the output unit 14 (step S306), and ends the output processing. Note that in the output processing, the output processing unit 153 generates data for an attention alert image GC for each projector 2 and outputs it to the corresponding projector 2.

When the projector 2 receives data for the attention drawing image GC from the output unit 14, it displays the attention drawing image GC pointing to the specific region R1 on the surface of the inspected product W, as shown in FIG. 2B, based on the received data for the attention drawing image GC. For example, in the case of FIG. 11A, as described above, the attention drawing image GC pointing to "SECTION 9" identified as the specific region R1 is projected onto the surface of the inspected product W.

Worker P performs a visual appearance inspection, being aware that defects tend to occur in the area indicated by the warning image GC. The output processing unit 153 executes output processing every time n hours have passed or whenever the manufactured product PR is changed, and responds to changes in model.

As described above, with the visual inspection support device 100 according to embodiment 1, a battery-powered drone DR1 equipped with a projector 2 is placed by utilizing the space in the visual inspection area. This reduces restrictions on the introduction of equipment due to the location of power sources in the visual inspection area, and also prevents the overall production line, including visual inspection equipment, from becoming too long. In addition, the projector 2 is used to display a warning image GC on the surface of the inspected product W, indicating areas where defects are likely to occur. This allows the operator P to easily identify areas where defects are likely to occur. This prevents defects from being overlooked, even when an operator P with low proficiency is in charge of the visual inspection. As a result, the quality of the visual inspection can be maintained. In other words, the quality of the visual inspection can be improved.

In addition, because worker P can easily identify the specific area R1, the time required for the visual inspection can be shortened, making the visual inspection more efficient.

In addition, since worker P can easily grasp the specific area R1, worker P can quickly become proficient.

If an operator P overlooks a defect, the inspected product W will have to be returned, which could result in a "rework" and loss of production time. However, the visual inspection support device 100 allows the operator P to easily identify the specific area R1, preventing defects from being overlooked. This prevents rework and reduces loss of production time and production time.

Furthermore, each time worker P moves a defective product D to the defective product storage area SD, worker P has camera 1 mounted on drone DR2 photograph the defective product D. In other words, worker P has camera 1 photograph all defective products D. As a result, the visual inspection support device 100 can collect data on all defective products D for which defects have occurred and statistically analyze it. Therefore, the reliability of the analysis results can be improved compared to when data on all defects cannot be analyzed, for example, because defects trends are analyzed manually. Furthermore, similar to the drone DR1 described above, battery-powered drone DR2 equipped with camera 1 is placed utilizing the space in the visual inspection area. This reduces restrictions on the introduction of equipment due to the location of power sources in the visual inspection area and prevents the overall production line, including visual inspection equipment, from becoming too long.

Furthermore, the visual inspection support device 100 only needs to identify the specific region R1 for the inspected product W where defects exist. This reduces the processing burden on the visual inspection support device 100 compared to when visual inspection is fully automated. Reducing the processing burden on the visual inspection support device 100 makes it possible to speed up visual inspection decisions. Alternatively, the quality of identifying the specific region R1 can be improved.

[Modification of the First Embodiment]
In the first embodiment, the worker P photographs the defective product D by operating the operation unit 11, but the person who photographs the defective product D is not limited to the worker P and may be, for example, a production manager, an analyst, etc.

In addition, in embodiment 1, defective products D were photographed each time they were placed in the defective product storage area SD, but defective products D do not need to be photographed each time they are placed in the defective product storage area SD. For example, defective products D may be photographed in batches every half day or every day. In this case, data indicating the date and time the defect occurred is entered into the control device 10 by the photographer based on the contents recorded in a document such as a defect discovery sheet.

The timing for photographing defective product D does not necessarily have to be determined by a person such as worker P, production manager, or analyst, but may also be determined by camera 1. For example, camera 1 photographs defective product D when it detects that defective product D has been placed in the defective product storage area SD using image recognition technology. Alternatively, camera 1 may photograph defective product D when it receives a signal indicating that defective product D has been detected from a sensor that detects the presence or absence of defective product D installed in the defective product storage area SD. Note that the sensor may be a reflective sensor, a transmissive sensor, or the like, as long as it is capable of detecting the presence or absence of defective product D.

Furthermore, while the defective areas were identified by dividing the defective product image GD and the non-defective product image GR into nine sections SC and comparing them, they may also be identified by dividing the defective product image GD and the non-defective product image GR into parts, such as the handle and lid, and comparing them.

Failure areas may also be identified using machine learning techniques. For example, the input processing unit 151 generates a trained model trained using training data in which data indicating the failure area is added to data on the defective product image GD. The input processing unit 151 may input the data on the defective product image GD as an input parameter into the generated trained model, and obtain data indicating the failure area that is output as an output parameter, thereby identifying the failure area. Machine learning software that has been trained in advance using training data may also be used, without generating a trained model.

The data stored in the image data storage unit 160 can also be changed as appropriate. For example, the analysis unit 152 may use a flag to distinguish the specific region R1 from other defective regions. The flag is further stored in the defective product image related data storage unit 164 in association with the specific region R1. The output processing unit 153 may generate image data indicating the specific region R1 based on the flag stored in the defective product image related data storage unit 164 and output the image data to the projector 2.

Also, as shown in Figures 14A to 14C, the projector 2 may change the projection angle at which the projector 2 projects an image onto the inspected product W within a predetermined range. In this modified example, the operation unit 11 accepts an operation to adjust the projection angle at which the projector 2 projects an image onto the inspected product W, and the transmission unit transmits a control signal to the drone DR1 instructing the drone DR1 to adjust the angle in accordance with the operation on the operation unit 11. The drone DR1 adjusts the projection angle in accordance with the control signal from the transmission unit. In this modified example, the predetermined projection angle is between 0 degrees and 90 degrees. In this modified example, the operation unit 11 accepts an operation to change the projection angle at which the projector 2 projects an image onto the inspected product W. When the projector 2 projects an image vertically downward as shown in Figure 14A, the projection angle is set to 0 degrees. As the projection angle is changed, for example, the projection angle shown in Figure 14B is changed to 90 degrees as shown in Figure 14C. As shown in Figures 14A to 14C, the projector 2 can project images from vertically downward to in a direction perpendicular to the vertical. According to this modification, even if the drone DR1 is not positioned vertically above the top surface of the product W at the visual inspection site SV, the projector 2 can project an image onto the top surface of the product W by changing the projection angle. The projector 2 in this modification also has a distortion correction function. For example, the projector 2 has a vertical keystone correction function that automatically corrects images distorted in the vertical direction, and a horizontal keystone correction function that automatically corrects images distorted in the horizontal direction. While the predetermined angle range that the projector 2 can change is 0 to 90 degrees in this modification, it is not limited to 0 to 90 degrees and may be any angle range, such as 0 to 135 degrees, as long as the effects of the present disclosure are achieved.

As shown in FIG. 15, drones DR1a and DR1b are placed vertically above the product W and at a position offset from the vertical position above the product W. The projection angles of projectors 2a and 2b are then changed so that they can project images onto the top surface of the product W and one of its side surfaces (the front surface). In this modification, inspection of two surfaces, the top surface and the front surface, of the product W can be performed at a single visual inspection site SV. While drone DR1 can hover at an angle of, for example, 45 degrees without changing the projection angle of projector 2, projecting an image onto the front surface of the product W, the hovering state of a tilted drone DR1 is more unstable than when the projection angle of projector 2 is changed. Placing multiple drones DR1a and DR1b at a single visual inspection site SV reduces the number of visual inspection sites SV and reduces the area used for visual inspection at the production site.

[Embodiment 2]
16 and 17 , a visual inspection support device 100 according to a second embodiment will be described. The basic configuration and basic operation of the visual inspection support device 100 according to the second embodiment are the same as those of the visual inspection support device 100 according to the first embodiment. However, the second embodiment differs from the first embodiment in that a weight is set based on the number of defects found and that the user can specify the weight of the specific region R1 to be output to the projector 2. The following mainly describes the differences from the first embodiment. Note that the "weight based on the number of defects" is an example of the "degree of defect occurrence" in the present disclosure. Furthermore, the weight is not limited to being based on the number of defects, and may be set based on the defect occurrence rate.

During the analysis process, the analysis unit 152 sets a weight for each specific region R1 based on the number of defects that occurred during a period set by the user. The weight is a value that evaluates the importance of the specific region R1; the higher the weight, the greater the importance of the specific region R1.

As shown in FIG. 16, the analysis unit 152 aggregates the data related to the defective product images GD stored in the defective product image-related data storage unit 164 shown in FIG. 8 by model, photographed part, and defective area (step S202), and sets a weight for the specific area R1 based on the aggregated number of records, i.e., the number of defects for each model, photographed part, and defective area (step S402).

For example, the analysis unit 152 sets a weight of "1" for a specific region R1 where the number of defect occurrences is between 3 and 5 during a period preset by the user, and a weight of "2" for a specific region R1 where the number of defect occurrences is 5 or more. Note that the weight is not limited to 2 levels, and can be preset by the user in 5 levels, 10 levels, etc. Furthermore, a weight equal to or greater than a predetermined weight reference value, for example "2," is considered "high," and a weight less than the weight reference value, for example "1," is considered "low."

In the example shown in Figure 8, for the one day of "August 26, 2020," there are seven records for the model "Rice Cooker," the photographed portion "Top Surface," and the defective area "SECTION 9." Therefore, the analysis unit 152 sets the weight of the specific area R1 "SECTION 9" to "2." After setting the weight, the analysis unit 152 generates data indicating the set weight.

The analysis unit 152 further stores the data indicating the generated weights as "weight data 165d" in the output preparation data storage unit 165 shown in FIG. 17 (step S206 in FIG. 16), and then terminates the analysis process.

The output preparation data storage unit 165 shown in Figure 17 stores weight data 165d in association with model data 165a, photographed portion data 165b, and specific area data 165c.

In the output process, the output processing unit 153 generates data for an attention drawing image GC that indicates the specific area R1 of the weight specified by the user as the output target, and outputs the data to the projector 2 via the output unit 14. The user can specify the weight by operating the operation unit 11. When the output processing unit 153 receives data indicating the specified weight from the operation unit 11, it generates data indicating the weight. The generated data indicating the weight is stored, for example, in the auxiliary storage device 16b. The output processing unit 153 generates data for the attention drawing image GC by referencing the data indicating the weight stored in the auxiliary storage device 16b.

More specifically, in step S306 of the output process shown in FIG. 13, the output processing unit 153 searches the data stored in the output preparation data storage unit 165 shown in FIG. 17 for a record that includes data indicating the specified weight in addition to the data acquired in steps S302 and S304. Specifically, the output processing unit 153 searches the data stored in the output preparation data storage unit 165 for a record in which the model data 161a acquired in step S302 matches the model data 165a, the examination content data 162d acquired in step S304 matches the photographed portion data 165b, and the weight data 165d indicates the specified weight.

When the output processing unit 153 finds a record that matches the search criteria, it generates data for an attention drawing image GC that points to the specific area R1 indicated by the specific area data 165c of that record. The output processing unit 153 outputs the data for the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in Figure 13), and then ends the output process.

For example, if "2" is specified as the weight, as shown in Figure 2B, an attention drawing image GC pointing to "SECTION 9" is displayed on the inspected product W. Through the above processing, an attention drawing image GC pointing to the specific region R1 for which the weight specified by the user has been set is displayed on the inspected product W.

As described above, the visual inspection support device 100 of embodiment 2 can display an attention-calling image GC with a specified weight set on the inspected product W. This improves convenience for the operator P and enables higher quality visual inspections.

[Modification of the Second Embodiment]
In addition, the output processing unit 153 may change attributes such as color, brightness, thickness, etc. of the image contained in the warning image GC according to the weight, so that the worker P can easily understand the weight, i.e., importance, of the specific area R1.

Third Embodiment
18 to 26, a visual inspection support device 100 according to a third embodiment will be described. The third embodiment differs from the first and second embodiments in that the attention-calling image GC displayed on the inspected product W changes depending on the proficiency level of the operator P in charge of the visual inspection. The following description will focus on the differences from the first and second embodiments.

The data storage unit 16 shown in FIG. 18 further includes data necessary to identify the proficiency level of the worker P in charge of the visual inspection. The data storage unit 16 further includes a schedule storage unit 166 that stores data related to the schedule of the worker P in charge of the visual inspection, and a worker master storage unit 167 that stores data related to the proficiency level of the worker P.

The assigned schedule storage unit 166 shown in FIG. 19 associates and stores data 166a of the person in charge of the visual inspection, assigned schedule data 166b indicating the schedule for which worker P is in charge of the visual inspection, and inspection content data 166e indicating the inspection content of the visual inspection for which worker P is in charge. The assigned schedule data 166b includes assigned start date and time data 166c indicating the start date and time of the assignment and assigned end date and time data 166d indicating the end date and time of the assignment. Data related to the assigned schedule is registered in advance in the assigned schedule storage unit 166 by, for example, the user.

The worker master storage unit 167 shown in FIG. 20 stores worker data 167a indicating worker P in charge of visual inspection and proficiency data 167b indicating worker P's proficiency, in association with each other. Proficiency data 167b is set, for example, as a three-level evaluation value ("1" to "3") based on worker P's years of service, experience, technical ability, etc. The higher the proficiency value, the higher the proficiency level of worker P. Note that proficiency evaluation is not limited to three levels, and can be preset by the user on a scale of five or ten, for example. Furthermore, proficiency levels equal to or above a predetermined proficiency standard value, for example, "2" or higher, are considered "high," while proficiency levels below the proficiency standard value, for example, "1," are considered "low."

In the input process shown in FIG. 21, when the input processing unit 151 identifies a defective area (step S110), it identifies the worker P who discovered the defect (step S502). The input processing unit 151 uses the inspection content data 162d and the photography time data acquired in step S108 as search conditions to search for records that match the search conditions from among the data stored in the in-charge schedule storage unit 166 shown in FIG. 19.

In detail, the input processing unit 151 searches the data stored in the assigned schedule storage unit 166 for records in which the inspection content data 162d acquired in step S108 of FIG. 21 matches the inspection content data 166e, and the photography time data is after the assigned start date and time data 166c and before the assigned end date and time data 166d. When the input processing unit 151 finds a record that matches the search criteria, it acquires the person in charge data 166a of that record to identify the worker P who discovered the defect.

When the input processing unit 151 identifies the worker P who discovered the defect, it associates the data acquired in steps S104 to S502 and stores it in the defective product image related data storage unit 164, as shown in FIG. 22 (step S112 in FIG. 21). In more detail, the input processing unit 151 further stores the person in charge data 166a acquired in step S502 as "worker data 164f" in the defective product image related data storage unit 164, and then terminates the input process.

The defective product image related data storage unit 164 stores defective product image identification name data 164a, model data 164b, photographed part data 164c, discovery date and time data 164d, defective area data 164e, and worker data 164f indicating the worker P who discovered the defect, in association with each other.

In the output process, the output processing unit 153 identifies the worker P in charge of the visual inspection and outputs data of the attention drawing image GC according to the proficiency level of that worker P to the projector 2 via the output unit 14. In other words, the output processing unit 153 outputs data of the attention drawing image GC according to the proficiency level of the worker P in charge of the visual inspection.

As shown in FIG. 23, once the output processing unit 153 identifies the content of the visual inspection (step S304), it identifies the worker P in charge of the visual inspection at the visual inspection site SV where the projector 2 to be output is installed (step S602). The output processing unit 153 searches for data stored in the in-charge schedule storage unit 166 shown in FIG. 19 using the inspection content data 162d acquired in step S304 and data indicating the current time as search conditions.

In detail, the output processing unit 153 searches the data stored in the assigned schedule storage unit 166 for records in which the inspection content data 162d acquired in step S304 matches the inspection content data 166e, and the shooting time data is after the assigned start date and time data 166c and before the assigned end date and time data 166d. When the output processing unit 153 finds a record that matches the search criteria, it acquires the person in charge data 166a of that record, thereby identifying the worker P in charge of the appearance inspection at the appearance inspection site SV where the projector 2 to be output is installed.

Next, the output processing unit 153 identifies the proficiency level of worker P identified in step S602 of FIG. 23 (step S604). The output processing unit 153 searches the data stored in the worker master storage unit 167 shown in FIG. 20 using the person in charge data 166a acquired in step S602 as a search condition.

In detail, the output processing unit 153 searches the data stored in the worker master storage unit 167 for records in which the person in charge data 166a acquired in step S602 matches the worker data 167a. When the output processing unit 153 finds a record that matches the search criteria, it acquires the proficiency data 167b of that record to identify the proficiency of worker P.

When the output processing unit 153 identifies the proficiency level of worker P, it obtains specific area data 165c associated with the identified proficiency level of worker P from the data stored in the output preparation data storage unit 165 shown in FIG. 24.

The output preparation data storage unit 165 stores model data 165a, photographed portion data 165b, specific area data 165c, and proficiency data 165e in association with each other. Note that the proficiency data 165e is registered by the user based on, for example, data stored in the defective product image related data storage unit 164 shown in FIG. 22.

When the output processing unit 153 acquires the specific area data 165c associated with the identified proficiency level of the worker P, it generates data for an attention drawing image GC that indicates the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the generated data for the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in Figure 23), and ends the output process.

When data for the attention drawing image GC is input from the output processing unit 153, each projector 2 projects the attention drawing image GC onto the surface of the inspected product W based on the input data for the attention drawing image GC.

For example, in the example shown in Figure 24, if the model of the object to be produced in the last n hours is a "rice cooker," the content of the visual inspection performed at the visual inspection site SV where projector 2 that outputs the data for the attention image GC is installed is "top surface," and the proficiency level of operator P in charge of the visual inspection is "1," then, as shown in Figure 25, an attention image GC including an image GC2 pointing to specific area R1 "SECTION 1" and an image GC1 pointing to specific area R1 "SECTION 9" is projected by projector 2 toward the inspected product W and displayed on the surface of the inspected product W. On the other hand, if operator P's proficiency level is "3," an attention image GC including an image GC2 pointing to specific area R1 "SECTION 1" is displayed on the surface of the inspected product W, as shown in Figure 26.

For example, a highly skilled worker P may feel cumbered if many images pointing to specific regions R1 are displayed on the surface of the inspected product W. On the other hand, a less skilled worker P may overlook defects if images pointing to all specific regions R1 are not displayed on the surface of the inspected product W. According to the visual inspection support device 100 of embodiment 3, the warning image GC displayed on the surface of the inspected product W can be changed according to the proficiency of the worker P. This improves convenience for the worker P and enables higher quality visual inspections.

Furthermore, according to embodiment 3, it is possible to tally up the defective areas discovered by each worker P, allowing the user to objectively grasp the proficiency level of each worker P.

[Modification of the Third Embodiment]
The order in which the input processes are performed can be changed as appropriate. The process of step S502 shown in Fig. 21 may be performed after step S108 and before step S112, for example, between step S108 and step S110.

[Fourth embodiment]
27 to 30, a visual inspection support device 100 according to a fourth embodiment will be described. The fourth embodiment differs from the first to third embodiments in that the tendency of defective areas for each time period is analyzed, and the attention-calling image GC changes depending on the time period during which the visual inspection is performed. The following description will focus on the differences from the first to third embodiments. Note that the "time period" is an example of the "time segment" in the present disclosure.

As shown in FIG. 27, once the input processing unit 151 identifies a defective area (step S110), it identifies the time period in which the defect was discovered (step S702). Based on the image capture time data, the input processing unit 151 determines whether the time period in which the defect was discovered belongs to one of the five time periods: "First Time Period," "Second Time Period," "Third Time Period," "Fourth Time Period," or "Other."

The time ranges for the five time periods, "First Time Period," "Second Time Period," "Third Time Period," "Fourth Time Period," and "Other," are set in advance by the user. For example, the user might set "First Time Period" as "9:00-10:00" after work begins, "Second Time Period" as "11:00-12:00" before lunch, "Third Time Period" as "13:00-14:00" after lunch, "Fourth Time Period" as "16:00-17:00" before work ends, and any time range outside of "First Time Period" to "Fourth Time Period" as "Other." When the input processing unit 151 identifies the time period in which the defect was discovered, it generates data indicating that time period.

Next, the input processing unit 151 associates and stores the data acquired in steps S104 to S702 (step S112). Specifically, the input processing unit 151 further stores the data indicating the time period acquired in step S702 as "time period data 164g" in the defective product image related data storage unit 164 shown in FIG. 28, and then terminates the input process.

The defective product image related data storage unit 164 shown in Figure 28 stores defective product image identification name data 164a, model data 164b, photographed portion data 164c, discovery date and time data 164d, defective area data 164e, and time period data 164g indicating the time period in which the defect was discovered, in association with each other.

In the analysis process, the analysis unit 152 analyzes the number of defects occurring in defective areas for each time period in which defects occurred during a period set by the user. The analysis unit 152 identifies defective areas in which the number of defects occurring for each time period is equal to or greater than the second defect threshold T2 as areas that are likely to experience defects during that time period. Note that the second defect threshold T2 may be set to the same value as the first defect threshold T1, or may be set to a different value. The second defect threshold T2 is an example of a criterion.

More specifically, in step S202 of the analysis process shown in FIG. 12, the analysis unit 152 aggregates data related to defective product images GD stored in the defective product image-related data storage unit 164 by model, photographed part, defective area, and time period.

Next, the analysis unit 152 identifies areas with a high tendency for defects to occur for each time period based on the aggregated number of records, i.e., the number of defects (step S204). For example, if the second defect threshold T2 is set to "3," and the data in the defective product image-related data storage unit 164 shown in FIG. 28 is aggregated for one day, "August 26, 2020," by model, photographed part, defective area, and time period, the number of records for the model "Rice Cooker," the photographed part "Top Surface," the time period "Fourth Time Period," and the defective area "SECTION 9" is four, which is more than three. Therefore, the analysis unit 152 identifies the model "Rice Cooker" and the defective area "SECTION 9" of the photographed part "Top Surface" as areas with a high tendency for defects to occur for the time period "Fourth Time Period," i.e., the identified area R1.

Once the analysis unit 152 has identified the specific area R1 for each time period, it further stores the data indicating the time period used for the aggregation as "time period data 165f" in the output preparation data storage unit 165 shown in FIG. 29 (step S206 in FIG. 12), and terminates the analysis process.

The output preparation data storage unit 165 shown in Figure 29 stores model data 165a, photographed portion data 165b, specific area data 165c, and time period data 165f in association with each other.

During the output process, the output processing unit 153 generates data for an attention-calling image GC that includes an image indicating the specific area R1 during the time period in which the visual inspection is to be performed, and outputs this data to the projector 2 via the output unit 14.

In the output process shown in FIG. 30, the output processing unit 153 identifies the content of the visual inspection (step S304), and then identifies which of the five time zones the most recent hour belongs to based on the current time data (step S802).

Next, the output processing unit 153 obtains the specific area data 165c associated with the identified time period from the data stored in the output preparation data storage unit 165 shown in FIG. 29.

When the output processing unit 153 acquires the specific area data 165c associated with the identified time period, it generates data for an attention drawing image GC that indicates the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the data for the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in Figure 30), and ends the output process.

When the projector 2 receives data for the attention drawing image GC from the output processing unit 153, it projects the attention drawing image GC onto the surface of the inspected product W based on the received data for the attention drawing image GC. Through the above processing, the attention drawing image GC corresponding to the time period during which the visual inspection is performed is displayed on the surface of the inspected product W.

Generally, the tendency for defects to occur changes as the time of day changes. For example, workers in each process tend to be highly concentrated immediately after the start of work, and less so immediately before the end of work. For this reason, defects caused by carelessness by workers, such as defects resulting from parts touching the conveyor, tend to occur more frequently immediately before the end of work than immediately after the start of work. On the other hand, defects caused by problems with the production line or product design tend to occur more frequently immediately after the start of work than defects caused by carelessness by workers. According to the visual inspection support device 100 of embodiment 4, the attention-calling image GC displayed on the surface of the inspected product W changes depending on the time of day when the visual inspection is performed. This makes it possible to improve the efficiency of visual inspection.

[Modification of the Fourth Embodiment]
The order in which the input processes are performed can be changed as appropriate. The process of step S702 shown in Fig. 27 may be performed before step S112, for example, between step S108 and step S110. Similarly, the order in which the output processes are performed can be changed as appropriate. The process of step S802 shown in Fig. 30 may be performed before step S306, for example, between step S302 and step S304.

Fifth Embodiment
31 to 35, a visual inspection support device 100 according to a fifth embodiment will be described. The fifth embodiment differs from the first to fourth embodiments in that the visual inspection support device 100 presents improvement target areas of the inspected product W based on the number of defects and items related to improvement measures that have been implemented for defects that have occurred in the past. The following describes the fifth embodiment, focusing on the differences from the first to fourth embodiments.

As shown in FIG. 31, the visual inspection support device 100 further includes an improvement measure presentation unit 154 that presents areas to be improved based on the number of defects and information related to implemented improvement measures. The improvement measure presentation unit 154 is realized by the processor 15 shown in FIG. 3 executing a control program and using the operation unit 11, display 12, input unit 13, output unit 14, main memory device 16a, and auxiliary memory device 16b as resources.

The data storage unit 16 further includes a trained model storage unit 168 and an improvement plan data storage unit 169.

The trained model storage unit 168 stores trained models. Trained models are generated using a publicly known machine learning library, for example, by a "supervised learning method" in which training data is trained using a set of data with correct answer labels attached. Note that trained models may also be trained and generated using other learning methods, such as "unsupervised learning methods," "reinforcement learning methods," and "semi-supervised learning methods."

The learning algorithm can be a well-known learning algorithm such as a neural network. A neural network has an input layer composed of multiple nodes to which different input parameters are input, an intermediate layer to which signals output from each node in the input layer are input, and an output layer to which signals output from the intermediate layer are input and which outputs output parameters. Note that a neural network may be composed of one or more intermediate layers.

Data related to implemented improvement measures is used as training data, and data indicating areas to be improved and data indicating priority points for improvement are used as correct labels.

Items related to implemented improvement measures include data indicating the model, data indicating the surface of the inspected product W to be inspected, data indicating the defective area, data indicating the number of defects, data indicating a rework coefficient set based on the rework time required to correct the defect, and data indicating a replacement coefficient set based on the cost of a replacement product required to correct the defect. The data indicating the model, data indicating the surface of the inspected product W to be inspected, data indicating the defective area, data indicating the number of defects, data indicating the rework coefficient, and data indicating the replacement coefficient are associated with each other. The number of defects is calculated by tallying up the data for each model, surface of the inspected product W to be inspected, and defective area.

The improvement priority points used for the correct answer label are set for each model, surface of the inspected product W to be inspected, and defective area. The improvement priority points are calculated by multiplying the number of defects by the rework time coefficient and the replacement coefficient. The improvement priority points are set in descending order of the value obtained by multiplying the number of defects by the rework time coefficient and the replacement coefficient. Alternatively, the improvement priority points may be set according to the range of the value obtained by multiplication, for example. The improvement measure points may be set to "1" when the value obtained by multiplication is equal to or greater than 10 and less than 10, and "2" when the value obtained by multiplication is equal to or greater than 10 and less than 50.

The improvement measure presentation unit 154 generates a trained model and presents improvement measures to the user using the trained model.

As shown in FIG. 31, the improvement measure presentation unit 154 includes a learning unit 154a that generates a trained model, and an inference unit 154b that infers improvement measures using the trained model.

The learning process performed by the learning unit 154a is described below. The learning process shown in Figure 32 begins when the operation unit 11 receives an instruction from the user to execute the process.

The learning unit 154a accepts data related to implemented improvement measures to which correct answer labels have been added as training data (step S902). The training data is input to the learning unit 154a via the input unit 13.

The learning unit 154a repeatedly learns using the training data (step S904) and analyzes the characteristics and rules of the data contained in the training data. The learning unit 154a stores the trained model in the trained model storage unit 168 (step S906) and ends the learning process.

The inference unit 154b uses the trained model to execute the inference process shown in Figure 33. The inference process is executed at a timing preset by the user. The user sets the timing for executing the inference process to, for example, once a week or once a month.

The inference unit 154b inputs data related to the implemented improvement measures as input parameters to the trained model (step S1002). The input parameters are data input via the operation unit 11, input unit 13, etc.

The inference unit 154b uses the trained model to perform inference based on the input parameters (step S1004), outputs data indicating areas to be improved and data indicating priority points for improvement as output parameters (step S1006), and terminates the inference process.

The improvement proposal presentation unit 154 acquires the output parameters output from the inference unit 154b as improvement proposal data. Upon acquiring the improvement proposal data, the improvement proposal presentation unit 154 stores the improvement proposal data in the improvement proposal data storage unit 169 shown in FIG. 34.

The improvement measure data storage unit 169 stores, in association with one another, priority order data 169a indicating the order of priority in which improvement is required, model data 169b indicating the model, part data 169c indicating the surface of the inspected product W to be inspected, and improvement target area data 169d indicating the area to be improved. The improvement measure presentation unit 154 stores data indicating the area to be improved contained in the output parameters as improvement target area data 169d. The improvement measure presentation unit 154 sets priorities based on data indicating improvement priority points contained in the output parameters output by the inference unit 154b, and stores the set data as priority order data 169a. The improvement measure presentation unit 154 sets priorities, for example, in descending order of the value of the improvement priority points.

When the improvement measure presentation unit 154 acquires the improvement measure data, it presents prioritized areas to be improved as improvement measures to the user based on the acquired improvement measure data.

The improvement measure presentation unit 154 may, for example, use an application such as a spreadsheet software to display improvement measures on the display 12 based on the improvement measure data. The improvement measure data can be used as training material for new employees. The improvement measure data can also be used as reference material when a similar defect occurs.

As described above, according to the visual inspection support device 100 of embodiment 5, the improvement measure presentation unit 154 presents improvement measures to the user using a trained model that has been trained using data including the rework time for defects that occurred in the past, the cost of replacement products, and the like. In other words, the improvement measure presentation unit 154 presents improvement measures to the user that take into account losses such as the rework time for defects that occurred in the past and the cost of replacement products. By implementing the improvement measures presented by the improvement measure presentation unit 154, the user can carry out efficient improvement activities. As a result, the occurrence of defects is reduced and the workload of visual inspections is lightened. This makes it possible to improve the efficiency of visual inspections and to improve the quality of visual inspections.

The improvement measure presentation unit 154 presents prioritized areas for improvement to the user, allowing the user to carry out improvement activities efficiently.

In addition, due to time constraints, some factories are unable to analyze all defects that occur. Analysis results based on a portion of the defects that occur are unreliable, and improvement measures based on unreliable analysis results may not be in line with the actual situation. Improvement measures that are unreliable often deviate from the improvement measures desired by the workers in charge of each process, and improvement measures that deviate from the workers' requests make it impossible to implement efficient improvement activities. However, with the visual inspection support device 100, data on all defects that occur is collected and analyzed, making it possible to obtain highly reliable analysis results. Therefore, improvement measures that are in line with the actual situation can be presented to the user, enabling efficient improvement activities to be implemented.

Furthermore, if a time lag occurs between when a defect occurs and when improvement activities are implemented, there is a possibility that the improvement measures implemented in the improvement activities may not be in line with the actual situation. However, the visual inspection support device 100 according to embodiment 5 can suppress the occurrence of a time lag between when a defect occurs and when improvement activities are implemented. This allows improvement measures that are in line with the actual situation to be presented to the user, enabling efficient improvement activities to be implemented.

[Modification of the Fifth Embodiment]
35, the improvement data storage unit 169 may further store, in association with one another, number data 169e indicating the number of defects that occurred during a period set by the user, rework time data 169f indicating the rework time required to correct defects that occurred in the past, and replacement cost data 169g indicating the cost of substitute products required to correct defects that occurred in the past. The rework time data 169f is calculated based on the rework coefficient, and the replacement cost data 169g is calculated based on the replacement coefficient.

The improvement measure presentation unit 154 may further present the number of defects, rework time, and replacement costs to the user based on the data stored in the improvement measure data storage unit 169. This allows the user to quantitatively and objectively grasp the details of the defects that have occurred, such as rework time and replacement costs.

In addition, the improvement measure presentation unit 154 presents prioritized areas to be improved as improvement measures to the user, but prioritization is not necessary. In this case, the training data learned by the learning unit 154a can omit the priority improvement points. Furthermore, the inference unit 154b outputs data indicating the areas to be improved as output parameters.

The input parameters input to the learning unit 154a may further include data indicating the degree of difficulty of improvement. The degree of difficulty of improvement is preset by the user as a 3-level, 5-level, or other level based on the cause of the defect (scratches, color unevenness) and the area of the defect. For example, a stain on the inspected product W can be remedied by a relatively easy task such as wiping off the stain, so the degree of difficulty of improvement is set to "1." A dent on the inspected product W can be remedied by repairing the dent, which requires more time than wiping off the stain, so the degree of difficulty of improvement is set to "2." A scratch on the inspected product W can be remedied by replacing the part, which requires more time than wiping off the stain and repairing the dent, so the degree of difficulty of improvement is set to "3." By further including the degree of difficulty of improvement as an input parameter input to the learning unit 154a, more efficient improvement activities can be carried out.

In embodiment 5, the learning unit 154a uses a neural network as the learning algorithm, but the learning unit 154a may also use other known methods, such as genetic programming, functional logic programming, or support vector machines, as the learning algorithm.

In embodiment 5, the learning unit 154a and the inference unit 154b are provided in the control device 10, but the learning unit 154a and the inference unit 154b may be provided in a device different from the control device 10. The improvement measure presentation unit 154 acquires, as improvement measure data, output parameters output from the inference unit 154b provided in a device different from the control device 10.

The improvement measure data storage unit 169 may also be stored in a device that can be shared among multiple departments. For example, by devising ways to hold workpieces during each process in a factory, collisions between the workpieces and factory equipment such as conveyors and workbenches can be reduced, often reducing the occurrence of defects. Therefore, by storing the improvement measure data storage unit 169 in a format that can be shared among multiple departments, effective improvement measures can be deployed throughout the factory, enabling improvement activities to be carried out efficiently.

In addition, the improvement measure presentation unit 154 may use data on implemented improvement measures for similar models as well as the same model. In other words, the improvement measure presentation unit 154 can also use data on implemented improvement measures for similar models as learning data.

Note that the present disclosure is not limited to the example embodiments described above, and can be implemented in other embodiments by making appropriate modifications.

For example, embodiments 1 to 5 can be combined as appropriate. In particular, the attention drawing image GC displayed on the inspected product W may change depending on the weight described in embodiment 2 and the proficiency level of the operator P in charge of the visual inspection described with reference to embodiment 3. For example, if the proficiency level of the operator P in charge of the visual inspection is low at "1," the attention drawing image GC may include an image pointing to the specific region R1 to which all weights "1" to "3" have been set, and if the proficiency level of the operator P in charge of the visual inspection is high at "3," the attention drawing image GC may include an image pointing to the specific region R1 to which the high weight "3" has been set.

In this disclosure, the tendency of defective areas was analyzed as the tendency of defects, but the tendency analyzed by the visual inspection support device 100 is not limited to defective areas, and may also include causes of defects such as scratches and color unevenness. Furthermore, while areas with a high tendency for defects to occur were identified as the tendency of defective areas, areas with a low tendency for defects to occur may also be identified.

In this disclosure, the defect location data is data of the defective product image GD, but the defect location data may be data related to the defect location of the defective product D and is not limited to data of the defective product image GD. The defect location data may also be, for example, text data, audio data, etc.

In this disclosure, the warning image GC is projected onto the inspected product W by the projector 2, but as long as the worker P can identify the area on the inspected product W, the warning image GC may also be displayed, for example, on a monitor installed in the visual inspection area SV.

In the present disclosure, data showing a warning image GC pointing to the specific region R1 is output as information indicating an area where defects are likely to occur. However, information about areas where defects are likely to occur is not limited to data showing the warning image GC, and may also be audio data explaining the specific region R1.

The images GC1 and GC2 pointing out the specific region R1 may be any shape that allows the worker P to recognize the specific region R1, and are not limited to being circular. The images GC1 and GC2 pointing out the specific region R1 may be a rectangle, oval, arrow, etc. Alternatively, they may be solid images in a color different from the external color of the inspected product W. Alternatively, they may be images of text such as "SECTION 9" or "Pay attention to SECTION 9."

In this disclosure, the data storage unit 16 is stored in the auxiliary storage device 16b of the control device 10, but the data storage unit 16 may also be stored in the memory area of a device different from the control device 10, such as an in-house shared server or cloud server.

In this disclosure, the air vehicles DR1 and DR2 are drones, but they are not limited to drones. As long as the air vehicles DR1 and DR2 can stop in the air, in other words, hover, and achieve the effects of this disclosure, they may be any type of unmanned or manned air vehicle, not just drones. For example, they may be manned or unmanned helicopters.

In this disclosure, the product PR was for electrical appliances, but product PR is not limited to electrical appliances and can be any product that requires visual inspection.

1 Camera, 2, 2a, 2b Projector, 10 Control device, 11 Operation unit, 12 Display, 13 Input unit, 14 Output unit, 15 Processor, 16 Data storage unit, 16a Main storage unit, 16b Auxiliary storage unit, 100 Visual inspection support device, 151 Input processing unit, 152 Analysis unit, 153 Output processing unit, 154 Improvement measure presentation unit, 154a Learning unit, 154b Inference unit, 160 Image data storage unit, 161 Production schedule storage unit, 161a Model data, 161b Production schedule data, 161c Production start date and time data, 161d Production end date and time data, 162 Equipment master storage unit, 162a Equipment name data, 162b Equipment identification name data, 162c Installation location data, 162d Inspection content data, 163 Non-defective product image related data storage unit, 163a Good product image identification name data, 163b model data, 163c photographed portion data, 164 defective product image related data storage unit, 164a defective product image identification name data, 164b model data, 164c photographed portion data, 164d discovery date and time data, 164e defective area data, 164f worker data, 164g time period data, 165 output preparation data storage unit, 165a model data, 165b photographed portion data, 165c specific area data, 165d weight data, 165e proficiency data, 165f time period data, 166 assigned schedule storage unit, 166a worker data, 166b assigned schedule data, 166c assigned start date and time data, 166d assigned end date and time data, 166e inspection content data, 167 worker master storage unit, 167a worker data, 167b Proficiency data, 168: trained model storage unit, 169: improvement measure data storage unit, 169a: priority data, 169b: model data, 169c: partial data, 169d: improvement target area data, 169e: number data, 169f: rework time data, 169g: replacement cost data, D: defective product, GC: warning image, GD: defective product image, GR: good product image, GS: image, P: worker, PR: product, R1: specific area, SC: section, SD: defective product storage area, SV: visual inspection area, T1: first defect threshold, T2: second defect threshold, W: inspected product, DR1, DR2, DR1a, DR1b: drone.

Claims (14)

An appearance inspection support device that supports the appearance inspection of an inspected product,
an input unit for inputting defect location data indicating defect locations of defective products among the products to be inspected;
an analysis unit that analyzes the defect location data of the plurality of defective products and sets a predetermined area where the number of occurrences of defects or the rate of occurrence of defects is equal to or greater than a standard as a specific area where the occurrence of defects is likely to occur;
an output unit that outputs information indicating an area where the number of occurrences of defects or the rate of occurrence of defects is equal to or greater than the standard;
a first flying object having a projector to which information indicating an area analyzed to determine that the number of defect occurrences or the defect occurrence rate is equal to or higher than a standard is input from the output unit;
Equipped with
The projector projects an image indicating the predetermined area onto the product to be inspected based on the information input from the output unit. The visual inspection support device of claim 1, wherein the first air vehicle comprises a drone. Further comprising an operation unit that receives an operation by a user and a transmission unit that transmits a control signal to the first flying object,
the transmitter transmits a control signal to the first aircraft instructing the first aircraft to perform an operation in accordance with the operation of the operation unit;
The visual inspection support device according to claim 1 or 2, wherein the first flying object operates in accordance with a control signal from the transmitter. Further comprising an operation unit that receives an operation by a user and a transmission unit that transmits a control signal to the first flying object,
the operation unit accepts an operation to adjust a projection angle at which the projector projects an image onto the product to be inspected,
the transmitter transmits a control signal to the first aircraft, the control signal instructing the first aircraft to adjust the angle in accordance with the operation of the operation unit;
the first flying object adjusts the projection angle in accordance with the control signal from the transmitter;
3. The visual inspection support device according to claim 1. The visual inspection support device of any one of claims 1 to 4, wherein the analysis unit sets an area where defects occur, where the number of defects or the defect rate is equal to or greater than a first defect threshold, as a specific area where the defects are likely to occur. the analysis unit determines the degree of occurrence of defects for each region of the product to be inspected based on the number of occurrences of defects or the rate of occurrence of defects;
6. The visual inspection support device according to claim 5, wherein the output unit outputs the image indicating an area where defects occur to a predetermined degree. a means for determining a proficiency level of the visual inspection of a worker in charge of the visual inspection,
7. The visual inspection support device according to claim 5, wherein the output unit outputs an image indicating an area where the defect occurs, according to the level of proficiency. The analysis unit
Analyzing the number of occurrences of defects at the defective part for each area including the defective part for each time segment in which the defect is detected;
Identifying an area including the defect portion where the analyzed number of defect occurrences is equal to or greater than a second defect threshold as an area where the defect is likely to occur;
8. The visual inspection support device according to claim 5, wherein the output unit outputs information indicating an area where the defect is likely to occur, the area corresponding to the time segment that includes a time when the visual inspection is performed. The visual inspection support device of any one of claims 5 to 8, further comprising an improvement measure suggestion unit that suggests areas of the inspected product that should be improved based on the number of defects and information related to implemented improvement measures. The improvement measure suggestion unit
a learning unit that generates a trained model that has learned data related to the implemented improvement measures;
an inference unit that inputs data related to the implemented improvement measures into the trained model and outputs data indicating the improvement target area and data indicating priority points for improvement;
Equipped with
The improvement measure suggestion unit
setting a priority order of improvement needs for each improvement target area based on the data indicating the priority points for improvement output by the inference unit;
The visual inspection support device according to claim 9 , further comprising: displaying the priority order and the improvement target area. a second flying object having a camera that photographs the defective product and generates image data of the defective product;
The visual inspection support device according to claim 1 , wherein the camera outputs image data of the defective product to the input unit as the defect location data. The visual inspection support device of claim 11, wherein the second air vehicle comprises a drone. an input step of inputting defect location data indicating defect locations of defective products in which defects are found among the products to be inspected;
an analysis step of analyzing the tendency of an area including the defective portion based on the defect portion data of the plurality of defective products, and identifying an area where the tendency of the occurrence of the defect is above a standard;
an output step of outputting information indicating an area where the tendency for defects to occur is equal to or greater than a standard;
A visual inspection support method, comprising: an input step of inputting defect location data indicating defect locations of defective products in which defects are found among the products to be inspected;
an analysis step of analyzing the tendency of an area including the defective portion based on the defect portion data of the plurality of defective products, and identifying an area where the tendency of the occurrence of the defect is above a standard;
and an output step of outputting information indicating the area where the tendency for defects to occur is above a standard.