JP7624617B2 - Illumination condition creation support device, captured image estimation method, and illumination condition creation method - Google Patents

Description

The present invention relates to an illumination condition creation support device that supports the creation of illumination conditions for a lighting means that illuminates a part, a captured image estimation method, and an illumination condition creation method.

The component mounting device uses a nozzle to pick up components supplied by a component supply device (such as a tape feeder), illuminates the components held by the nozzle with lighting means while capturing an image with a camera (imaging means), and after recognizing the holding state of the components, such as the amount of positional deviation of the components relative to the nozzle, mounts the components held by the nozzle on a board. The optimal lighting conditions under which the lighting means illuminates the components differ for each component depending on the material and shape of the component, so work is carried out to set the optimal lighting conditions before production begins, etc.

Patent Document 1 discloses a component mounting device (electronic component mounting device) that, when an image of a component captured by a camera during the production of a mounted board is determined to be defective, switches to an arrangement mode in which an operator changes the lighting conditions and has the camera re-image the component, and then resets the lighting conditions of the lighting means, which is made up of four types of lighting units, while viewing the captured image displayed on the operation screen.

JP 2004-186175 A

However, in conventional technologies including Patent Document 1, it takes several seconds for a camera to capture an image of a component, and furthermore, in order to determine the optimal lighting conditions for a lighting means consisting of multiple lighting units, it is necessary to capture images under a combination of multiple conditions. This creates the problem that it takes a long time to capture images of the many components mounted on a single mounting board under multiple lighting conditions and then set the optimal lighting conditions based on the captured images.

The present invention aims to provide an illumination condition creation support device, an image capture estimation method, and an illumination condition creation method that can capture images of parts under multiple illumination conditions in a short period of time.

The illumination condition creation support device of the present invention is an illumination condition creation support device that supports creation of illumination conditions for a lighting means that illuminates a part imaged by an imaging means, and includes: a learning unit that generates a learning model based on a plurality of captured images captured by the imaging means of a part illuminated by the lighting means under a plurality of lighting conditions and the lighting conditions at the time of imaging; an image estimation unit that uses the generated learning model to estimate, from one captured image captured by the imaging means of a part illuminated by the lighting means under one lighting condition, another captured image that will be obtained when the imaging means images a part illuminated by the lighting means under another lighting condition; an input receiving unit that receives input of the other lighting conditions; and a display unit that displays the other captured image estimated by the image estimation unit based on the input other lighting conditions .

The captured image estimation method of the present invention includes a first imaging step of imaging a part illuminated by a lighting means under a plurality of lighting conditions with an imaging means; a learning step of generating a learning model based on the plurality of captured images obtained by the first imaging step and the lighting conditions at the time of imaging; a second imaging step of imaging a part illuminated by the lighting means under a single lighting condition with the imaging means; an image estimation step of estimating, using the generated learning model, another captured image that would be obtained when the imaging means images the one part illuminated by the lighting means under a different lighting condition, from the captured image of the one part obtained by the second imaging step; an input receiving step of receiving an input of the other lighting conditions; and a display step of displaying the other captured image estimated in the image estimation step based on the input other lighting conditions .

an illumination condition creating method of the present invention for creating illumination conditions used when a lighting means illuminates a part imaged by an imaging means, the illumination condition creating method including: a first imaging step of imaging, with the imaging means, a part illuminated by the lighting means under a plurality of lighting conditions; a learning step of generating a learning model based on a plurality of captured images obtained by the first imaging step and the lighting conditions at the time of imaging; a second imaging step of imaging, with the imaging means, a part illuminated by the lighting means under a single lighting condition; an image estimation step of estimating, from the one captured image obtained by the second imaging step, another captured image that will be obtained when the imaging means images the one part illuminated by the lighting means under a different lighting condition, using the learning model generated by the learning step; an illumination condition creating step of creating illumination conditions suitable for the lighting means to illuminate the one part, based on the other captured image of the one part estimated by the image estimation step; an input receiving step of receiving an input of the other lighting conditions; and a display step of displaying the other captured image estimated in the image estimation step based on the input other lighting conditions .

The present invention makes it possible to obtain images of parts under multiple lighting conditions in a short period of time.

FIG. 1 is a configuration explanatory diagram of a component mounting system according to an embodiment of the present invention; FIG. 1 is a configuration explanatory diagram of a component mounting device according to an embodiment of the present invention; FIG. 1 is a configuration explanatory diagram of a component recognition camera provided in a component mounting device according to an embodiment of the present invention; FIG. 1 is a block diagram showing a configuration of a control system of a component mounting apparatus according to an embodiment of the present invention. FIG. 1 is an explanatory diagram of a captured image estimation according to an embodiment of the present invention; FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; FIG. 2 is a flow diagram of a captured image estimation method according to an embodiment of the present invention. FIG. 2 is a flow diagram of a method for creating illumination conditions according to an embodiment of the present invention.

An embodiment of the present invention will be described in detail below with reference to the drawings. The configurations, shapes, etc. described below are merely examples for explanatory purposes and can be modified as appropriate depending on the specifications of the component mounting device, component recognition camera, management computer, and offline camera device. In the following, the same reference numerals are used for corresponding elements in all drawings, and duplicated explanations will be omitted. In FIG. 2 and in some parts described below, the X-axis in the board transport direction (the direction perpendicular to the paper surface in FIG. 2) and the Y-axis perpendicular to the board transport direction (the left-right direction in FIG. 2) are shown as two axes that are perpendicular to each other in the horizontal plane. In FIG. 2 and in some parts described below, the Z-axis (the up-down direction in FIG. 2) is shown as the height direction perpendicular to the horizontal plane.

First, the configuration of component mounting system 1 will be described with reference to FIG. 1. Component mounting system 1 is configured by connecting multiple component mounting devices M1 to M3. Component mounting devices M1 to M3 are connected to management computer 3 via communication network 2. Management computer 3 manages the production of mounted boards by component mounting system 1, and also assists in creating lighting conditions used by component mounting devices M1 to M3, which will be described later. Note that the number of component mounting devices M1 to M3 provided in component mounting system 1 is not limited to three, and may be one, two, four or more.

The component mounting system 1 includes an offline camera device 4 that is not connected to the component mounting devices M1 to M3. The offline camera device 4 includes an offline camera with a configuration similar to the component recognition camera 20 (see FIG. 3) included in the component mounting devices M1 to M3 described below. In the offline camera device 4, prior to the production of a mounted board, an image of a new component to be mounted on the board is captured by the offline camera. The image captured by the offline camera is transmitted to the management computer 3 and used to create the lighting conditions to be used by the component mounting devices M1 to M3. Alternatively, the image captured by the offline camera is processed by an information processing device included in the offline camera device 4 to create the lighting conditions to be used by the component mounting devices M1 to M3.

Next, the configuration of component mounting devices M1 to M3 will be described with reference to FIG. 2. Component mounting devices M1 to M3 have the same configuration, and below, component mounting device M1 will be described. Component mounting device M1 has the function of mounting components D onto board B. A board transport mechanism 6 provided on the upper surface of base 5 transports board B along the X-axis, positions it, and holds it. A head movement mechanism 7 provided above base 5 moves a mounting head 8, which is detachably attached via a plate 7a, in horizontal directions (X-axis direction, Y-axis direction). A suction nozzle 9 is detachably attached to the lower end of mounting head 8.

A plurality of tape feeders 10 are mounted in a line along the X-axis on the feeder base 11a on the top of the carriage 11, which is connected to the base 5 on the side of the board transport mechanism 6. The carriage 11 holds a carrier tape 12 wound around a reel 13 and storing the components D to be supplied to the component mounting device M1. The carrier tape 12 inserted into the tape feeder 10 is pitch-fed, and the components D stored on the carrier tape 12 are supplied in sequence to the component supply port 10a provided on the top of the tape feeder 10.

In FIG. 2, in the component mounting operation, the mounting head 8 is moved above the tape feeder 10 by the head movement mechanism 7, and picks up the component D supplied to the component supply port 10a of the tape feeder 10 by vacuum suction with the suction nozzle 9 (arrow a). The mounting head 8 holding the component D is moved above the board B held by the board transport mechanism 6 by the head movement mechanism 7, and mounts the component D at a predetermined component mounting position Ba on the board B (arrow b).

In FIG. 2, a board recognition camera 14 with its optical axis facing in the negative direction of the Z axis is attached to plate 7a. The board recognition camera 14 moves integrally with the mounting head 8 in the positive and negative directions of the X axis and the positive and negative directions of the Y axis by the head movement mechanism 7. As the mounting head 8 moves, the board recognition camera 14 moves above board B positioned by the board transport mechanism 6, and captures an image of a board mark (not shown) provided on board B.

A component recognition camera 20 with its optical axis facing upward is attached to the top surface of the base 5 between the board transport mechanism 6 and the tape feeder 10. The component recognition camera 20 captures an image of the holding posture of the component D held by the suction nozzle 9 as the suction nozzle 9 that picked up the component D passes overhead. When the mounting head 8 mounts the component on the board B, the mounting position is corrected taking into account the image of the component D captured by the component recognition camera 20 and the image of the board position captured by the board recognition camera 14.

In FIG. 2, a touch panel 15 operated by the worker is installed in front of the component mounting device M1 at the position where the worker works. The touch panel 15 displays various information on its display, and the worker inputs data and operates the component mounting device M1 using operation buttons and other input elements displayed on the display.

Next, the configuration of the component recognition camera 20 will be described with reference to FIG. 3. The component recognition camera 20 is equipped with an imaging unit 22 having an imaging element such as a one-dimensional CCD or two-dimensional CMOS sensor inside a housing 21. A ceiling plate 21a made of plate glass or the like is installed on the top surface of the housing 21. A lens 23 is disposed above the imaging unit 22. The imaging unit 22 is installed with its optical axis 22a facing upward, and images the component D held by the suction nozzle 9 through the lens 23 and the ceiling plate 21a. In other words, the imaging unit 22 and the lens 23 form an imaging means C that images the component D from below.

Inside the housing 21, a reflective illumination unit 24, a transmitted illumination unit 25, and a side illumination unit 26, each of which is configured with multiple LED chips, etc., are installed. The light emitted from the reflective illumination unit 24 mainly illuminates the bottom surface Da of the component D. The light emitted from the transmitted illumination unit 25 is mainly reflected by the flange 9a of the suction nozzle 9 and illuminates the component D from above. The light emitted from the side illumination unit 26 mainly illuminates the bumps Db, etc., formed on the bottom surface Da of the component D.

In FIG. 3, the component recognition camera 20 includes an illumination driver 27. The illumination driver 27 supplies currents for the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 to emit light. The illumination driver 27 changes the current values of the currents supplied to the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 based on the illumination values included in the illumination conditions transmitted from a control device 30 (see FIG. 4) included in the component mounting device M1. In other words, the illumination conditions are a combination of the illumination values of the multiple illumination units.

Illumination conditions are registered for each type of part D, and the intensity (brightness) of the light irradiated from the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 is changed according to the part D being imaged. In this way, the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 are illumination means L that illuminates the part D imaged by the imaging means C (imaging unit 22, lens 23). In other words, the illumination means L includes multiple illumination units (reflective illumination unit 24, transmitted illumination unit 25, side illumination unit 26) that illuminate the part D, and the intensity of the light irradiated by the multiple illumination units is changed according to the illumination value.

In FIG. 3, the component mounting device M1, with the lighting means L irradiating light according to the lighting conditions, moves the suction nozzle 9 holding the component D in the horizontal direction (arrow c) while imaging (scanning) the component D with the imaging means C. The direction in which the component D is moved is changed as appropriate according to the specifications of the mounting head 8 and the component recognition camera 20. In this way, scanning while moving the component D takes an image takes about several seconds.

Next, referring to Figure 4, the configuration of the control system for component mounting devices M1 to M3 will be explained, focusing on the function of creating the lighting conditions for the component recognition camera 20. Component mounting devices M1 to M3 have the same configuration, and below, component mounting device M1 will be explained. Component mounting device M1 is equipped with a control device 30, a board transport mechanism 6, a head moving mechanism 7, a mounting head 8, a tape feeder 10, a board recognition camera 14, a touch panel 15, and a component recognition camera 20.

The control device 30 includes a memory unit 31, a mounting control unit 32, an image capture processing unit 33, a learning unit 34, an image estimation unit 35, an input reception unit 36, an illumination condition creation unit 37, and an illumination condition feasibility determination unit 38. The memory unit 31 is a storage device, and stores mounting data 39, captured image data 40, a learning model 41, estimated image data 42, illumination condition data 43, inter-device correction data 44, etc. In the mounting data 39, various information such as the component type and size of component D to be mounted on board B, and the coordinates of component mounting position Ba on board B, is stored for each board type of mounting board to be produced.

In FIG. 4, the mounting control unit 32 executes a component mounting operation based on mounting data 39, in which the mounting head 8 picks up the component D supplied by the tape feeder 10 using the suction nozzle 9 and mounts it at the component mounting position Ba on the board B. The imaging processing unit 33 scans and images the component D held by the suction nozzle 9 using the imaging means C (imaging unit 22, lens 23) while moving the suction nozzle 9 holding the component D above the component recognition camera 20 during the component mounting operation, and recognizes the holding orientation of the component D.

When scanning and imaging the component D, the imaging processing unit 33 controls the illumination driving unit 27 according to the illumination conditions included in the illumination condition data 43 to irradiate a predetermined amount of light from the illumination means L (the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26). The imaging processing unit 33 also executes a learning data creation process in which the illumination means L illuminates the component D held by the suction nozzle 9 under multiple illumination conditions while the imaging means C scans and images the component D to obtain multiple captured images. The imaging processing unit 33 associates information identifying the captured component D and the illumination conditions at the time of capturing the image with the captured image and stores the captured image data 40 in the storage unit 31. The learning data creation process is executed during spare time during component mounting work or during a changeover work to change the board type of the mounting board to be produced.

In FIG. 4, the learning unit 34 uses multiple captured images of the part D illuminated by the lighting means L under multiple lighting conditions and captured by the imaging means C, which are included in the captured image data 40, and the lighting conditions at the time of capturing the images as teacher data to generate a learning model 41 that estimates the captured image of the part D described below, using a learning algorithm using machine learning or the like. As the learning algorithm, a neural network (including deep learning using a multi-layered neural network), genetic programming, a decision tree, a Bayesian network, a support vector machine (SVM), or the like can be used. The generated learning model 41 is stored in the memory unit 31.

In this way, the learning unit 34 provided in the component mounting device M1 generates a learning model 41 based on multiple captured images of the component D illuminated under multiple lighting conditions by the lighting means L provided in the component mounting device M1 and captured by the imaging means C provided in the component mounting device M1, and the lighting conditions at the time of capturing the images.

In FIG. 4, the image estimation unit 35 uses the generated learning model 41 to estimate (generate) from one captured image captured by the imaging unit C of one part D illuminated by the lighting unit L under one lighting condition, another captured image (hereinafter referred to as an "estimated image") obtained when the imaging unit C captures an image of one part D illuminated by the lighting unit L under another lighting condition. The image estimation unit 35 associates the generated estimated image with information identifying the part D and the lighting condition, and stores the image in the storage unit 31 as estimated image data 42.

Now, referring to FIG. 5, the captured image estimation process for generating an estimated image will be described. First, the learning unit 34 generates a learning model 41 based on the captured images of the part D captured under a plurality of lighting conditions included in the captured image data 40 and the lighting conditions at the time of capturing the images (ST1: learning process). The created learning model 41 is stored in the memory unit 31. Next, the image estimation unit 35 uses the generated learning model 41 to estimate (generate) a captured image (estimated image) that would be obtained when the part D is captured under one lighting condition from the captured image of the part D captured under another lighting condition (ST2: image estimation process). The generated estimated image is stored in the memory unit 31 as estimated image data 42.

In the example of FIG. 5, the image estimation unit 35 uses the learning model 41 to generate an estimated image that would be obtained if part D99 were captured under lighting condition 6 and lighting condition 7 from the captured image of part D99 captured under lighting condition 1 included in the captured image data 40 (ST2), and stores the image in the estimated image data 42.

In FIG. 4, the input reception unit 36 displays an illumination condition input screen on the touch panel 15 for inputting illumination conditions, and receives input of the illumination conditions when the image estimation unit 35 generates an estimated image.

Now, referring to FIG. 6, an example of the lighting condition input screen 50 displayed on the touch panel 15 will be described. On the lighting condition input screen 50, an image display frame 51, an image information display frame 52, a lighting condition input frame 53, an automatic adjustment button 54, an image capture button 55, a verification button 56, and a decision button 57 are displayed. On the image display frame 51, a captured image of the part D captured by the imaging means C, or an estimated image of the part D generated by the image estimation unit 35 is displayed. On the image information display frame 52, if the image displayed in the image display frame 51 is an image captured by the imaging means C, "captured image" is displayed, and if it is an image generated by the image estimation unit 35, "estimated image" is displayed.

The lighting condition input frame 53 displays an illumination value display section 53a that displays the illumination values set for the reflective lighting section 24, the transmitted lighting section 25, and the side lighting section 26, an increase button 53b that increases the illumination value, and a decrease button 53c that decreases the illumination value. When the operator operates the increase button 53b, the illumination value displayed in the illumination value display section 53a increases by a predetermined amount, and when the decrease button 53c is operated, the illumination value decreases by a predetermined amount. The operator operates the increase button 53b or decrease button 53c to set the illumination value between "0" and "100".

The image display frame 51 in Figure 6 displays an image captured by the imaging means C while the illumination means L is illuminating the component D99 held by the suction nozzle 9 under illumination condition 1 (reflection: 20, transmission: 0, side: 10). The illumination value display section 53a of the illumination condition input frame 53 displays the illumination value included in illumination condition 1 at the time of imaging. The image display frame 51 shows the main body Dc of the component D99 and the five electrodes Dd formed on the bottom surface of the main body. Under illumination condition 1, the amount of light irradiated from the reflection illumination section 24 and side illumination section 26 is insufficient, so the main body Dc and electrodes Dd of the component D99 are not clearly visible.

Figure 7 shows the lighting condition input screen 58 in which the lighting value has been changed by operating the increase button 53b or decrease button 53c on the lighting condition input screen 50 shown in Figure 6. On the lighting condition input screen 58, the lighting condition has been changed to lighting condition 6 (reflection: 70, transmission: 0, side 40). When the increase button 53b or decrease button 53c is operated on the lighting condition input screen 58 to change the lighting value, the image estimation unit 35 generates an estimated image of the part D99 illuminated by the lighting means L under the lighting conditions set by the imaging means C. The generated estimated image is displayed in the image display frame 51, and "Estimated Image" is displayed in the image information display frame 52.

The process of generating an estimated image by the image estimation unit 35 is completed in an extremely short time, so when the worker operates the increase button 53b or the decrease button 53c to change the lighting value, the estimated image of the changed lighting conditions is immediately displayed in the image display frame 51. That is, each time the worker operates the increase button 53b or the decrease button 53c, the estimated image displayed in the image display frame 51 is updated to one of the changed lighting conditions. In this way, the touch panel 15 is a display unit that displays another captured image (estimated image) estimated by the image estimation unit 35 based on the input lighting condition 6 (another lighting condition).

In FIG. 7, the operator operates the increase button 53b or decrease button 53c while viewing the estimated image displayed in the image display frame 51 to search for lighting conditions under which the body part Dc and electrodes Dd of the part D99 are clearly visible. When the automatic adjustment button 54 is operated, the lighting condition creation unit 37 automatically searches for lighting conditions (combinations of lighting values) that maximize the contrast difference between the body part Dc of the part D99 and the background, and between the body part Dc and the electrodes Dd. The determined lighting value is then displayed in the lighting value display unit 53a, and an estimated image under the determined lighting conditions is displayed in the image display frame 51. Note that the method by which the lighting condition creation unit 37 searches for lighting conditions that are considered optimal is not limited to the method that maximizes the contrast difference, and various methods are possible.

In FIG. 7, when the image capture button 55 is operated, the image capture processing unit 33 scans the component D99 held by the suction nozzle 9 with the image capture means C under illumination conditions set in the illumination condition input screen 58 by the illumination means L under illumination conditions 6. When the image capture is completed, the captured image of the component D99 is displayed in the image display frame 51. On the illumination condition input screen 58, the user can operate the increase button 53b or decrease button 53c while viewing the estimated image displayed in the image display frame 51, or operate the automatic adjustment button 54 to input the illumination conditions that are considered optimal, and then operate the image capture button 55 to check the image of the component D99 that was actually captured under those illumination conditions, thereby efficiently determining whether the illumination conditions are acceptable in a short time.

Figure 8 shows an illumination condition input screen 59 in which the verification button 56 is operated on the illumination condition input screen 58 shown in Figure 7 and the illumination condition feasibility determination unit 38 executes the illumination condition feasibility determination process. In the illumination condition feasibility determination process, it is determined whether the position and orientation of the component D99 and the position of the electrode Dd can be correctly recognized from the captured image captured by the imaging means C when the illumination means L illuminates the component D99 under the illumination condition 6 set in the illumination condition input frame 53.

When the verification button 56 is operated, the illumination condition determination unit 38 first causes the illumination means L to illuminate the component D99 under the illumination condition 6 set in the illumination condition input frame 53, rotates the component D99 held by the suction nozzle 9 around the Z axis at a predetermined angle (30 degrees in this case), and causes the imaging means C to scan and image the component. The captured image is displayed in the image display frame 51, and "Captured Image" is displayed in the image information display frame 52.

In FIG. 8, the illumination condition feasibility determination unit 38 then performs image processing on the captured image to extract the center position of the main body Dc of the component D99 and the center position of the electrode Dd, and determines whether the extracted positions are as expected. If the extracted center position of the main body Dc and the center position of the electrode Dd are within a predetermined range from the expected positions, the illumination condition feasibility determination unit 38 determines that the illumination conditions are adoptable. In the captured image of the component D99 displayed in the image display frame 51, a cross mark is superimposed on the extracted center position of the main body Dc and the center position of the electrode Dd. In the example of FIG. 8, the illumination conditions are determined to be adoptable, and the determination result "OK" is displayed in the image display frame 51.

In this way, the lighting condition feasibility determination unit 38 determines whether lighting condition 6 is acceptable based on the captured image captured by the imaging means C while changing the posture (rotating 30 degrees around the Z axis) of component D99 (one component) illuminated by the illumination means L under lighting condition 6 (another lighting condition). This makes it possible to confirm in advance whether component D99 can be correctly recognized even if the holding posture deviates within a specified range from the normal state when the component recognition camera 20 images component D99 under lighting condition 6 during component mounting work to recognize the holding posture.

In FIG. 8, when the decision button 57 is operated on the lighting condition input screen 59, the lighting condition creation unit 37 creates lighting conditions suitable for the lighting means L to illuminate the part D99 (one part) based on the lighting condition 6 set in the lighting condition input frame 53. The lighting condition creation unit 37 associates information identifying the part D99 with the created lighting conditions and stores them in the memory unit 31 as lighting condition data 43. That is, the lighting condition creation unit 37 creates lighting conditions suitable for the lighting means L to illuminate the part D99 based on the captured image (estimated image) of D99 (one part) estimated by the image estimation unit 35.

Specifically, when creating lighting conditions to be used for the lighting means L that illuminates the component D99 held by the suction nozzle 9 of the mounting head 8 imaged by the imaging means C of the component mounting device M1, the lighting condition creation unit 37 stores the lighting condition 6 set in the lighting condition input frame 53 as it is as the lighting condition data 43. When creating lighting conditions to be used by the component mounting device M2, the lighting condition creation unit 37 creates the lighting conditions based on information on the difference in lighting conditions between the component recognition cameras 20 of the component mounting devices M1 and M2 contained in the inter-device correction data 44 and the lighting condition 6 set in the lighting condition input frame 53.

For example, if the illumination value of the reflective illumination unit 24 is stored as "-10" in the inter-device correction data 44 as the correction value of the component recognition camera 20 of the component mounting device M1 and the component mounting device M2, the illumination condition creation unit 37 creates illumination condition 7 (reflection: 60, transmission: 0, side: 40) in which the illumination value of the reflective illumination unit 24 is corrected to -10 as the illumination condition, and stores it in the memory unit 31 of the component mounting device M2 as the illumination condition data 43. In other words, the inter-device correction data 44 includes information that corrects the characteristic difference between the illumination means L and the imaging means C of the component recognition camera 20 equipped in the component mounting devices M1 to M3 that has been acquired in advance. This allows the component mounting device M1 to create the illumination conditions of the component recognition camera 20 of the other component mounting device M2 or component mounting device M3.

Next, a captured image estimation method for estimating a captured image of component D will be described with reference to FIG. 7 along the flow of FIG. 9. Hereinafter, the same steps as those in the captured image estimation process described with reference to FIG. 5 will be given the same reference numerals, and detailed description will be omitted. First, the imaging processing unit 33 causes the imaging unit C to scan and image the component D illuminated by the lighting unit L under multiple lighting conditions, and creates learning data that associates information identifying the captured component D and the lighting conditions at the time of imaging with the captured image (ST11: first imaging step). The learning data is stored as captured image data 40. The imaging processing unit 33 also stores the captured image of component D captured during component mounting work as learning data.

Next, a learning process (ST1) is executed, and a learning model 41 is generated based on the learning data included in the captured image data 40. That is, the learning model 41 is generated based on the multiple captured images obtained in the first imaging process (ST11) and the lighting conditions at the time of imaging. Next, the imaging processing unit 33 causes the imaging means C to scan and image the part D99 (one part) illuminated by the lighting means L under one lighting condition 1 (ST12: second imaging process). Next, the input receiving unit 36 inputs the lighting condition 6 (lighting value) by operating the increase button 53b or decrease button 53c on the lighting condition input screen 58 (see FIG. 7) displayed on the touch panel 15 (ST13: lighting condition input process).

9, when the illumination condition 6 is input, the image estimation process (ST2) is executed. That is, using the generated learning model 41, another captured image (estimated image) obtained when the illumination means L captures the component D99 illuminated under another illumination condition 6 by the imaging means C is estimated from the captured image of D99 (one component) obtained by the second imaging process (ST12) (see FIG. 7). Next, until any of the capture button 55, the verification button 56, and the decision button 57 is operated on the illumination condition input screen 58 (No in ST14), the illumination condition input process (ST13) is returned to and the image estimation process (ST2) is repeatedly executed. In this way, the captured image estimation method of this embodiment can estimate captured images of the component D99 under multiple illumination conditions in a short time based on one captured image of the component D99 captured under illumination condition 1.

Next, following the flow of Figure 10, and with reference to Figures 7 and 8, a method for creating lighting conditions to be used when the lighting means L illuminates the part D imaged by the imaging means C will be described. Hereinafter, the same steps as those in the captured image estimation process described with reference to Figure 5 and the captured image estimation method described with reference to Figure 9 will be given the same reference numerals, and detailed descriptions will be omitted. The first imaging step (ST11) to the image estimation step (ST2) are the same as those in the captured image estimation method, and descriptions will be omitted.

When the image capture button 55 is operated (operating the "photograph button" in ST21) while the estimated captured image (another captured image) of part D99 in the image estimation process (ST2) is displayed in the image display frame 51 of the illumination condition input screen 58 shown in FIG. 7, the imaging processing unit 33 causes the imaging means C to scan and capture the part D99 held by the suction nozzle 9 under illumination conditions set by the illumination means L under the illumination condition 6 set on the illumination condition input screen 58 (ST22: part imaging process).

In FIG. 10, when the verification button 56 is operated on the lighting condition input screen 58 (operating the "Verify button" in ST21), the lighting condition determination unit 38 causes the lighting means L to illuminate the part D99 under lighting condition 6, and rotates the part D99 held by the suction nozzle 9 at a predetermined angle around the Z axis (ST23: part rotation step). Next, the lighting condition determination unit 38 causes the imaging means C to scan and image the part D99 (ST24: third imaging step). That is, in the third imaging step (ST24), the lighting means L illuminates the part D99 (one part) under lighting condition 6 (another lighting condition), and the imaging means C images the part D99 with its posture changed.

Then, the illumination condition feasibility determination unit 38 performs image processing on the captured image to extract the position of the center of the main body Dc of the component D99 and the position of the center of the electrode Dd, and determines whether the extracted positions are as expected (ST25: illumination condition feasibility determination process) (see FIG. 8). That is, based on the captured image obtained in the third imaging process (ST24), the feasibility of illumination condition 6 (another illumination condition) is determined.

In FIG. 10, after the part imaging process (ST22) or the lighting condition feasibility determination process (ST25), when the decision button 57 is operated on the lighting condition input screen 59 (see FIG. 8) (Yes in ST26), the lighting condition creation unit 37 creates lighting conditions suitable for the lighting means L to illuminate the part D99 based on another captured image of the part D99 (one part) estimated in the image estimation process (ST2) (ST27: lighting condition creation process). Until any of the imaging button 55, verification button 56, or decision button 57 is operated on the lighting condition input screen 59 (No in ST26), the process returns to the lighting condition input process (ST13) and the image estimation process (ST2) is repeatedly executed.

Also, when the decision button 57 is operated on the lighting condition input screen 58 (operating the "decision button" in ST21), the lighting condition creation process (ST27) is executed. In the lighting condition creation process (ST27), the lighting condition creation unit 37 creates lighting conditions to be used by the component mounting device M1, and can also use the inter-device correction data 44 to create lighting conditions to be used by the other component mounting devices M2 and M3.

That is, in the illumination condition creation step (ST27), illumination conditions are also created for the illumination means L of the other component mounting devices M2 and M3 that are different from the illumination means L of the component mounting device M1 that illuminated D99 (one component) in the second imaging step (ST12). In this way, the illumination condition creation method of this embodiment can create appropriate illumination conditions in a short time by estimating captured images of component D99 under multiple illumination conditions based on one captured image of component D99 captured under illumination condition 1.

As described above, the component mounting devices M1 to M3 of the present embodiment are a lighting condition creation support device that supports the creation of lighting conditions for the lighting means L that illuminates the component D imaged by the imaging means C, and includes a learning unit 34 that generates a learning model 41 based on multiple captured images captured by the imaging means C of the component D illuminated by the lighting means L under multiple lighting conditions and the lighting conditions at the time of capturing images (captured image data 40), and an image estimation unit 35 that uses the generated learning model 41 to estimate another captured image (estimated image) that is obtained when the imaging means C captures a component illuminated by the lighting means L under another lighting condition from one captured image captured by the imaging means C of a component (component D99) illuminated by the lighting means L under one lighting condition. This allows captured images of the component D99 under multiple lighting conditions to be acquired (estimated) in a short time, and lighting conditions can be appropriately created.

The lighting condition creation support device, which includes a learning unit 34 and an image estimation unit 35 and supports the creation of lighting conditions for the lighting means L that illuminates the component D imaged by the imaging means C, is not limited to the component mounting devices M1 to M3, and may be configured with the management computer 3 or the offline camera device 4. For example, in the case of the offline camera device 4, multiple captured images are acquired by the imaging means C of the component D illuminated by the lighting means L under multiple lighting conditions using the offline camera included in the offline camera device 4. The inter-device correction data 44 also includes information regarding the difference in lighting conditions between the offline camera device and the lighting means L included in the component mounting devices M1 to M3.

In the case of the management computer 3, the learning unit 34 generates the learning model 41 using the captured images captured by the offline camera device 4 as learning data (teacher data). That is, the learning unit 34 of the management computer 3 generates the learning model 41 based on the lighting means L of the offline camera device 4, which is different from the lighting means L of the component mounting devices M1 to M3, illuminating the component D under multiple lighting conditions, and the multiple captured images captured by the imaging means C (offline camera) of the offline camera device 4, which is different from the imaging means C of the component mounting devices M1 to M3, and the lighting conditions at the time of capturing the captured images. Note that the learning unit 34 of the management computer 3 may generate the learning model 41 using the captured images (captured image data 40) captured by the component mounting devices M1 to M3 as learning data (teacher data).

The illumination condition creation support device, captured image estimation method, and illumination condition creation method of the present invention have the effect of being able to obtain captured images of components under multiple illumination conditions in a short period of time, and are useful in the field of mounting components on a circuit board.

3. Management computer (lighting condition creation support device)
4. Offline camera device (lighting condition creation support device)
8 Mounting head 15 Touch panel (display unit)
24 Reflective lighting unit (lighting unit)
25 Transmitted illumination unit (illumination unit)
26 Side lighting unit (lighting unit)
C: Imaging means D, D99: Component L: Illumination means M1 to M3: Component mounting device (illumination condition creation support device)

Claims (15)

1. A lighting condition creation support device that supports creation of lighting conditions for a lighting means that illuminates a part imaged by an imaging means, comprising:
a learning unit that generates a learning model based on a plurality of captured images of a component illuminated by a lighting unit under a plurality of lighting conditions and the lighting conditions at the time of capturing the captured images;
an image estimation unit that estimates, from one captured image of a part illuminated by the lighting means under one lighting condition and captured by the imaging means, another captured image that is obtained when the imaging means captures the one part illuminated by the lighting means under another lighting condition, using the generated learning model;
an input receiving unit that receives an input of the other lighting conditions;
a display unit that displays the other captured image estimated by the image estimation unit based on the other lighting condition that is input . 2. The lighting condition creation support device according to claim 1, further comprising: a lighting condition creation unit that creates lighting conditions suitable for a lighting means to illuminate the one part based on the other captured image of the one part estimated by the image estimation unit. 3. The lighting condition creation support device according to claim 1, further comprising an illumination condition suitability determination unit that determines suitability of the other lighting condition based on an image captured by an imaging means of the one component illuminated by the illumination means under the other lighting condition. 4. The lighting condition creation support device according to claim 3, wherein the lighting condition suitability determination unit determines suitability of the other lighting condition based on an image captured by the imaging means while changing an attitude of the one part illuminated by the illumination means under the other lighting condition. 3. The illumination condition creation support device according to claim 2 , wherein the illumination condition creation unit creates illumination conditions used for illumination means that illuminates a component held by a mounting head that is imaged by imaging means included in the component mounting device. 6. The lighting condition creation support device according to claim 5, wherein the learning unit generates a learning model based on a plurality of captured images of components illuminated under a plurality of lighting conditions by a lighting means provided in the component mounting device, the captured images being captured by an imaging means provided in the component mounting device, and the lighting conditions at the time of capturing the captured images. The lighting condition creation support device according to claim 5, wherein the learning unit generates a learning model based on a plurality of captured images of components illuminated under a plurality of lighting conditions by a lighting means other than the lighting means provided in the component mounting device, the captured images being captured by an imaging means other than the imaging means provided in the component mounting device, and the lighting conditions at the time of capturing the captured images. The illumination means includes a plurality of illumination units for illuminating the component,
The illumination condition creation support device according to claim 1 , wherein the illumination condition is a combination of illumination values of the plurality of illumination units. a first imaging step of imaging the component illuminated by the illumination means under a plurality of illumination conditions with the imaging means;
A learning process of generating a learning model based on the plurality of captured images obtained by the first imaging process and the lighting conditions at the time of capturing the images;
a second imaging step of imaging the component illuminated by the illumination means under one illumination condition with the imaging means;
an image estimation step of estimating, from the captured image of the one component obtained in the second imaging step, another captured image that would be obtained when the imaging means captures an image of the one component illuminated by the illumination means under another lighting condition, using the generated learning model;
an input receiving step of receiving an input of the other lighting conditions;
a display step of displaying the other captured image estimated in the image estimation step based on the other lighting condition that has been input . The illumination means includes a plurality of illumination units for illuminating the component,
The captured image estimating method according to claim 9 , wherein the illumination condition is a combination of illumination values of the plurality of illumination units. 1. A method for creating lighting conditions for use when a lighting unit illuminates a part imaged by an imaging unit, comprising:
a first imaging step of imaging the component illuminated by the illumination means under a plurality of illumination conditions with the imaging means;
A learning process of generating a learning model based on the plurality of captured images obtained by the first imaging process and the lighting conditions at the time of capturing the images;
a second imaging step of imaging, by the imaging means, one component illuminated by the illumination means under one illumination condition;
an image estimation step of estimating, from the one captured image obtained in the second imaging step, another captured image that is obtained when the imaging means captures the one component illuminated by the illumination means under another illumination condition, using the learning model generated in the learning step;
an illumination condition creating step of creating an illumination condition suitable for illumination of the one component by an illumination means based on the other captured image of the one component estimated by the image estimation step;
an input receiving step of receiving an input of the other lighting conditions;
a display step of displaying the other captured image estimated in the image estimation step based on the other lighting condition that has been input . a third imaging step of imaging the one component illuminated by the illumination means under the different illumination condition with the imaging means;
12. The illumination condition creating method according to claim 11 , further comprising an illumination condition suitability determining step of determining suitability of the other illumination condition based on the captured image obtained in the third imaging step. The illumination condition creating method according to claim 12 , wherein in the third imaging step, an image of the one component is taken with a changed attitude. The illumination means includes a plurality of illumination units for illuminating the component,
The illumination condition creating method according to claim 11 , wherein the illumination condition is a combination of illumination values of the plurality of illumination units. 15. The illumination condition creating method according to claim 11 , wherein in the illumination condition creating step, an illumination condition for an illumination means different from the illumination means that illuminated the one component in the second imaging step is created.

Images