JP2022118301A - Lighting Condition Creation Support Device, Captured Image Estimation Method, and Lighting Condition Creation Method - Google Patents

Abstract

To provide a lighting condition creation support device, a captured image estimation method, and a lighting condition creation method that can acquire in a short time captured images of a component under a plurality of lighting conditions.SOLUTION: A captured image estimation method includes: a first imaging step of capturing images of a component lit by lighting means under a plurality of lighting conditions with imaging means; a learning step (ST1) of generating a learning model 41 on the basis of the plurality of captured images obtained in the first imaging step and the lighting conditions (captured image data 40) at the time of imaging; a second imaging step of imaging one component D99 lit by the lighting means under one lighting condition 1 with the imaging means; and an image estimation step (ST2) of estimating other captured images (estimated image data 42) obtained when the imaging means images the one component D99 lit by the lighting means under other lighting conditions 6 and 7 from the captured image (captured image data 40) of the component D99 obtained in the second imaging step using the generated learning model 41.SELECTED DRAWING: Figure 5

Description

The present invention relates to an illumination condition creation support device, a captured image estimation method, and an illumination condition creation method for assisting creation of illumination conditions for illumination means for illuminating parts.

A component mounting apparatus picks up a component supplied by a component supply device (tape feeder, etc.) with a nozzle, and images the component held by the nozzle with a camera (image capturing means) while illuminating the component with a lighting means to determine the displacement of the component with respect to the nozzle. After recognizing the holding state of the parts such as the nozzle, the parts held by the nozzle are mounted on the board. Since the optimum illumination conditions for the lighting means to illuminate the parts differ from part to part due to the material, shape, etc. of the parts, work is performed to set the optimum illumination conditions before starting production.

In Patent Document 1, when a captured image of a component captured by a camera during production of a mounting board is determined to be defective, the operator changes the lighting conditions to cause the camera to re-capture the component, which is displayed on the operation screen. Also disclosed is a component mounting apparatus (electronic component mounting apparatus) that shifts to an arrange mode for resetting illumination conditions for each of illumination means composed of four types of illumination units while viewing a captured image.

JP 2004-186175 A

However, in the prior art including Patent Document 1, it takes several seconds for the camera to pick up an image of the part, and furthermore, in order to determine the optimum lighting conditions for the lighting means composed of a plurality of lighting units, it is necessary to use a plurality of lighting units. Since it is necessary to capture images under a combination of conditions, it takes a long time to capture images of a large number of components mounted on a single mounting board under multiple lighting conditions, and to set the optimal lighting conditions based on the captured images. There was a problem that the work of

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an illumination condition creation support device, a captured image estimation method, and an illumination condition creation method that can acquire captured images of a component under a plurality of illumination conditions in a short time.

An illumination condition creation support device of the present invention is an illumination condition creation support device for assisting creation of illumination conditions for an illumination means for illuminating a part to be imaged by an imaging means, wherein the illumination means illuminates the part under a plurality of illumination conditions. a learning unit that generates a learning model based on a plurality of captured images captured by an imaging unit and an illumination condition at the time of capturing; Image estimation for estimating another captured image obtained when the imaging means captures an image of the one part illuminated by the illumination means under another illumination condition from one captured image captured by the imaging means of the part and

A captured image estimating method of the present invention includes a first imaging step of capturing an image of a part illuminated by an illumination means under a plurality of illumination conditions with an imaging means, a plurality of captured images obtained by the first imaging step and the illumination at the time of imaging. a learning step of generating a learning model based on conditions; a second imaging step of capturing an image of one part illuminated by the lighting means under one lighting condition with the imaging means; and using the generated learning model. , from the captured image of the one component obtained by the second imaging step, another captured image obtained when the imaging means captures the image of the one component illuminated by the illumination means under another illumination condition; and an image estimation step of estimating.

An illumination condition creation method of the present invention is a method for creating illumination conditions used when an illumination means illuminates a component imaged by an imaging means, wherein the illumination means creates a component illuminated under a plurality of illumination conditions. a first imaging step of imaging with an imaging means; a learning step of generating a learning model based on a plurality of captured images obtained by the first imaging step and illumination conditions at the time of imaging; A second imaging step of capturing an image of one part illuminated under a condition with the imaging means; an image estimating step of estimating another captured image obtained when the imaging means captures an image of the one part illuminated by the lighting means under another illumination condition; and an illumination condition creating step of creating an illumination condition suitable for illumination means to illuminate the one component based on the other captured image.

ADVANTAGE OF THE INVENTION According to this invention, the captured image of components by several illumination conditions can be acquired in a short time.

1 is an explanatory diagram of the configuration of a component mounting system according to an embodiment of the present invention; FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of the configuration of a component mounting apparatus according to an embodiment of the present invention; FIG. 1 is an explanatory diagram of the configuration of a component recognition camera provided in a component mounting apparatus according to an embodiment of the present invention; 1 is a block diagram showing the configuration of a control system of a component mounting apparatus according to one embodiment of the present invention; FIG. Explanatory diagram of captured image estimation according to one embodiment of the present invention FIG. 4 is a diagram showing an example of a lighting condition input screen displayed on a touch panel included in a component mounting apparatus according to an embodiment of the present invention; FIG. 4 is a diagram showing an example of a lighting condition input screen displayed on a touch panel included in a component mounting apparatus according to an embodiment of the present invention; FIG. 4 is a diagram showing an example of a lighting condition input screen displayed on a touch panel included in a component mounting apparatus according to an embodiment of the present invention; 1 is a flowchart of a captured image estimation method according to an embodiment of the present invention; FIG. 2 is a flowchart of a lighting condition creation method according to an embodiment of the present invention;

An embodiment of the present invention will be described in detail below with reference to the drawings. The configuration, shape, and the like described below are examples for explanation, and can be changed as appropriate according to the specifications of the component mounting device, component recognition camera, management computer, and offline camera device. In the following, the same reference numerals are given to the corresponding elements in all the drawings, and redundant explanations are omitted. In FIG. 2 and a part described later, two axes perpendicular to each other in the horizontal plane are the X-axis in the substrate transfer direction (perpendicular to the paper surface in FIG. 2) and the Y-axis (horizontal direction in FIG. 2) perpendicular to the substrate transfer direction. is shown. In FIG. 2 and a part to be described later, the Z-axis (vertical direction in FIG. 2) is shown as the height direction orthogonal to the horizontal plane.

First, the configuration of a component mounting system 1 will be described with reference to FIG. The component mounting system 1 is configured by connecting a plurality of component mounting apparatuses M1 to M3. The component mounting apparatuses M1 to M3 are connected to a management computer 3 via a communication network 2. FIG. The management computer 3 not only manages the production of mounted boards by the component mounting system 1, but also supports the creation of illumination conditions used by the component mounting apparatuses M1 to M3, which will be described later. Note that the number of component mounting apparatuses M1 to M3 provided in the component mounting system 1 is not limited to three, and may be one, two, or four or more.

The component mounting system 1 includes an off-line camera device 4 that is not connected to the component mounting apparatuses M1-M3. The off-line camera device 4 includes an off-line camera having the same configuration as the component recognition camera 20 (see FIG. 3) provided in component mounting apparatuses M1 to M3, which will be described later. In the off-line camera device 4, images of new components to be mounted on the board are captured by the off-line camera prior to production of the mounting board. Images captured by the off-line camera are transmitted to the management computer 3 and used to create illumination conditions and the like used by the component mounting apparatuses M1 to M3. Alternatively, an image captured by the offline camera is processed by an information processing device provided in the offline camera device 4 to create illumination conditions used by the component mounting apparatuses M1 to M3.

Next, referring to FIG. 2, the configuration of component mounting apparatuses M1 to M3 will be described. The component mounting apparatuses M1 to M3 have the same configuration, and the component mounting apparatus M1 will be described below. The component mounting apparatus M1 has a function of mounting a component D on a board B. FIG. A substrate transport mechanism 6 provided on the upper surface of the base 5 transports the substrate B along the X-axis, positions it, and holds it. A head moving mechanism 7 provided above the base 5 moves a mounting head 8 detachably mounted via a plate 7a in horizontal directions (X-axis direction and Y-axis direction). A suction nozzle 9 is detachably attached to the lower end of the mounting head 8 .

A plurality of tape feeders 10 are mounted side by side along the X-axis on a feeder base 11a on top of a carriage 11 coupled to the base 5 on the side of the substrate transfer mechanism 6. As shown in FIG. A carrier tape 12 for storing components D to be supplied to the component mounting apparatus M1 is wound around a reel 13 and held on the carriage 11 . The carrier tape 12 inserted into the tape feeder 10 is pitch-fed, and the components D stored in the carrier tape 12 are sequentially supplied to the component supply port 10a provided at the top of the tape feeder 10 .

In FIG. 2, in the component mounting operation, the mounting head 8 is moved above the tape feeder 10 by the head moving mechanism 7, and the component D supplied to the component supply port 10a of the tape feeder 10 is vacuum-absorbed by the suction nozzle 9. to pick up (arrow a). The mounting head 8 holding the component D is moved above the board B held by the board transfer mechanism 6 by the head moving mechanism 7, and mounts the component D at a predetermined component mounting position Ba on the board B (arrow b ).

In FIG. 2, a substrate recognition camera 14 is attached to the plate 7a with its optical axis directed in the negative direction of the Z axis. The board recognition camera 14 is moved integrally with the mounting head 8 in the positive and negative directions of the X-axis and in the positive and negative directions of the Y-axis by the head moving mechanism 7 . As the mounting head 8 moves, the board recognition camera 14 moves above the board B positioned on the board transfer mechanism 6 and picks up an image of a board mark (not shown) provided on the board B. FIG.

A component recognition camera 20 is mounted on the upper surface of the base 5 between the board transfer mechanism 6 and the tape feeder 10, with the optical axis directed upward. The component recognition camera 20 captures an image of the holding posture of the component D held by the suction nozzle 9 when the suction nozzle 9 that has picked up the component D passes above. In the operation of mounting the component on the board B by the mounting head 8, the mounting position is corrected by taking into account the result of imaging the component D by the component recognition camera 20 and the result of imaging the board position by the board recognition camera .

In FIG. 2, a touch panel 15 operated by a worker is installed at a position where the worker works on the front surface of the component mounting apparatus M1. The touch panel 15 displays various kinds of information on its display section, and the operator inputs data and operates the component mounting apparatus M1 using operation buttons, etc., which are input sections displayed on the display section.

Next, the configuration of the component recognition camera 20 will be described with reference to FIG. The component recognition camera 20 includes an imaging unit 22 having an imaging element such as a one-dimensional CCD or a two-dimensional CMOS sensor inside a housing 21 . A ceiling plate 21 a made of sheet glass or the like is installed on the upper surface of the housing 21 . A lens 23 is arranged above the imaging unit 22 . The imaging unit 22 is installed with an optical axis 22a directed upward, and images the component D held by the suction nozzle 9 through the lens 23 and the ceiling plate 21a. That is, the imaging unit 22 and the lens 23 are imaging means C that images the component D from below.

Inside the housing 21, a reflective lighting unit 24, a transmissive lighting unit 25, and a side lighting unit 26, which are configured with a plurality of LED chips and the like, are installed. The light emitted from the reflected illumination unit 24 mainly illuminates the bottom surface Da of the component D. As shown in FIG. The light emitted from the transmission illumination unit 25 is mainly reflected by the flange 9a of the suction nozzle 9 and illuminates the component D from above. The light emitted from the side lighting section 26 mainly illuminates the bumps Db formed on the bottom surface Da of the component D and the like.

In FIG. 3 , the component recognition camera 20 has an illumination driving section 27 . The illumination drive unit 27 supplies currents for the reflected illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 to emit light. Illumination drive unit 27 controls reflection illumination unit 24, transmission illumination unit 25, and side illumination unit 26 based on illumination values included in illumination conditions transmitted from control device 30 (see FIG. 4) provided in component mounting apparatus M1. Change the current value of the current supplied to each. That is, the illumination condition is a combination of illumination values of the plurality of illumination units.

Illumination conditions are registered for each type of component D, and the intensity (brightness) of the light emitted from the reflected illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 corresponds to the imaged component D. changed by As described above, the reflected illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 are the illumination means L for illuminating the component D imaged by the imaging means C (the imaging unit 22 and the lens 23). That is, the illumination means L includes a plurality of illumination units (reflected illumination unit 24, transmitted illumination unit 25, side illumination unit 26) that illuminate the component D, and the plurality of illumination units emit light according to the illumination value. strength is changed.

In FIG. 3, the component mounting apparatus M1 picks up the component with the imaging means C while moving the suction nozzle 9 holding the component D in the horizontal direction (arrow c) in a state where the illumination means L irradiates light according to the illumination conditions. D is imaged (scanned). Note that the direction in which the component D is moved is appropriately changed according to the specifications of the mounting head 8 and the component recognition camera 20 . In this manner, scanning imaging for imaging while moving the component D requires a time of about several seconds.

Next, with reference to FIG. 4, the configuration of the control system of the component mounting apparatuses M1 to M3 will be described, focusing on the function of creating illumination conditions for the component recognition camera 20. FIG. The component mounting apparatuses M1 to M3 have the same configuration, and the component mounting apparatus M1 will be described below. The component mounting apparatus M1 includes a control device 30, a substrate transport mechanism 6, a head moving mechanism 7, a mounting head 8, a tape feeder 10, a substrate recognition camera 14, a touch panel 15, and a component recognition camera 20.

The control device 30 includes a storage unit 31 , a mounting control unit 32 , an imaging processing unit 33 , a learning unit 34 , an image estimation unit 35 , an input reception unit 36 , an illumination condition creation unit 37 and an illumination condition availability determination unit 38 . The storage unit 31 is a storage device, and stores mounting data 39, captured image data 40, learning model 41, estimated image data 42, illumination condition data 43, inter-device correction data 44, and the like. The mounting data 39 stores various types of information such as the component type and size of the component D to be mounted on the board B, the coordinates of the component mounting position Ba on the board B, and the like, for each board type of the mounted board to be produced.

In FIG. 4, the mounting control unit 32 picks up the component D supplied from the tape feeder 10 by the suction nozzle 9 of the mounting head 8 based on the mounting data 39, and mounts it at the component mounting position Ba on the board B. get the work done. The imaging processing unit 33 moves the suction nozzle 9 holding the component D above the component recognition camera 20 during the component mounting work, and moves the component D held by the suction nozzle 9 to the imaging means C (the imaging unit 22, the lens 23) scans and captures and recognizes the holding posture of the component D. FIG.

When scanning and imaging the component D, the imaging processing unit 33 controls the illumination driving unit 27 according to the illumination conditions included in the illumination condition data 43 so that the illumination means L (the reflection illumination unit 24, the transmission illumination unit 25, the side A predetermined amount of light is emitted from the side illumination unit 26). Further, the imaging processing unit 33 acquires a plurality of captured images by scanning and imaging the component D held by the suction nozzle 9 with the imaging unit C while the illumination unit L illuminates the component D under a plurality of illumination conditions. run. The imaging processing unit 33 associates the information specifying the imaged part D and the lighting conditions at the time of imaging with the captured image, and stores them in the storage unit 31 as the captured image data 40 . The learning data creation process is executed during spare time during component mounting work, during changeover work for changing the board type of the mounting board to be produced, or the like.

In FIG. 4, the learning unit 34 uses a plurality of captured images captured by the imaging unit C of the part D illuminated under a plurality of illumination conditions by the illumination unit L included in the captured image data 40 and the illumination conditions at the time of capturing as teacher data. , a learning model 41 for estimating a captured image of the component D, which will be described later, is generated by a learning algorithm using machine learning or the like. Neural networks (including deep learning using multi-layered neural networks), genetic programming, decision trees, Bayesian networks, support vector machines (SVM), etc. can be used as learning algorithms. The generated learning model 41 is stored in the storage unit 31 .

In this way, the learning unit 34 included in the component mounting apparatus M1 acquires a plurality of images captured by the imaging unit C included in the component mounting apparatus M1 of the component D illuminated under a plurality of illumination conditions by the illumination unit L included in the component mounting apparatus M1. A learning model 41 is generated based on the image and the lighting conditions at the time of imaging.

In FIG. 4, the image estimating unit 35 uses the generated learning model 41 to extract one part D illuminated by the illumination means L under one illumination condition from one captured image captured by the imaging means C by the illumination means L estimates (generates) another captured image (hereinafter referred to as an "estimated image") obtained when the imaging means C captures an image of one part D illuminated under another illumination condition. The image estimating unit 35 associates the generated estimated image with the information specifying the part D and the illumination conditions, and stores the result in the storage unit 31 as estimated image data 42 .

Here, a captured image estimation process for generating an estimated image will be described with reference to FIG. First, the learning unit 34 generates a learning model 41 based on the captured images of the component D captured under a plurality of lighting conditions included in the captured image data 40 and the lighting conditions at the time of imaging (ST1: learning step). The created learning model 41 is stored in the storage unit 31 . Next, the image estimation unit 35 uses the generated learning model 41 to obtain a captured image (estimated image ) is estimated (generated) (ST2: image estimation step). The generated estimated image is stored in the storage unit 31 as estimated image data 42 .

In the example of FIG. 5 , the image estimating unit 35 uses the learning model 41 to extract the image of the part D99 under the illumination conditions 6 and 7 from the captured image of the part D99 captured under the illumination condition 1 included in the captured image data 40 . is generated (ST 2 ) and stored in the estimated image data 42 .

In FIG. 4, the input reception unit 36 displays an illumination condition input screen for inputting illumination conditions on the touch panel 15, and accepts input of illumination conditions when the image estimation unit 35 generates an estimated image.

Here, an example of the lighting condition input screen 50 displayed on the touch panel 15 will be described with reference to FIG. An image display frame 51 , an image information display frame 52 , an illumination condition input frame 53 , an automatic adjustment button 54 , an imaging button 55 , a verification button 56 and an OK button 57 are displayed on the illumination condition input screen 50 . In the image display frame 51, the captured image of the component D captured by the imaging means C or the estimated image of the component D generated by the image estimating section 35 is displayed. In the image information display frame 52, if the image displayed in the image display frame 51 is an image captured by the imaging means C, "captured image" is displayed. image” is displayed.

The illumination condition input frame 53 includes an illumination value display portion 53a for displaying illumination values set for the reflected illumination portion 24, the transmitted illumination portion 25, and the side illumination portion 26, an increase button 53b for increasing the illumination value, an illumination value A decrease button 53c for decreasing is displayed. When the operator operates the increase button 53b, the illumination value displayed on the illumination value display portion 53a is increased by a predetermined amount, and when the decrease button 53c is operated, the illumination value is decreased by a predetermined amount. The operator operates the increase button 53b or the decrease button 53c to set the illumination value between "0" and "100".

In the image display frame 51 of FIG. 6, the imaging means C captures an image while the illumination means L illuminates the part D99 held by the suction nozzle 9 under illumination condition 1 (reflection: 20, transmission: 0, side 10). The captured image is displayed. The illumination value display portion 53a of the illumination condition input frame 53 displays the illumination value included in the illumination condition 1 at the time of imaging. The image display frame 51 shows the body portion Dc of the component D99 and five electrodes Dd formed on the bottom surface of the body portion. Under illumination condition 1, the amount of light emitted from the reflected illumination section 24 and the side illumination section 26 is insufficient, so the body portion Dc and the electrode Dd of the component D99 appear unclear.

FIG. 7 shows an illumination condition input screen 58 in which the illumination value is changed by operating the increase button 53b or the decrease button 53c on the illumination condition input screen 50 shown in FIG. On the illumination condition input screen 58, the illumination condition is changed to illumination condition 6 (reflection: 70, transmission: 0, side 40). When the illumination value is changed by operating the increase button 53b or the decrease button 53c on the illumination condition input screen 58, the image estimation unit 35 causes the part D99 illuminated by the illumination unit L under the set illumination condition to be captured by the imaging unit C. generates an estimated image captured by The generated estimated image is displayed in the image display frame 51, and "estimated image" is displayed in the image information display frame 52. FIG.

Since the generation processing of the estimated image by the image estimation unit 35 is completed in a very short time, when the operator operates the increase button 53b or the decrease button 53c to change the illumination value, the changed illumination condition is immediately displayed in the image display frame 51. is displayed. That is, each time the operator operates the increase button 53b or the decrease button 53c, the estimated image displayed in the image display frame 51 is updated to that of the changed illumination condition. As described above, the touch panel 15 is a display unit that displays another captured image (estimated image) estimated by the image estimation unit 35 based on the input illumination condition 6 (another illumination condition).

In FIG. 7, the operator operates the increase button 53b or the decrease button 53c while looking at the estimated image displayed in the image display frame 51, and searches for the illumination condition in which the body part Dc and the electrode Dd of the part D99 can be clearly seen. do. Further, when the automatic adjustment button 54 is operated, the illumination condition creating unit 37 creates an illumination condition (illumination value combination) automatically. Then, the determined illumination value is displayed in the illumination value display section 53a, and the estimated image of the determined illumination condition is displayed in the image display frame 51. FIG. It should be noted that the method of searching for the optimal lighting condition by the lighting condition generator 37 is not limited to the method of maximizing the contrast difference, and various methods can be employed.

In FIG. 7 , when the imaging button 55 is operated, the imaging processing unit 33 scans the component held by the suction nozzle 9 under the condition that the illumination means L illuminates under the illumination condition 6 set on the illumination condition input screen 58 . D99 is scanned and imaged by the imaging means C. FIG. When the imaging is completed, the captured image of the component D99 that has been captured is displayed in the image display frame 51. FIG. On the illumination condition input screen 58, while viewing the estimated image displayed in the image display frame 51, the user operates the increase button 53b or the decrease button 53c, or operates the automatic adjustment button 54 to input the optimal illumination condition. Furthermore, by operating the imaging button 55 and confirming the captured image of the component D99 actually captured under the lighting conditions, it is possible to efficiently determine the quality of the lighting conditions in a short period of time.

FIG. 8 shows an illumination condition input screen 59 after the verification button 56 has been operated on the illumination condition input screen 58 shown in FIG. In the illumination condition determination process, the illumination means L illuminates the component D99 under the illumination condition 6 set in the illumination condition input frame 53, and the imaged image captured by the imaging means C is used to determine the position and orientation of the component D99 and the electrode Dd. It is determined whether the position can be recognized correctly.

When the verification button 56 is operated, first, the illumination condition determination unit 38 causes the illumination unit L to illuminate the component D99 under the illumination condition 6 set in the illumination condition input frame 53, and the suction nozzle 9 holds the component D99. The part D99 is rotated around the Z axis by a predetermined angle (here, 30 degrees), and the imaging means C scans and images it. The picked-up image is displayed in the image display frame 51, and the image information display frame 52 displays "captured image".

In FIG. 8, the illumination condition determination unit 38 then performs image processing on the captured image to extract the center position of the body part Dc of the component D99 and the center position of the electrode Dd. Determine if the position is as expected. If the extracted center position of the main body Dc and the extracted center position of the electrode Dd are within a predetermined range from the expected position, the illumination condition propriety determination unit 38 determines that the illumination condition is acceptable. In the picked-up image of the component D99 displayed in the image display frame 51, a cross mark is superimposed on the center position of the extracted main body Dc and the center position of the electrode Dd. In the example of FIG. 8, it is determined that the lighting conditions are acceptable, and the image display frame 51 displays "OK" as the determination result.

In this way, the illumination condition propriety determination unit 38 changes the posture of the component D99 (one component) illuminated by the illumination means L under the illumination condition 6 (another illumination condition) (rotates 30 degrees around the Z axis). Based on the captured image captured by the imaging means C, it is determined whether the illumination condition 6 is acceptable. As a result, when the component recognition camera 20 picks up an image of the component D99 under illumination condition 6 in the component mounting work and recognizes the holding posture, the component D99 can be correctly recognized even if the holding posture deviates within a predetermined range from the normal state. You can check in advance whether it is possible or not.

In FIG. 8, when the decision button 57 is operated on the illumination condition input screen 59, the illumination condition creation unit 37 determines that the illumination means L is the part D99 (one Create lighting conditions suitable for illuminating The illumination condition creating unit 37 associates the created illumination condition with the information specifying the component D99, and stores the created illumination condition in the storage unit 31 as the illumination condition data 43. FIG. That is, the illumination condition creation unit 37 creates illumination conditions suitable for the illumination unit L to illuminate the component D99 based on the captured image (estimated image) of D99 (one component) estimated by the image estimation unit 35. do.

Specifically, the illumination condition creation unit 37 sets the illumination condition used by the illumination means L for illuminating the component D99 held by the suction nozzle 9 of the mounting head 8 imaged by the imaging means C provided in the component mounting apparatus M1. , the illumination condition 6 set in the illumination condition input frame 53 is stored as the illumination condition data 43 as it is. When creating the illumination conditions used by the component mounting apparatus M2, the illumination condition creating unit 37 sets the illumination conditions of the component recognition cameras 20 of the component mounting apparatuses M1 and M2 included in the inter-apparatus correction data 44. , and the illumination condition 6 set in the illumination condition input frame 53, the illumination condition is created.

For example, when the illumination value of the reflected illumination unit 24 is stored as "-10" as the correction value of the component recognition camera 20 of the component mounting apparatus M1 and the component mounting apparatus M2 in the inter-apparatus correction data 44, the illumination condition is created. The unit 37 creates illumination conditions 7 (reflection: 60, transmission: 0, side: 40) in which the illumination value of the reflected illumination unit 24 is corrected to -10, and stores them in the storage unit 31 of the component mounting apparatus M2. It is stored as lighting condition data 43 . That is, the inter-apparatus correction data 44 includes information for correcting the characteristic difference between the lighting means L and the imaging means C of the component recognition cameras 20 provided in the component mounting apparatuses M1 to M3, which has been acquired in advance. As a result, in the component mounting apparatus M1, illumination conditions for the component recognition camera 20 of the other component mounting apparatus M2 or the component mounting apparatus M3 can be created.

Next, along the flow of FIG. 9, a captured image estimation method for estimating the captured image of the component D will be described with reference to FIG. Hereinafter, steps that are the same as those of the captured image estimation processing described with reference to FIG. First, the image pickup processing unit 33 causes the image pickup unit C to scan and image a component D illuminated under a plurality of illumination conditions by the illumination unit L, and converts information specifying the imaged component D and the illumination conditions at the time of image pickup into a captured image. Associated learning data is created (ST11: first imaging step). The learning data is stored as captured image data 40 . Note that the imaging processing unit 33 also stores captured images of the component D captured during component mounting work as learning data.

Next, a learning step (ST1) is performed, and a learning model 41 is generated based on learning data included in the captured image data 40. FIG. That is, the learning model 41 is generated based on the plurality of captured images obtained in the first imaging step (ST11) and the lighting conditions at the time of imaging. Next, the image pickup processing unit 33 causes the image pickup unit C to scan and image the component D99 (one component) illuminated by the illumination unit L under one illumination condition 1 (ST12: second image pickup step). Next, the illumination condition 6 (illumination value) is input by operating the increase button 53b or the decrease button 53c of the illumination condition input screen 58 (see FIG. 7) displayed on the touch panel 15 by the input reception unit 36 (ST13: illumination condition input process).

In FIG. 9, when illumination condition 6 is input, the image estimation step (ST2) is executed. That is, using the generated learning model 41, from the captured image of D99 (one component) obtained in the second imaging step (ST12), the illumination means L illuminates the component D99 under another illumination condition 6. Another captured image (estimated image) obtained when the image is captured by means C is estimated (see FIG. 7). Next, until any one of the image capture button 55, verification button 56, and decision button 57 is operated on the illumination condition input screen 58 (No in ST14), the illumination condition input step (ST13) is returned to and the image estimation step (ST2) is performed. Executed repeatedly. As described above, the captured image estimation method of the present embodiment estimates captured images of the component D99 under a plurality of illumination conditions in a short time based on one captured image of the component D99 captured under the illumination condition 1. can be done.

Next, along the flow of FIG. 10, an illumination condition creation method for creating illumination conditions used when the illumination means L illuminates the component D imaged by the imaging means C will be described with reference to FIGS. 7 and 8. . Hereinafter, the same steps as those of the captured image estimation process described with reference to FIG. 5 and the captured image estimation method described with reference to FIG. The first imaging step (ST11) to the image estimating step (ST2) are the same as the captured image estimating method, and description thereof will be omitted.

The imaging button 55 is operated while the captured image (another captured image) of the component D99 estimated in the image estimation step (ST2) is displayed in the image display frame 51 of the illumination condition input screen 58 shown in FIG. Then (the "shooting button" is operated in ST21), the imaging processing unit 33 captures the component held by the suction nozzle 9 under the condition that the illumination means L illuminates under the illumination condition 6 set on the illumination condition input screen 58. D99 is scanned and imaged by the imaging means C (ST22: component imaging step).

In FIG. 10, when the verification button 56 is operated on the illumination condition input screen 58 (the "verification button" is operated in ST21), the illumination condition propriety determination unit 38 causes the illumination means L to illuminate the part D99 under the illumination condition 6. , the component D99 held by the suction nozzle 9 is rotated at a predetermined angle around the Z-axis (ST23: component rotation step). Next, the illumination condition propriety determination unit 38 scans and images the component D99 by the imaging means C (ST24: third imaging step). That is, in the third imaging step (ST24), the illumination means L illuminates the part D99 (one part) under the illumination condition 6 (another illumination condition), and the part D99 whose posture is changed is imaged by the imaging means C. be done.

Next, the illumination condition determination unit 38 performs image processing on the captured image to extract the center position of the body part Dc of the component D99 and the center position of the electrode Dd. It is determined whether or not there is (ST25: illumination condition propriety determination step) (see FIG. 8). That is, based on the captured image obtained in the third imaging step (ST24), it is determined whether or not illumination condition 6 (another illumination condition) is possible.

In FIG. 10, after the component imaging step (ST22) or after the illumination condition propriety determination step (ST25), when the determination button 57 is operated on the illumination condition input screen 59 (see FIG. 8) (Yes in ST26). , the illumination condition creation unit 37 determines illumination conditions suitable for the illumination means L to illuminate the part D99 based on another captured image of the part D99 (one part) estimated in the image estimation step (ST2). create (ST27: illumination condition creation step). Until any one of the image capturing button 55, verification button 56, and decision button 57 is operated on the lighting condition input screen 59 (No in ST26), the process returns to the lighting condition input step (ST13) and the image estimation step (ST2) is repeated. executed.

Further, when the enter button 57 is operated on the illumination condition input screen 58 (operating the "enter button" in ST21), the illumination condition creation step (ST27) is also executed. In the illumination condition creation step (ST27), the illumination condition creating unit 37 creates illumination conditions to be used in the component mounting apparatus M1, and also uses the inter-apparatus correction data 44 to create illumination conditions for use in the other component mounting apparatuses M2 and M3. You can also create lighting conditions that

That is, in the illumination condition creation step (ST27), other component mounting apparatuses M2 and M3, which are different from the illumination means L provided in the component mounting apparatus M1 that illuminated D99 (one component) in the second imaging step (ST12), are provided. Illumination conditions for the illumination means L are also created. As described above, the illumination condition creation method according to the present embodiment estimates captured images of the component D99 under a plurality of illumination conditions based on one captured image of the component D99 captured under the illumination condition 1. Lighting conditions can be created in a short time.

As described above, the component mounting apparatuses M1 to M3 of the present embodiment provide a plurality of captured images captured by the imaging means C of the component D illuminated under a plurality of illumination conditions by the illumination means L and the illumination conditions ( A learning unit 34 that generates a learning model 41 based on the captured image data 40), and using the generated learning model 41, one part (part D99) illuminated under one lighting condition by the lighting means L is imaged. An image estimating unit for estimating another captured image (estimated image) obtained when the imaging means C captures an image of one part illuminated by the illumination means L under another illumination condition from one captured image captured by the means C. 35, and assists the creation of the illumination conditions of the illumination means L for illuminating the part D imaged by the imaging means C. This makes it possible to acquire (estimate) captured images of the component D99 under a plurality of illumination conditions in a short time, and appropriately create the illumination conditions.

Note that the illumination condition creation support device that includes the learning unit 34 and the image estimation unit 35 and assists creation of the illumination conditions of the illumination means L that illuminates the component D imaged by the imaging means C is limited to the component mounting apparatuses M1 to M3. However, it may be configured by the management computer 3 or by the offline camera device 4 . For example, in the case of the offline camera device 4, the off-line camera of the offline camera device 4 acquires a plurality of captured images captured by the imaging means C of the part D illuminated by the lighting means L under a plurality of lighting conditions. The inter-apparatus correction data 44 also includes information on the difference in illumination conditions of the illumination means L provided in the off-line camera apparatus and the component mounting apparatuses M1 to M3.

In the case of the management computer 3, the learning unit 34 generates the learning model 41 by using the image captured by the offline camera device 4 as learning data (teacher data). That is, the learning unit 34 provided in the management computer 3 illuminates the component D under a plurality of lighting conditions with the lighting means L provided in the off-line camera device 4, which is different from the lighting means L provided in the component mounting apparatuses M1 to M3. A learning model 41 is generated based on a plurality of captured images captured by the imaging means C (offline camera) provided in the offline camera device 4 different from the imaging means C provided in M1 to M3 and the lighting conditions at the time of imaging. The learning unit 34 included in the management computer 3 may generate the learning model 41 using captured images (captured image data 40) captured by the component mounting apparatuses M1 to M3 as learning data (teacher data).

The illumination condition creation support device, captured image estimation method, and illumination condition creation method of the present invention have the effect of being able to acquire captured images of a component under a plurality of illumination conditions in a short time, and mount the component on a board. useful in the field.

3 Management computer (illumination condition creation support device)
4 Off-line camera device (illumination condition creation support device)
8 mounting head 15 touch panel (display unit)
24 reflected illumination unit (illumination unit)
25 transmitted illumination section (illumination section)
26 side illumination unit (illumination unit)
C imaging means D, D99 parts L lighting means M1 to M3 parts mounting device (lighting condition creation support device)

Claims (16)

An illumination condition creation support device for assisting creation of an illumination condition of an illumination means for illuminating a part imaged by an imaging means,
a learning unit that generates a learning model based on a plurality of captured images captured by an imaging unit of a component illuminated under a plurality of illumination conditions by the illumination unit and the illumination conditions at the time of imaging;
Using the generated learning model, the one part illuminated under one illumination condition by the illumination means is illuminated under another illumination condition from one captured image captured by the imaging means. and an image estimating unit for estimating another captured image obtained when the image capturing means captures the image of the part of (1). an input reception unit that receives input of the other lighting conditions;
2. The illumination condition creation support device according to claim 1, further comprising a display unit that displays said other captured image estimated by said image estimation unit based on said input other illumination condition. An illumination condition creation unit that creates illumination conditions suitable for the illumination means to illuminate the one part based on the other captured image of the one part estimated by the image estimation unit, further comprising: 3. The illumination condition creation support device according to claim 1. 2. The apparatus according to claim 1, further comprising an illumination condition propriety determining unit that determines propriety of said other lighting condition based on an image captured by said imaging means of said one component illuminated by said lighting means under said other lighting condition. 4. The illumination condition creation support device according to any one of 1 to 3. The illumination condition permission/inhibition determination unit determines the permission/inhibition of the other illumination condition based on the captured image captured by the imaging means by changing the orientation of the one part illuminated by the illumination means under the other illumination condition. 5. The illumination condition creation support device according to claim 4, which determines. 6. The illumination condition creation unit according to any one of claims 3 to 5, wherein the illumination condition creating unit creates an illumination condition used by an illumination means for illuminating a component held by a mounting head imaged by an imaging means provided in the component mounting apparatus. Lighting condition creation support device. The learning unit is configured to create a learning model based on a plurality of captured images of a component illuminated under a plurality of illumination conditions by an illumination unit provided in the component mounting apparatus and an illumination condition at the time of capturing. 7. The illumination condition creation support device according to claim 6, which generates a . The learning unit captures a plurality of captured images captured by an imaging means different from the imaging means provided in the component mounting apparatus and a component illuminated under a plurality of illumination conditions by an illumination means different from the illumination means provided in the component mounting apparatus. 7. The lighting condition creation support device according to claim 6, which generates a learning model based on the lighting conditions of the time. The illumination means includes a plurality of illumination units that illuminate the parts,
9. The illumination condition creation support device according to claim 1, wherein said illumination condition is a combination of illumination values of said plurality of illumination units. a first imaging step of capturing an image of the component illuminated by the illumination means under a plurality of illumination conditions;
a learning step of generating a learning model based on the plurality of captured images obtained in the first imaging step and the lighting conditions at the time of imaging;
a second imaging step of imaging one part illuminated by the illumination means under one illumination condition with the imaging means;
Using the generated learning model, the imaging means images the one part illuminated by the lighting means under another lighting condition from the imaged image of the one part obtained in the second imaging step. and an image estimation step of estimating another captured image obtained in the case. The illumination means includes a plurality of illumination units that illuminate the parts,
11. The captured image estimation method according to claim 10, wherein said illumination condition is a combination of illumination values of said plurality of illumination units. An illumination condition creating method for creating an illumination condition used when an illumination means illuminates a component imaged by an imaging means, comprising:
a first imaging step of capturing an image of the component illuminated by the illumination means under a plurality of illumination conditions;
a learning step of generating a learning model based on the plurality of captured images obtained in the first imaging step and the lighting conditions at the time of imaging;
a second imaging step of imaging one part illuminated by the illumination means under one illumination condition with the imaging means;
Using the learning model generated in the learning step, the image pickup means picks up the one part illuminated by the lighting means under another illumination condition from the one captured image obtained in the second image pickup step. an image estimation step of estimating another captured image obtained when
an illumination condition creating step of creating an illumination condition suitable for illumination means to illuminate the one component based on the other captured image of the one component estimated by the image estimation step; How to create lighting conditions. a third imaging step of imaging the one component illuminated by the illumination means under the other illumination condition with the imaging means;
13. The illumination condition creation method according to claim 12, further comprising an illumination condition propriety determination step of judging propriety of said other illumination condition based on the captured image obtained by said third imaging step. 14. The illumination condition creation method according to claim 13, wherein the one component whose posture is changed is imaged in the third imaging step. The illumination means includes a plurality of illumination units that illuminate the parts,
15. The illumination condition creation method according to claim 12, wherein said illumination condition is a combination of illumination values of said plurality of illumination units. 16. The illumination condition creating method according to any one of claims 12 to 15, wherein in said illumination condition creating step, an illumination condition is created for an illumination means different from the illumination means that illuminated said one component in said second imaging step. .

Images