JPWO2019155593A1 - Trained model creation system for component image recognition and trained model creation method for component image recognition - Google Patents

Abstract

When a component sucked on the suction nozzle (31) of the component mounting machine (12) or a component mounted on the circuit board (11) is set as an image pickup target, and the image pickup target is imaged by a camera (18, 22) to recognize an image. In the training model creation system for component image recognition for creating a trained model to be used, a computer (23) for acquiring a reference trained model used for image recognition of a reference component is provided. This computer collects sample part images for each type of part having a predetermined similarity relationship with the reference part, and adds the sample part image for each type of the part as teacher data of the reference trained model. By re-learning, a learned model for each part used for image recognition of the part is created for each type of the part.

Description

In the present specification, a component sucked on a suction nozzle of a component mounting machine or a component mounted on a circuit board is targeted for imaging, and a component for creating a learned model to be used when the imaging target is imaged by a camera and image recognition is performed. It discloses a technique for creating a trained model for image recognition and a method for creating a trained model for component image recognition.

As for the suction posture of the parts sucked to the suction nozzle of the parts mounting machine, if the parts are sucked normally, the parts are sucked horizontally, but for some reason, the parts are sucked in an abnormal posture such as diagonally. I have something to do. Since such abnormal adsorption causes defective component mounting, conventionally, the component mounting machine is equipped with a camera that captures the components adsorbed on the suction nozzle, and the image captured by the camera is processed. It is determined whether the suction posture of the parts is normal suction or abnormal suction, the parts judged to be abnormal suction are discarded, and only the parts judged to be normal suction are mounted on the circuit board.

In conventional general image processing, normal adsorption / abnormal adsorption is discriminated by using image processing component shape data including the size of the component, but the component adsorbed on the suction nozzle is a minute component. In some cases, it may be difficult to distinguish between normal adsorption and abnormal adsorption in image processing using conventional image processing component shape data.

Therefore, as described in Patent Document 1 (Japanese Unexamined Patent Publication No. 2008-1330865), a learned model for discriminating normal adsorption / abnormal adsorption is prepared in advance by using a machine learning method such as a neural network. , There is a device that processes a component image taken by a camera of a component mounting machine during production and discriminates between normal adsorption and abnormal adsorption using a learned model.

Japanese Unexamined Patent Publication No. 2008-1330865

For example, even parts with the same electrical specifications may differ in size, color, material, manufacturing company, manufacturing lot, etc., and the difference may cause differences in image recognition results. is there. However, before the start of production, for parts with the same electrical specifications, the types of parts are subdivided according to size, color, material, manufacturing company, manufacturing lot, etc., and all types have been learned by machine learning methods. If you try to create a trained model, you have to create a large number of trained models, and it takes a lot of time and effort to create the trained model.

Therefore, for parts of a type that are similar in shape to parts for which a trained model has already been created, normal adsorption / abnormal adsorption may be determined using an existing trained model for the time being. In that case, the expected discrimination accuracy may not be obtained during production. In this case, it is necessary to quickly create a trained model specialized for the component, but it takes time and effort to create a trained model from the beginning by the conventional method.

In order to solve the above problem, a trained model is used when a component sucked on a suction nozzle of a component mounting machine or a component mounted on a circuit board is imaged and the imaged object is imaged by a camera to recognize an image. In the trained model creation system for component image recognition to be created, a computer for acquiring a reference trained model used for image recognition of a reference component is provided, and the computer has a predetermined similarity relationship with the reference component. By collecting sample part images for each type of part, adding the sample part image for each type of part as teacher data of the reference-learned model, and re-learning, the part for each type of part. This is to create a trained model for each part used for image recognition.

In short, for parts that have a predetermined similarity to the reference parts for which the reference-learned model is created, sample part images are collected for each type of the part, and the sample parts are collected for each type of the part. By adding an image as teacher data of the reference-learned model and re-learning, a component-specific trained model used for image recognition of the component is created for each type of the component. In this way, it is relatively easy to create a component-specific trained model used for image recognition of a component having a predetermined similarity relationship with the reference component from the reference trained model.

FIG. 1 is a block diagram showing a configuration example of a component mounting line of one embodiment. FIG. 2 is a front view illustrating normal adsorption. FIG. 3 is a front view illustrating oblique adsorption. FIG. 4 is a flowchart showing the processing flow of the component suction posture determination program. FIG. 5 is a flowchart showing the processing flow of the trained model creation program for each part.

An embodiment will be described below.
First, the configuration of the component mounting line 10 will be described with reference to FIG.

In the component mounting line 10, one or a plurality of component mounting machines 12 and mounting related machines such as a solder printing machine 13 and a flux coating device (not shown) are arranged along the transport direction of the circuit board 11. It is configured. An inspection machine 14 for inspecting the quality of the mounting state of each component mounted on the circuit board 11 is installed on the board carry-out side of the component mounting line 10.

The component mounting machine 12, the solder printing machine 13, and the inspection machine 14 of the component mounting line 10 are connected to each other via the network 16 so as to be communicable with the production management computer 21, and the component mounting line is connected by the production management computer 21. 10 productions are controlled. The control device 17 of each component mounting machine 12 is mainly composed of one or a plurality of computers (CPUs), and is mounted head (shown) according to a production job (production program) transferred from the production management computer 21. The component (see FIGS. 2 and 3) supplied from the feeder 19 is moved to the suction nozzle 31 (see FIGS. 2 and 3) of the mounting head by moving the component suction position → component imaging position → component mounting position. ), The component is imaged from below by the component imaging camera 18, the captured image is processed by the image processing function of the control device 17 of the component mounting machine 12, and the learned model described later is used. It is determined whether the suction posture of the component is normal suction (see FIG. 2) or abnormal suction (see FIG. 3). As a result, if it is determined to be abnormal adsorption, the component is discarded in a predetermined disposal box (not shown), and if it is determined to be normal adsorption, the adsorption positions X and Y of the component and the angle θ are measured, and the component A predetermined number of components are mounted on the circuit board 11 by repeating the operation of correcting the deviations of the positions X and Y and the angle θ and mounting the components on the circuit board 11.

Further, the control device 20 of the inspection machine 14 is mainly composed of one or a plurality of computers (CPUs), and the mounting state of each component on the carried-in circuit board 11 is captured by the inspection camera 22 from above. Then, the captured image is processed to recognize the presence / absence of each component on the circuit board 11 and the mounting state such as the mounting position deviation, and based on the recognition result, the presence / absence of mounting failure (inspection failure) of each component. Inspect. At this time, the presence / absence of each component on the circuit board 11 may be determined by using the learned model described later.

A learning computer 23 that collects and learns teacher data (sample component images) used for creating a reference trained model and a component-specific trained model, which will be described later, is connected to the network 16 of the component mounting line 10.

The control device 17 of each component mounting machine 12 executes the component suction posture determination program of FIG. 4 described later during production to select a learned model according to the type of component sucked on the suction nozzle 31. From the processing result of the captured image of the component, it is determined whether the suction posture of the component is normal suction or abnormal suction, and the captured image determined to be normal suction is transferred to the learning computer 23 as a sample component image of normal suction. The captured image determined to be adsorbed is transferred to the learning computer 23 as a sample component image of abnormal adsorption.

On the other hand, the learning computer 23 executes the learning model creation program for each part of FIG. 5, which will be described later, to obtain a sample part image of normal adsorption / abnormal adsorption transferred from the control device 17 of each component mounting machine 12. In addition to classifying and collecting parts by type, the inspection result information of the inspection machine 14 is acquired, the mounting defect occurrence rate is calculated for each part type, and the mounting defect occurrence rate exceeds the judgment threshold value. If there is, the trained model for each part used for image recognition of the part by adding the sample part image of normal adsorption / abnormal adsorption collected for the part as the teacher data of the reference trained model and re-learning. Is created, and the trained model for each component is transferred to the control device 17 of each component mounting machine 12. As the re-learning method, a machine learning method such as a neural network or a support vector machine may be used.

Here, the reference trained model is a trained model used for image recognition of the reference component, and the learning computer 23 collects sample component images of normal adsorption / abnormal adsorption of the reference component as teacher data. Then, the reference-learned model may be created by learning by machine learning such as a neural network or a support vector machine, or the reference-learned model created by an external computer may be imported into the learning computer 23. .. Further, the reference parts are not limited to specific parts, and parts for which a pre-learned model has been created may be defined as "reference parts".

The control device 17 of each component mounting machine 12 associates the reference learned model and the learned model for each component transferred from the learning computer 23 with the type of component that recognizes an image using the model, and stores the storage device (shown). Remember in). At that time, the image processing component shape data prepared for each component type includes the reference-learned model or the component-specific trained model and stores it. In the following description, the term "trained model" includes both the standard trained model and the component-based trained model. The image processing component shape data is data that represents external features such as the size of the body portion of the component, the position of terminals such as bumps and leads, size, pitch, and number, and determines the type of image-recognized component. It is also used to measure the suction position and angle of parts. The process of including the trained model created for each component type in the image processing component shape data prepared for each component type may be performed by the control device 17 of each component mounting machine 12, or for learning. You may do it with the computer 23. Alternatively, the trained model is transferred from the learning computer 23 to the production management computer 21, and the production management computer 21 performs a process of including the trained model in the image processing part shape data for production management. The image processing component shape data including the trained model may be transferred from the computer 21 to the control device 17 of each component mounting machine 12.

The control device 17 of each component mounting machine 12 is used for the component if the learned model stored for each type of component includes a learned model used for image recognition of the component sucked by the suction nozzle 31. The trained model of the above is selected and the image recognition of the part is performed. However, if the trained model for the part does not exist, the part sucked to the suction nozzle 31 is selected from the parts having the trained model. The parts having a predetermined similar relationship are regarded as "reference parts", and the trained model for the reference parts is used as the "reference trained model" to perform image recognition of the parts sucked on the suction nozzle 31. Do. At this time, the trained model for the reference part may be a part-specific trained model created from the standard trained model for other parts. In this case, the trained model for other parts is used. The trained model for each part created from is used as the standard trained model.

In this case, the parts having a predetermined similar relationship are, for example, parts having the same or similar shapes even if any of the size, color, material, manufacturing company, manufacturing lot, etc. of the parts is different. .. If there is a certain similarity between the parts, even if the image recognition of the other part is performed using the trained model for one part, the image will be imaged with a certain degree of accuracy (generally, the accuracy above the minimum required for production). It is recognizable. In other words, if the trained model for one component can be used to perform image recognition of the other component with a certain degree of accuracy, it can be said that these two components have a predetermined similarity relationship.

Next, the processing flow of the component suction posture determination program of FIG. 4 and the learned model creation program for each component of FIG. 5 will be described.

[Part suction posture discrimination program]
The component suction posture determination program of FIG. 4 is performed by the control device 17 of each component mounting machine 12 at each timing when the component imaged by the component imaging camera 18 captures the component sucked by the suction nozzle 31 of each component mounting machine 12 during production. Will be executed.

When the control device 17 of each component mounting machine 12 starts this program, first, in step 101, the component sucked by the suction nozzle 31 is imaged by the component imaging camera 18, and the captured image is captured. After that, the process proceeds to step 102, and it is determined whether or not there is a trained model for the imaged component among the trained models stored in the storage device (not shown) for each type of component. If there is a trained model for the imaged component, the process proceeds to step 103, and the trained model for the imaged component is selected as the trained model used for the image recognition this time.

On the other hand, if there is no trained model for the imaged component among the trained models stored in the storage device for each type of component, the process proceeds to step 104, and the storage device stores each component type. From the trained models that have been learned, a trained model for a component that has a predetermined similarity relationship with the imaged component is selected as the trained model to be used for this image recognition.

After selecting the trained model to be used for the image recognition this time as described above, the process proceeds to step 105, the captured image this time is processed by the image processing function of the control device 17, and the selected trained model is used. , It is determined whether the suction posture of the imaged component is normal suction (see FIG. 2) or abnormal suction (see FIG. 3).

After that, the process proceeds to step 106 to determine whether or not the suction posture determination result is normal suction, and if it is normal suction, the process proceeds to step 107, and the image captured this time is used as a sample component image of normal suction by the learning computer 23. Transfer to and exit this program. On the other hand, if the determination result of the suction posture is not normal suction but abnormal suction, the process proceeds to step 108, and the image captured this time is transferred to the learning computer 23 as a sample component image of abnormal suction to end this program. As a result, the learning computer 23 collects sample component images of normal adsorption / abnormal adsorption from the control device 17 of each component mounting machine 12.

The control device 17 of each component mounting machine 12 may temporarily collect sample component images of normal adsorption / abnormal adsorption. In this case, every time the control device 17 of each component mounting machine 12 collects a predetermined number of sample component images (or every time a predetermined number of sample component images are collected), the sample component images collected so far are collectively transferred to the learning computer 23. Or, every time the sample component image transfer request is output from the learning computer 23, the sample component images collected by the control device 17 of each component mounting machine 12 are collectively collected by the learning computer 23. You may try to transfer to. Alternatively, the production management computer 21 may collect sample part images from the control device 17 of each component mounting machine 12 and transfer the sample component images from the production management computer 21 to the learning computer 23. .. With either method, the learning computer 23 can finally collect sample component images.

[Learned model creation program for each part]
The learning computer 23 repeatedly executes the learned model creation program for each part in FIG. 5 at a predetermined cycle. When the learning computer 23 starts this program, first, in step 201, sample component images of normal adsorption / abnormal adsorption are collected from the control device 17 of each component mounting machine 12 or the production management computer 21 for each component type. .. Then, in the next step 202, information on the inspection result is acquired from the inspection machine 14.

After that, the process proceeds to step 203, and from the collected sample component images of normal adsorption, the sample component image obtained by imaging the component determined to be improperly mounted by the inspection machine 14 is discarded. This is because even if it is determined to be normal adsorption, the component determined to be improperly mounted by the inspection machine 14 may actually be abnormal adsorption. The process of step 203 is performed by the control device 17 of each component mounting machine 12 or the production management computer 21, and only the image of the component that is not determined to be mounting failure by the inspection machine 14 is sampled for normal adsorption. The learning computer 23 may collect the component images.

After that, the process proceeds to step 204, and the mounting defect occurrence rate is calculated for each type of component based on the information of the inspection result acquired from the inspection machine 14. After that, the process proceeds to step 205, and it is determined whether or not there is a component whose calculated mounting defect occurrence rate exceeds a predetermined determination threshold value, and there is no component whose mounting defect occurrence rate exceeds the determination threshold value. Is determined that the accuracy of image recognition using the selected trained model is ensured (it is not necessary to create a trained model for each part), and this program is terminated.

On the other hand, if there is a component whose mounting defect occurrence rate exceeds the judgment threshold, the accuracy of image recognition is not ensured for that component (it is necessary to create a trained model for each component). After making a judgment, the process proceeds to step 206, and the sample part image of normal adsorption / abnormal adsorption collected for the part is added as teacher data of the reference trained model used for image recognition of the part and relearned. Create a part-specific trained model for the part. After that, the process proceeds to step 207, and the created component-specific trained model is transferred to the control device 17 of each component mounting machine 12 to end this program. As a result, the control device 17 of each component mounting machine 12 is in a state where image recognition can be performed using the component-specific learned model transferred from the learning computer 23.

According to the present embodiment described above, for a part having a predetermined similarity relationship with the reference part for which the reference-learned model is created, a sample part image is collected for each type of the part and the relevant part is concerned. By adding the sample part image for each part type as teacher data of the reference trained model and re-learning, a trained model for each part used for image recognition of the part is created for each type of the part. Therefore, it is possible to relatively easily create a trained model for each part used for image recognition of a part having a predetermined similarity relationship with the reference part from the standard trained model, and the work of creating the trained model. You can save time and effort.

Moreover, in this embodiment, the trained model for each part created for each part type is included in the image processing part shape data prepared for each part type, so that the image processing part shape data is included. Even with component mounting machines on other component mounting lines that can be used, similar image recognition using the trained model for each component is possible, which has the advantage of improving production quality and stabilizing.

However, the trained model for each part may be managed independently without being associated with the part shape data for image processing.

Further, in this embodiment, a component sucked by the suction nozzle 31 of each component mounting machine 12 during production is imaged by the component imaging camera 18, and the image is processed so that the suction posture of the component is normal suction or abnormal. It is in production because it is determined whether it is adsorption and the captured image determined to be normal adsorption is collected as a sample part image of normal adsorption, and the captured image determined to be abnormal adsorption is collected as a sample part image of abnormal adsorption. The image captured by the component imaging camera 18 can be collected as a sample component image of normal adsorption / abnormal adsorption, and the labor of collecting the sample component image can be saved.

However, the method of collecting the sample part image is not limited to the method of collecting the sample part image only during the production. The image may be taken by the imaging camera 18 and the captured image may be collected as a sample component image of normal adsorption / abnormal adsorption. Alternatively, a dedicated imaging device for capturing a sample component image may be used to collect a sample component image of normal adsorption / abnormal adsorption captured by the imaging device. When a dedicated imaging device is used, sample component images of normal adsorption / abnormal adsorption can be collected before, during, or after the end of production.

Further, in this embodiment, when a component whose mounting defect occurrence rate exceeds a predetermined determination threshold value occurs during production, the sample component image of normal adsorption / abnormal adsorption collected for the component is used as an image of the component. By adding it as teacher data of the reference trained model used for recognition and retraining it, a trained model for each part for the relevant part is created and transferred to the control device 17 of each component mounting machine 12. , Every time a part whose mounting defect occurrence rate exceeds a predetermined judgment threshold occurs during production, it is possible to create a learned model for each part used for image recognition of the part, and the part is manufactured during production. It is possible to improve the accuracy of image recognition and reduce the incidence of mounting defects.

However, the trained model for each part may be created before the start of production or after the end of production. Alternatively, a trained model for each component may be created when the number of collected sample component images of normal adsorption / abnormal adsorption exceeds a predetermined number.

Further, the trained model of this embodiment is a trained model that determines whether the suction posture of the component sucked on the suction nozzle 31 is normal suction or abnormal suction, but determines the presence or absence of the component sucked on the suction nozzle 31. It may be a trained model. In this case, the image captured by the component imaging camera 18 with the components adsorbed on the suction nozzle 31 is collected as a sample component image with the components, and the component imaging camera without the components adsorbed on the suction nozzle 31 is collected. The image captured in 18 is collected as a sample part image without parts, and the sample part image with / without parts classified according to the type of the part is used as the reference trained model teacher data for image recognition of the part. By adding and re-learning as, a part-specific trained model for the part may be created for each type of the part. In this case as well, the sample component image may be collected by using a dedicated imaging device.

In addition, the inspection machine 14 may inspect the presence or absence of parts on the circuit board 11 by using a learned model for determining the presence or absence of parts mounted on the circuit board 11. In this case, the control device 20 of the inspection machine 14 captures the mounted state of each component on the carried-in circuit board 11 with the inspection camera 22, processes the captured image, and uses the trained model. The presence or absence of each component on the circuit board 11 is inspected, and the captured image determined to have a component is collected as a sample component image with a component, and the captured image determined to have no component is collected as a sample component image without a component. By adding the sample part image with / without parts classified according to the type of the part as the teacher data of the reference trained model used for the image recognition of the part and re-learning, for each type of the part. A part-specific trained model for the part may be created. In this case as well, the sample component image may be collected by using a dedicated imaging device.

In addition, the present invention may be modified in various ways without departing from the gist, such as changing the configuration of the component mounting line 10 or appropriately changing the processing contents and processing order of the programs of FIGS. 4 and 5. Needless to say, it can be implemented.

10 ... Parts mounting line, 11 ... Circuit board, 12 ... Parts mounting machine, 14 ... Inspection machine, 17 ... Parts mounting machine control device, 18 ... Parts imaging camera, 19 ... Feeder, 20 ... Inspection machine control device, 21 ... Production control computer, 22 ... Inspection camera, 23 ... Learning computer, 31 ... Suction nozzle

Claims (9)

A component mounted on a suction nozzle of a component mounting machine or a component mounted on a circuit board is targeted for imaging, and a trained model is created by capturing the imaged object with a camera and recognizing the image. In the modeling system
Equipped with a computer to acquire the reference trained model used for image recognition of the reference part
The computer collects sample part images for each type of part having a predetermined similarity relationship with the reference part, and uses the sample part image for each type of the part as teacher data of the reference-learned model. A trained model creation system for component image recognition that creates a trained model for each component used for image recognition of the component for each type of component by adding and re-learning. The part image recognition trained according to claim 1, wherein the learned model for each part created by the computer for each type of the part is included in the image processing part shape data prepared for each type of the part. Modeling system. A part having a predetermined similarity relationship with the reference part has the same or similar shape as the reference part even if the size, color, material, manufacturing company, or manufacturing lot is different from the reference part. The trained model creation system for component image recognition according to claim 1 or 2, which is a component. The learning for component image recognition according to any one of claims 1 to 3, wherein the computer collects an image captured by the camera of the component mounting machine or the camera of the inspection machine as the sample component image during production. Completed modeling system. The reference trained model and the component-specific trained model are the trained models for discriminating whether the suction posture of the parts sucked by the suction nozzle is normal suction or abnormal suction, according to any one of claims 1 to 4. A trained model creation system for component image recognition. The trained model for component image recognition according to any one of claims 1 to 4, wherein the reference trained model and the component-specific trained model are trained models that determine the presence or absence of components adsorbed on the suction nozzle. Creation system. The trained model for component image recognition according to any one of claims 1 to 4, wherein the reference trained model and the component-specific trained model are trained models that determine the presence or absence of components mounted on the circuit board. Creation system. The trained model creation system for component image recognition according to any one of claims 1 to 7, wherein the computer transfers the created trained model for each component to a component mounting machine or an inspection machine that uses the model. A component sucked on the suction nozzle of the component mounting machine or a component mounted on the circuit board is targeted for imaging, and a trained model is created by capturing the imaged object with a camera and recognizing the image. In the model creation method
The process of acquiring the reference trained model used for image recognition of the reference part, and
A process of collecting sample part images for each type of part having a predetermined similarity relationship with the reference part, and
By adding the sample part image acquired for each type of the part as teacher data of the reference-learned model and re-learning, a trained model for each part used for image recognition of the part is created for each type of the part. A trained model creation method for component image recognition, including the steps to be performed.

Images