JP6899762B2 - Component mounting system - Google Patents

Description

The present invention relates to a component mounting system including a component mounting device for mounting components on a substrate.

A component mounting device for mounting an electronic component (hereinafter, simply referred to as a “component”) on a substrate such as a printed wiring board includes a head unit having a holder for holding the component. The head unit moves to a predetermined component mounting position while holding the component by the holder, and mounts the component on the board at the component mounting position.

When mounting a component on a substrate by a head unit, the posture of the component held by the holder may become a problem. That is, when the posture of the component held by the holder is in an abnormal posture, a defective mounting of the component on the substrate may occur. Patent Documents 1 and 2 disclose techniques for solving such problems. In the technique disclosed in Patent Documents 1 and 2, the component held by the holder is imaged, and the posture of the component is determined based on the captured image.

By the way, in the operation control of the head unit, when the posture of the component held by the holder is the mounting permitted posture that allows mounting on the board, the mounting operation of the component is executed and the mounting on the board is not permitted. In the case of the unauthorized posture, the component disposal operation is executed. That is, depending on the determination result of the posture of the component held by the holder, it is determined whether to shift to the component mounting operation or the component disposal operation. Therefore, the setting of the determination condition used when determining the posture of the component based on the captured image is extremely important for determining which operation of the component mounting operation or the disposal operation is to be performed.

However, the relationship between the posture of the component held by the holder and the improper mounting of the component on the substrate is not uniquely determined, and is influenced by the shape and size of the component, or the retainer. Therefore, it is difficult to set the determination conditions used when determining the component posture based on the captured image. For example, regarding the posture of the parts held by the holder, when strict judgment conditions are set that excessively narrow the allowable range of the mounting permission posture, it is possible to reduce the occurrence of mounting defects of the parts on the board. Even if the posture of the parts does not actually result in mounting failure, it is determined that the posture is not permitted to be mounted, and the parts are undesirably discarded. On the other hand, if a loose judgment condition is set that excessively widens the allowable range of the mounting permitted posture, it is determined that the component posture that actually causes mounting failure is also the mounting permitted posture, and the component mounting operation on the board is performed. When executed, the occurrence of defective mounting of components on the board increases.

Japanese Unexamined Patent Publication No. 11-258178 Japanese Unexamined Patent Publication No. 2012-160525

The present invention has been made in view of such circumstances, and an object of the present invention is to preferably determine whether or not a component held by a holder is in a posture that allows mounting on a substrate. It is an object of the present invention to provide a component mounting system capable of reducing the disposal of undesired components and reducing the occurrence of defective mounting of components on a substrate.

The component mounting system according to one aspect of the present invention includes a component mounting device for mounting components on a board, and a machine learning device connected to the component mounting device so as to be capable of data communication. The component mounting device has a holder for holding the component, and holds the component held by the retainer on the head unit and the holder before the head unit mounts the component on the substrate. Mounting permission information in the case of an imaging device that captures an image of the mounted component and acquires a captured image, and a component posture that determines the posture of the component held by the holder based on the captured image and permits mounting on the substrate. Includes a component posture determination unit that outputs mounting disapproval information in the case of a component posture that disallows mounting on the board. The machine learning device is a teacher data generation unit that generates teacher data by adding inspection result information representing an inspection result of a state of mounting a component on a substrate to a feature amount related to the captured image or a component posture based on the captured image. And, machine learning is executed using each of the teacher data storage unit that sequentially accumulates and stores the teacher data generated by the teacher data generation unit and the teacher data stored in the teacher data storage unit. Includes a learning unit that generates determination conditions used by the component orientation determination unit to determine the orientation of the component.

According to this component mounting system, the component mounting device and the machine learning device are connected so as to be capable of data communication. In the component mounting device, the image pickup device images the component held by the holder of the head unit and acquires the captured image. The captured image acquired by this imaging device is referred to when the component posture determination unit determines the component posture. The component posture determination unit determines the posture of the component held by the holder based on the captured image, outputs the mounting permission information in the case of the component posture that allows mounting on the board, and disallows mounting on the board. Outputs mounting disapproval information in the case of posture.

Here, the determination condition used by the component posture determination unit to determine the posture of the component is generated by the learning unit of the machine learning device. The learning unit generates the determination condition by executing machine learning using the teacher data generated by the teacher data generation unit stored in the teacher data storage unit. The teacher data used by the learning unit when executing machine learning is data in which the captured image or the feature amount related to the component posture based on the captured image is added with the inspection result information indicating the inspection result of the mounted state of the component on the substrate. is there. Since the posture of the parts held by the holder is determined based on the judgment conditions generated by executing machine learning using the teacher data, it is preferable to take a posture that allows mounting on the board. Can be determined. Therefore, it is possible to realize a component mounting system capable of reducing the disposal of undesired components and reducing the occurrence of component mounting defects on the substrate.

In the above component mounting system, the teacher data generation unit generates first teacher data in which the inspection result information is added to the captured image as the teacher data, and the learning unit generates the first teacher data. The deep learning using the above may be executed as the machine learning.

In this aspect, the learning unit generates a determination condition used when the component posture determination unit determines the posture of the component by executing deep learning using the first teacher data including the captured image itself. As a result, regarding the posture of the parts held by the holder, the mounting permitted posture that permits mounting on the board and the mounting disallowed posture that disallows mounting on the board can be set with higher accuracy and reliability. It can be distinguished by the state.

In the above component mounting system, the teacher data generation unit generates second teacher data in which the inspection result information is added to the feature amount as the teacher data, and the learning unit generates the second teacher data. The supervised learning using the above may be executed as the machine learning.

In this aspect, the learning unit is used when the component posture determination unit determines the posture of the component by executing supervised learning using the second supervised learning including the feature amount related to the component posture based on the captured image. Generate a judgment condition. As a result, the time required for the learning unit to generate the determination condition can be shortened, and the load of the machine learning process by the learning unit can be reduced.

In the above-mentioned component mounting system, the machine learning device has a history storage unit that stores history information representing the history of the number of outputs of the mounting disapproval information by the component attitude determination unit for each predetermined period, and the history information. Based on this, the configuration may further include a correction unit that monitors the transition of the number of outputs and corrects the determination condition so that the upward tendency is alleviated when the excessive upward tendency of the number of outputs is detected. ..

Machine learning by the learning unit is executed regularly, and as the machine learning progresses, the allowable range of the mounting permission posture that allows mounting on the board is excessively narrowed with respect to the posture of the parts held by the holder, which is a strict judgment. Conditions may be generated. In this case, although it is possible to reduce the occurrence of component mounting defects on the board to the utmost limit, it is determined that the component posture that does not actually result in mounting defects is also the mounting disapproval posture, and the components are undesirably discarded. It can be done. Therefore, the correction unit monitors the transition of the output number of times of mounting disapproval information by the component posture determination unit, and when it detects an excessive increase tendency of the output number, corrects the determination condition so that the increase tendency is alleviated. To do. As a result, when a strict determination condition that excessively narrows the allowable range of the mounting permission posture is generated, the determination condition can be suitably corrected. By determining the posture of the component held by the holder based on the determination condition corrected in this way, it is possible to preferably determine whether or not the posture is permitted to be mounted on the substrate.

In the above component mounting system, the teacher data storage unit stores the teacher data by dividing it according to the shape of the component, and the learning unit uses the teacher data divided according to the shape of the component to store the component. The machine learning may be executed for each shape of.

In this aspect, the learning unit generates a determination condition used when the component posture determination unit determines the posture of the component by executing machine learning using the teacher data classified for each shape of the component. As a result, the range of machine learning executed by the learning unit can be limited for each shape of the part, and the time required for the learning unit to generate the determination condition can be shortened. Further, the determination conditions generated by the learning unit are set for each shape of the parts, and the same determination conditions can be applied to the parts having the same shape.

In the above-mentioned component mounting system, the imaging device is held by a first imaging unit that captures a component held by the holder from a first direction and acquires a first captured image, and the holder. The component may be configured to include a second imaging unit that acquires a second captured image by imaging the component from a second direction intersecting the first direction.

In this aspect, the first captured image and the second captured image captured from different directions are acquired for the parts held by the holder. In this case, the learning unit executes machine learning using the teacher data including the first captured image and the second captured image, and generates a determination condition used when the component posture determination unit determines the posture of the component. It will be. As a result, regarding the posture of the parts held by the holder, the mounting permitted posture that permits mounting on the board and the mounting disallowed posture that disallows mounting on the board can be set with higher accuracy and reliability. It can be distinguished by the state.

In the above-mentioned component mounting system, an inspection device which is connected to the machine learning device so as to be capable of data communication and inspects the mounted state of the components mounted on the board by the head unit is further provided, and the teacher data generation unit is described. The inspection result information may be acquired from the inspection device and the teacher data may be generated.

Further, in the above-mentioned component mounting system, the inspection device is used after the components are mounted on the substrate by the head unit and before the reflow process for fixing the components to the substrate by using solder is performed. The configuration may be such that the mounting state of the component on the board is inspected.

When the reflow process is performed after mounting the component on the substrate, the component is pulled into a predetermined position on the substrate by the self-alignment effect due to the agglutination action of the solder. That is, when the inspection of the mounting state of the component on the board is performed after the reflow process, it is not possible to grasp the true mounting state immediately after the component is mounted. Therefore, when the learning unit executes machine learning, when the teacher data to which the inspection result information after the reflow processing is added is used as the inspection result information of the component mounting state on the board, the component mounting state on the board is used. It may not be possible to generate a determination condition for determining the directly associated component orientation. Therefore, the inspection device is configured so that the inspection of the mounted state of the components on the substrate is performed before the reflow processing. As a result, the learning unit can generate a determination condition for determining the component posture directly associated with the mounting state of the component on the board.

In the component mounting system, the machine learning device includes an operation unit that accepts an operation for inputting the inspection result information, and the teacher data generation unit acquires the inspection result information input by the operation on the operation unit. However, the teacher data may be generated.

In this aspect, when generating the teacher data to which the test result information is added, the teacher data generation unit can use the test result information input by the operation on the operation unit.

As described above, according to the present invention, it is suitably determined whether or not the parts held by the holder are in a posture that allows mounting on the substrate, the disposal of undesired parts is reduced, and the substrate is used. It is possible to provide a component mounting system capable of reducing the occurrence of component mounting defects.

It is a figure which shows roughly the structure of the component mounting system which concerns on one Embodiment of this invention. It is a top view which shows the structure of the component mounting apparatus provided in the component mounting system. It is a figure which shows schematic the component supply apparatus provided in the component mounting apparatus. It is a figure which shows the structure of the tape guide provided in the component supply apparatus. It is a perspective view of the component storage tape attached to the component supply device. It is a side view of the head unit provided in the component mounting apparatus. It is a top view of the head unit. It is a block diagram which shows the control system of a component mounting apparatus. It is a figure which shows typically the captured image acquired by the component recognition imaging apparatus provided in the component mounting apparatus. It is a figure for demonstrating the determination result information output from the component posture determination part provided in the component mounting apparatus. It is a figure for demonstrating the inspection result information output from an inspection apparatus. It is a block diagram which shows the control system of a machine learning apparatus. It is a figure for demonstrating the 1st teacher data generated by the teacher data generation part provided in the machine learning apparatus. It is a figure for demonstrating the 2nd teacher data generated by a teacher data generation part. It is a figure for demonstrating the judgment condition information generated by the learning part provided in the machine learning apparatus. It is a figure for demonstrating the distinction between the mounting permitted posture and the mounting disallowed posture in the judgment of the component posture using the judgment condition information generated by the learning unit. It is a figure for demonstrating the operation of the correction part provided in the machine learning apparatus. It is a flowchart which shows the control operation of a machine learning apparatus. It is a flowchart which shows the component mounting operation of the component mounting apparatus.

[Overall configuration of component mounting system]
Hereinafter, the component mounting system according to the embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram schematically showing a configuration of a component mounting system 1S according to an embodiment of the present invention. The component mounting system 1S includes a component mounting device 1, a machine learning device 20, and an inspection device 28, and each of these devices is connected so as to be capable of data communication via a network NW. The network NW is constructed by, for example, a local area network (LAN).

<Configuration of component mounting device>
The component mounting device 1 is a device for producing an electronic circuit board by mounting (mounting) components on a board. The component mounting device 1 will be described as follows with reference to FIG. FIG. 2 is a plan view showing the configuration of the component mounting device 1 provided in the component mounting system 1S. In the following, the directional relationship will be described using the XYZ orthogonal coordinate axes. The X direction is a direction parallel to the horizontal plane, the Y direction is a direction orthogonal to the X direction on the horizontal plane, and the Z direction is a vertical direction orthogonal to both the X and Y directions. Further, the one-way side in the X direction is referred to as "+ X side", and the other direction side opposite to the one-way side in the X direction is referred to as "-X side". Further, the one-way side in the Y direction is referred to as "+ Y side", and the other direction side opposite to the one-way side in the Y direction is referred to as "-Y side". Further, the one-way side in the Z direction is referred to as "+ Z side", and the other direction side opposite to the one-way side in the Z direction is referred to as "-Z side".

The component mounting device 1 includes a device main body 1a, a moving frame 2, a conveyor 3, a component supply unit 4 on which the component supply device 5 is mounted, a head unit 6, a first drive mechanism 7, and a second drive mechanism. 8 and a component recognition imaging device 9.

The device main body 1a is a structure in which each part constituting the component mounting device 1 is arranged, and is formed in a substantially rectangular shape in a plan view seen from the Z direction. The conveyor 3 extends in the X direction and is arranged on the apparatus main body 1a. The conveyor 3 conveys the substrate P in the X direction. The substrate P is conveyed on the conveyor 3 and is positioned at a predetermined component mounting position (position where components are mounted on the substrate P).

The component supply units 4 are arranged at four locations in total, two locations in the X direction, respectively, in the + Y side and −Y side region portions in the Y direction of the device main body 1a. The component supply unit 4 is an area in which a plurality of component supply devices 5 are mounted side by side in the device main body 1a, and is a target to be sucked by a suction nozzle 63 which is a holder provided in the head unit 6 described later. The set position of each component supply device 5 is divided for each component.

The parts supply device 5 is detachably attached to the parts supply unit 4 of the device main body 1a. The component supply device 5 is not particularly limited as long as it is configured to be capable of supplying electronic components (hereinafter, simply referred to as components). As the component supply device 5, for example, a tape feeder of a type that supplies small pieces such as ICs, transistors, and capacitors using a tape as a carrier, and a pallet including a tray on which the components are placed are moved. This makes it possible to adopt a tray feeder that supplies parts, a stick feeder that supplies parts stored in a tubular stick while pushing them out of the stick, and the like. In this embodiment, the component supply device 5 is a tape feeder. The parts supply device 5 will be described with reference to FIGS. 3 to 5. FIG. 3 is a diagram schematically showing a component supply device 5 provided in the component mounting device 1. FIG. 4 is a diagram showing a configuration of a tape guide 45 provided in the component supply device 5. FIG. 5 is a perspective view of the component storage tape 100 mounted on the component supply device 5.

The component supply device 5 is attached to an attachment portion 31 provided in the component supply unit 4. The mounting portion 31 is provided with a plurality of slots 32 arranged at regular intervals in the X direction and extending in parallel with each other in the Y direction, and a fixing base 33 extending in the X direction at a position in front of the slots 32. Then, the component supply device 5 is set in each slot 32, and each component supply device 5 is fixed to the fixing base 33. As a result, a plurality of component supply devices 5 are arranged side by side in the X direction in the component supply unit 4.

The component supply device 5 includes a main body 41 having an elongated shape in the front-rear direction (Y direction). The component supply device 5 is fixed to the fixing base 33 with the main body 41 inserted (set) in the slot 32.

The parts supply device 5 is further provided in the first tape sending section 42 provided in the front end portion of the main body 41, the second tape sending section 43 provided in the rear end portion of the main body 41, and the main body 41. The tape passage 44 and the tape guide 45 are provided.

The tape passage 44 is a passage for guiding the component storage tape 100. The tape passage 44 extends obliquely upward from the rear end portion of the main body portion 41 toward the front upper portion. The component storage tape 100 as a component storage member is introduced into the inside from the rear end portion of the main body portion 41, and is guided to the upper surface front portion of the main body portion 41 through the tape passage 44.

As shown in FIG. 5, the component storage tape 100 is a long tape composed of a tape main body 101 and a cover tape 102. A large number of component storage portions 103 (recesses) opened at the top of the tape main body 101 are formed at regular intervals in the longitudinal direction (tape feed direction), and the component E is stored in each component storage portion 103. The cover tape 102 is adhered to the upper surface of the tape main body 101, whereby each component storage portion 103 is closed by the cover tape 102. Further, on the side of the component storage portion 103 of the tape main body 101, a plurality of fitting holes 104 are provided which are arranged at regular intervals in the longitudinal direction and penetrate the tape main body 101 in the thickness direction thereof.

In the component supply device 5, the tape guide 45 is provided on the upper surface of the front portion of the main body portion 41. The tape guide 45 covers the component storage tape 100 that has passed through the tape passage 44, and guides the component storage tape 100 substantially horizontally along the upper surface of the main body 41 to the component supply position P1. The component supply position P1 is a position at which the head unit 6 is allowed to take out components, and is set at a position close to the front end of the upper surface of the main body 41.

As shown in FIG. 4, an opening 45A is provided at a position corresponding to the component supply position P1 in the tape guide 45, and a component exposed portion 451 is provided at a position rearward from the opening 45A. ing. The component exposed portion 451 exposes the component E in the component accommodating portion 103 of the component accommodating tape 100 guided by the tape guide 45. The structure of the component exposed portion 451 is not particularly limited as long as the component E can be exposed. The component exposure portion 451 employs a wide variety of component exposure methods, such as a method of exposing the component E by cutting the cover tape 102 and a method of exposing the component E by peeling the cover tape 102. Can be done. In the present embodiment, the component exposed portion 451 adopts a method of exposing the component E by cutting the cover tape 102, and includes an insertion portion 4511, a blade portion 4512, and a cover tape post-processing portion 4513.

The insertion portion 4511 is a thin plate-shaped portion formed in a tapered shape, and the tape main body 101 and the cover tape 102 are provided with respect to the component storage tape 100 in a state where the tip is guided by the tape guide 45 and the tip is a free end. It is inserted in between. The blade portion 4512 in the component exposed portion 451 is arranged on the downstream side in the tape feeding direction with respect to the insertion portion 4511, and cuts the cover tape 102 in a straight line along the tape feeding direction according to the traveling of the component storage tape 100. To do. In the component exposed portion 451 the cover tape post-processing portion 4513 is arranged on the downstream side in the tape feeding direction with respect to the blade portion 4512, and performs a process of spreading the cover tape 102 cut by the blade portion 4512. As a result, the component E can be exposed in the component storage section 103 of the component storage tape 100. The component E exposed in the component storage unit 103 in this way is attracted by the suction nozzle 63 of the head unit 6 in the component mounting device 1 through the opening 45A of the tape guide 45 at the component supply position P1. Taken out.

The first tape delivery unit 42 has a plurality of transmission gears that transmit the driving force of the first sprocket 51, the first motor 52, and the first motor 52 arranged below the tape guide 45 to the first sprocket 51. It is provided with a first gear group 53 including the above. The first sprocket 51 has teeth that fit into the fitting holes 104 of the component storage tape 100 that is guided along the tape guide 45. That is, the first tape delivery unit 42 sends out the component storage tape 100 toward the component supply position P1 by rotationally driving the first sprocket 51 by the first motor 52.

The second tape delivery unit 43 is a plurality of sheets that transmit the driving force of the second sprocket 54, the second motor 55, and the second motor 55 arranged at the rear end of the main body 41 to the second sprocket 54. A second gear group 56 including a transmission gear is provided. The second sprocket 54 faces into the tape passage 44 from above, and has teeth that fit into the fitting holes 104 of the component storage tape 100 guided along the tape passage 44. That is, the second tape delivery unit 43 sends the component storage tape 100 forward (part supply position P1) by rotationally driving the second sprocket 54 by the second motor 55.

The component storage tape 100 is intermittently delivered toward the component supply position P1 by the delivery portions 42 and 43, and the component E is taken out through the opening 45A of the tape guide 45 at the component supply position P1.

Next, referring to FIG. 2, the moving frame 2 provided in the component mounting device 1 extends in the X direction and is movably supported by the device main body 1a in a predetermined moving direction (Y direction). The head unit 6 is mounted on the moving frame 2. The head unit 6 is mounted on the moving frame 2 so as to be movable in the X direction. That is, the head unit 6 can move in the Y direction as the moving frame 2 moves, and can move in the X direction along the moving frame 2. The head unit 6 is movable between the component supply position P1 of the component supply device 5 mounted on the component supply unit 4 and a predetermined component mounting position of the substrate P conveyed by the conveyor 3, and is movable at the component supply position P1. The component E is taken out from the component supply device 5, and the taken out component E is mounted (mounted) on the substrate P at the component mounting position. Details of the head unit 6 will be described later.

The first drive mechanism 7 is arranged at the + X side and −X side ends of the apparatus main body 1a. The first drive mechanism 7 is a mechanism for moving the moving frame 2 in the Y direction. The first drive mechanism 7 includes, for example, a drive motor, a ball screw shaft extending in the Y direction and connected to the drive motor, and a ball nut disposed on the moving frame 2 and screwed with the ball screw shaft. Consists of. The first drive mechanism 7 having such a configuration moves the moving frame 2 in the Y direction by moving the ball nut forward and backward along the ball screw shaft as the ball screw shaft is rotationally driven by the drive motor.

The second drive mechanism 8 is arranged on the moving frame 2. The second drive mechanism 8 is a mechanism for moving the head unit 6 in the X direction along the moving frame 2. Similar to the first drive mechanism 7, the second drive mechanism 8 includes, for example, a drive motor, a ball screw shaft extending in the X direction and connected to the drive motor, and a ball screw shaft disposed on the head unit 6. It is configured to include a ball nut to be screwed. The second drive mechanism 8 having such a configuration moves the head unit 6 in the X direction by moving the ball nut forward and backward along the ball screw shaft as the ball screw shaft is rotationally driven by the drive motor.

In this example, the first drive mechanism 7 and the second drive mechanism 8 have a configuration in which the moving frame 2 and the head unit 6 are moved by the drive motor via the ball screw shaft. However, for example, a linear motor may be used as a drive source to directly drive the moving frame 2 and the head unit 6.

The head unit 6 will be described with reference to FIGS. 6 and 7 in addition to FIG. FIG. 6 is a side view of the head unit 6, and FIG. 7 is a plan view of the head unit 6 as viewed from below. The head unit 6 includes a head body 61, a rotating body 62, and a suction nozzle 63. The head main body 61 constitutes a main body portion of the head unit 6. The rotating body 62 is formed in a columnar shape, and is provided on the head main body 61 so as to be rotatable around a vertical axis (axis extending in the Z direction) by a rotating body driving mechanism 66 (see FIG. 8 described later).

A plurality of suction nozzles 63 are arranged at the outer peripheral edge of the rotating body 62 at predetermined intervals in the circumferential direction. The suction nozzle 63 is a holder capable of sucking and holding the component E supplied to the component supply position P1 by the component supply device 5. The suction nozzle 63 can communicate with any of the negative pressure generator, the positive pressure generator, and the atmosphere via the electric switching valve. That is, the negative pressure is supplied to the suction nozzle 63 to enable the suction nozzle 63 to hold the suction of the component E (take out the component), and then the positive pressure is supplied to hold the component E to suction. It will be released. In the present embodiment, the holder other than the suction nozzle 63 may be, for example, a chuck that grips and holds the component E.

The suction nozzle 63 is provided on the rotating body 62 so as to be able to move up and down in the vertical direction (Z direction) by the nozzle raising and lowering drive mechanism 67 (see FIG. 8 described later). The suction nozzle 63 is located in the Z direction (vertical direction) between the holding position where the component E supplied to the component supply position P1 by the component supply device 5 can be held and the retracted position on the upper side of the holding position. It is possible to move along. That is, when the component E supplied to the component supply position P1 is held, the suction nozzle 63 is lowered from the retracted position to the holding position by the nozzle elevating drive mechanism 67, and the component E is sucked and held at the holding position. On the other hand, the suction nozzle 63 after the suction and holding of the component E is raised from the holding position to the retracted position by the nozzle elevating drive mechanism 67.

Further, as shown in FIGS. 6 and 7, a substrate recognition camera 65 is fixed to the lower surface of the head body 61 of the head unit 6 via a mounting member 64 on the outer side (−X side) of the rotating body 62. There is. The substrate recognition camera 65 moves together with the head unit 6 to identify and position the type of the substrate P, and images various marks written on the upper surface of the substrate P from above.

Further, the component mounting device 1 according to the present embodiment includes a component recognition imaging device 9 as shown in FIGS. 2, 6 and 7. The component recognition imaging device 9 is an imaging device that acquires an image captured by imaging the component E held by the suction nozzle 63 before mounting the component E on the substrate P by the head unit 6. The component recognition imaging device 9 includes a first imaging unit 91 and a second imaging unit 92. The first imaging unit 91 images the component E held by the suction nozzle 63 from the first direction to acquire the first captured image G1 (see FIG. 9A described later). The second imaging unit 92 images the component E held by the suction nozzle 63 from a second direction intersecting the first direction to acquire a second captured image G2 (see FIG. 9B described later). To do.

In the present embodiment, as shown in FIG. 2, the first image pickup unit 91 is arranged at a position between each component supply unit 4 and the conveyor 3 on the apparatus main body 1a, and is, for example, a CMOS (Complementary metal-axis). -A component recognition camera equipped with an image sensor such as a CMOS device or a CCD (Charged-coupled device). The first imaging unit 91 is attracted and held by the suction nozzle 63 while the head unit 6 is moving from the component supply position P1 of the component supply device 5 to the component mounting position of the substrate P conveyed by the conveyor 3. The component E is imaged from the first direction (direction from the lower side to the upper side) to acquire the first captured image G1.

On the other hand, as shown in FIGS. 6 and 7, the second image pickup unit 92 is fixed to a mounting member 64 projecting from the lower surface of the head body 61 of the head unit 6, and includes, for example, an image pickup element such as CMOS or CCD. It is a parts recognition camera. The second imaging unit 92 images the component E attracted and held by the suction nozzle 63 from the second direction (side) to acquire the second captured image G2.

<Control system for component mounting equipment>
Next, the control system of the component mounting device 1 will be described with reference to the block diagram of FIG. The component mounting device 1 includes a control unit 10. The control unit 10 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory) for storing a control program, a RAM (Random Access Memory) used as a work area of the CPU, and the like. The control unit 10 comprehensively controls the operation of the component mounting device 1 by executing the control program stored in the ROM by the CPU. As shown in FIG. 8, the control unit 10 includes a communication unit 11, a component supply control unit 12, a head drive control unit 13, and a component posture determination unit 14.

The communication unit 11 is an interface circuit for performing data communication with the machine learning device 20 via the network NW. The component supply control unit 12 controls the supply operation of the component E to the component supply position P1 by the component supply device 5.

The head drive control unit 13 controls the movement of the head unit 6 by the first drive mechanism 7 and the second drive mechanism 8 on the horizontal plane with respect to the X direction and the Y direction. Further, the head drive control unit 13 controls the elevating operation of the suction nozzle 63 by the nozzle elevating drive mechanism 67. Further, the head drive control unit 13 controls the rotational operation of the rotating body 62 by the rotating body driving mechanism 66.

The component posture determination unit 14 determines the posture of the component E held by the suction nozzle 63 based on the first image G1 acquired by the first image pickup unit 91 and the second image G2 acquired by the second image pickup unit 92. To judge.

Here, the first captured image G1 and the second captured image G2 that the component posture determining unit 14 refers to when determining the posture of the component E will be described with reference to FIG. FIG. 9A is a diagram schematically showing the first captured image G1 acquired by the first imaging unit 91, and FIG. 9B is a second captured image acquired by the second imaging unit 92. It is a figure which shows G2 schematically. The first captured image G1 includes a background region G11 composed of pixels corresponding to the background in the first captured image G1 and a component region G12 composed of pixels corresponding to the component E held by the suction nozzle 63. In the first captured image G1, the component region G12 is a bright region brighter than the background region G11. On the other hand, the second captured image G2 includes a nozzle tip region G21 composed of pixels corresponding to the tip portion 631 of the suction nozzle 63 and a component region G22 composed of pixels corresponding to the component E held by the suction nozzle 63. In the second captured image G2, the nozzle tip region G21 and the component region G22 are dark regions darker than the background region.

The component posture determination unit 14 determines the posture of the component E held by the suction nozzle 63 based on the first captured image G1 and the second captured image G2, and determines the determination result information J1 (FIG. 10) representing the determination result. See) is output. As shown in FIG. 10, the determination result information J1 includes, for example, the part identification information J11 (part ID) for identifying the part E, the part type information J12 (part type) indicating the type of the part E, and the part E. This is information in which the mounting number information J13 (mounting number) representing the mounting number for the board P and the component posture determination result information J14 (component posture determination result) representing the component posture determination result are associated with each other. The component posture determination result information J14 includes mounting permission information J141 and mounting disapproval information J142. The mounting permission information J141 is output from the component posture determination unit 14 when the posture of the component E held by the suction nozzle 63 is a posture permitting mounting on the substrate P (hereinafter, referred to as “mounting permission posture”). It is the information to be done. On the other hand, the mounting disapproval information J142 is a component when the posture of the component E held by the suction nozzle 63 is a posture disallowing mounting on the substrate P (hereinafter, referred to as “mounting disallowed posture”). This is information output from the posture determination unit 14.

In the example shown in FIG. 10, the mounting permission is given to the component E in which the mounting number information J13 corresponds to "M001", the component identification information J11 is indicated by "PID01", and the component type information J12 is indicated by "P001". Information J141 is associated and it is shown that the component E is in a mounting permission position. Similarly, the mounting permission information J141 is associated with the component E in which the mounting number information J13 corresponds to "M002", the component identification information J11 is indicated by "PID01", and the component type information J12 is indicated by "P001". It is shown that the component E is in the mounting permission posture. Similarly, for the component E in which the mounting number information J13 corresponds to "M003", the component identification information J11 is indicated by "PID01", and the component type information J12 is indicated by "P001", the mounting disapproval information J142 is provided. It is associated and the component E is shown to be in a mounting disallowed position.

When the mounting permission information J141 is output from the component posture determination unit 14, the head drive control unit 13 is indicated by the component identification information J11, the component type information J12, and the mounting number information J13 associated with the mounting permission information J141. The operation of the head unit 6 is controlled so as to execute the component mounting operation of the component E on the substrate P. On the other hand, when the mounting disapproval information J142 is output from the component posture determination unit 14, the head drive control unit 13 determines the component identification information J11, the component type information J12, and the mounting number information J13 associated with the mounting disapproval information J142. The operation of the head unit 6 is controlled so as to execute the component disposal operation of discarding the component E indicated by.

Next, a more detailed configuration of the component posture determination unit 14 will be described. As shown in FIG. 8, the component posture determination unit 14 includes a feature amount calculation unit 141, a determination unit 142, a determination condition update unit 143, and a determination condition storage unit 144.

Based on the first captured image G1 and the second captured image G2, the feature amount calculation unit 141 calculates the feature amount related to the posture of the component E for each determination index that is an index for determining the posture of the component E. As the determination index, for example, "edge strength", "side orthogonality", "part angle", "part center deviation amount" based on the first captured image G1, and "part thickness" based on the second captured image G2. And "part lower surface inclination" and the like.

The “edge strength” of the determination index is an index indicating the degree of change in the shade of the edge which is the boundary portion between the background region G11 and the component region G12 in the first captured image G1. For example, when the component E is held in a parallel posture parallel to the component holding surface of the tip portion 631 of the suction nozzle 63, the boundary portion between the background region G11 and the component region G12 in the first captured image G1 is formed. A shadow region representing the shadow of the component E is not formed. On the other hand, when the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, the component E is formed at the boundary portion between the background region G11 and the component region G12 in the first captured image G1. Since the shadow region representing the shadow of is formed, the "edge strength" is lower than that in the case of the parallel posture. The feature amount calculation unit 141 calculates the degree of change in the shade of the edge which is the boundary portion between the background region G11 and the component region G12 as the feature amount related to the "edge strength" of the determination index based on the first captured image G1.

The “side orthogonality” of the determination index is an index indicating the degree of orthogonality between adjacent sides on each side constituting the component region G12 in the first captured image G1. For example, when the rectangular component E in a plan view is held in a parallel posture parallel to the component holding surface of the tip 631 of the suction nozzle 63, the adjacent sides of the component region G12 in the first captured image G1 are It is orthogonal. On the other hand, when the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, the adjacent sides of the component region G12 in the first captured image G1 are not orthogonal to each other, so that the parallel posture The "side orthogonality" is lower than in the case of. The feature amount calculation unit 141 calculates the degree of orthogonality between adjacent sides of the component region G12 as a feature amount related to the "side orthogonality" of the determination index based on the first captured image G1.

The "part angle" of the determination index is an index indicating the degree of rotation of the part region G12 in the frame of the image in the first captured image G1. For example, when the suction nozzle 63 holds the component E in a predetermined normal posture, whereas the component E is held in a rotational posture rotated at a predetermined rotation angle, the component region G12 in the first captured image G1. The "part angle" is larger than that in the normal posture. The feature amount calculation unit 141 calculates the degree of rotation of the component area G12 as a feature amount related to the "part angle" of the determination index based on the first captured image G1.

The "part center deviation amount" of the determination index is an index indicating the deviation amount of the center of the component region G12 from the center of the frame of the image in the first captured image G1. For example, when the component E is held by the suction nozzle 63 in a predetermined normal posture, the center of the component holding surface of the suction nozzle 63 and the center of the component E are located coaxially with each other, so that the first captured image G1 In, the center of the component region G12 coincides with the center of the frame of the image. On the other hand, when the component E is held in a state of being displaced with respect to the suction nozzle 63, the center of the component holding surface of the suction nozzle 63 and the center of the component E are not located coaxially. In the captured image G1, the center of the component region G12 is displaced with respect to the center of the frame of the image. The feature amount calculation unit 141 calculates the amount of deviation from the center of the frame of the image at the center of the part region G12 as the feature amount related to the "part center deviation amount" of the determination index based on the first captured image G1.

The "part thickness" of the determination index is an index indicating the length from the upper end to the lower end of the part region G22 in the second captured image G2. For example, when the component E is held in a normal posture in which the upper surface is in contact with the component holding surface of the suction nozzle 63, while the component E is held in an upright posture in which the side surfaces are in contact with each other, or when the suction nozzle 63 is held in an upright posture. When the component E is held in an inclined posture inclined with respect to the component holding surface, the "component thickness" represented by the length from the upper end to the lower end of the component region G22 in the second captured image G2 becomes large. The feature amount calculation unit 141 calculates the length from the upper end to the lower end of the part area G22 as a feature amount related to the "part thickness" of the determination index based on the second captured image G2.

The "part lower surface inclination degree" of the determination index is an index indicating the inclination degree of the lower side in each side constituting the component region G22 in the second captured image G2. For example, when the side view rectangular component E is held in a parallel posture parallel to the component holding surface of the suction nozzle 63, the lower side of the component region G22 in the second captured image G2 is the rectangular frame of the image. It is parallel to the bottom side. On the other hand, when the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, the lower side of the component region G22 in the second captured image G2 is relative to the lower side of the rectangular frame of the image. Therefore, the "part lower surface inclination" becomes larger than in the case of the parallel posture. The feature amount calculation unit 141 calculates the inclination degree of the lower side of the component area G22 as a feature amount related to the "part lower surface inclination degree" of the determination index based on the second captured image G2.

The determination unit 142 reads out the determination condition included in the determination condition information D2 (see FIG. 15 described later) stored in the determination condition storage unit 144 described later, and the determination condition and the feature amount calculated by the feature amount calculation unit 141. By referring to and, it is determined whether the posture of the component E held by the suction nozzle 63 is the mounting permitted posture or the mounting disallowed posture. The determination unit 142 outputs the mounting permission information J141 when it is determined to be in the mounting permission posture, and outputs the mounting disapproval information J142 when it is determined to be in the mounting disapproval posture.

The determination condition used by the determination unit 142 when determining the posture of the component E is a determination that serves as a threshold value for distinguishing between at least one determination index suitable for determining the component attitude and the mounting permitted posture and the mounting disallowed posture. Includes reference values and. In the judgment conditions, a judgment reference value is set for each judgment index. The determination unit 142 compares the feature amount calculated by the feature amount calculation unit 141 with the determination reference value for each determination index, so that the posture of the component E held by the suction nozzle 63 is the mounting permission posture or the above-mentioned mounting permission posture. Determine which posture is the mounting disallowed posture. When the feature amount satisfies the determination reference value in all the determination indexes, the determination unit 142 determines that the posture of the component E held by the suction nozzle 63 is the mounting permission posture. On the other hand, when the feature amount does not satisfy the determination reference value in at least one determination index, the determination unit 142 determines that the posture of the component E held by the suction nozzle 63 is the mounting disapproval posture.

The determination condition storage unit 144 is a storage unit that stores the determination condition information D2 including the determination condition generated by the learning unit 262 (see FIG. 12) of the machine learning device 20 described later. Further, in the determination condition update unit 143, each time the learning unit 262 generates a determination condition and the determination condition information D2 including the determination condition is output from the learning unit 262, the determination condition stored in the determination condition storage unit 144 Information D2 is updated.

When the head unit 6 controlled by the head drive control unit 13 executes the component mounting operation of the component E with respect to the board P in response to the output of the mounting permission information J141 from the determination unit 142, the board P on which the component E is mounted is subjected to the component mounting operation. It is transported to the inspection device 28. The inspection device 28 is a device that inspects the mounted state of the component E mounted on the substrate P by the head unit 6. In the present embodiment, the inspection device 28 is after the component E is mounted on the substrate P by the head unit 6 and before the reflow process for fixing the component E to the substrate P by using solder is performed. The mounting state of the component E on the substrate P is inspected.

The inspection device 28 outputs the inspection result information J2 (see FIG. 11) representing the inspection result of the mounted state of the component E on the substrate P. The inspection result information J2 output from the inspection device 28 is transmitted to the machine learning device 20 via the network NW. As shown in FIG. 11, the inspection result information J2 includes, for example, the part identification information J21 (part ID) for identifying the part E, the part type information J22 (part type) indicating the type of the part E, and the part E. This is information in which the mounting number information J23 (mounting number) indicating the mounting number for the board P and the inspection result J24 of the mounting state of the component E are associated with each other. The inspection result J24 includes mounting good information J241 and mounting defect information J242. The mounting good information J241 is information output from the inspection device 28 when the mounting state of the component E on the substrate P is good. On the other hand, the mounting defect information J242 is information output from the inspection device 28 when the mounting state of the component E on the substrate P is defective.

In the example shown in FIG. 11, the mounting number information J23 corresponds to "M001", the component identification information J21 is indicated by "PID01", and the component type information J22 is indicated by "P001". Information J241 is associated with it, indicating that the component E is in good condition. Similarly, the mounting defect information J242 is associated with the component E in which the mounting number information J23 corresponds to "M002", the component identification information J21 is indicated by "PID01", and the component type information J22 is indicated by "P001". It is shown that the mounting state of the component E is defective. The inspection result J24 is associated with the part E in which the mounting number information J23 corresponds to "M003", the part identification information J21 is indicated by "PID01", and the part type information J22 is indicated by "P001". Not. This is because the head unit 6 executed the component disposal operation for the component E in response to the output of the mounting disapproval information J142 from the determination unit 142, so that the component E was excluded from the inspection by the inspection device 28. ..

<Configuration of machine learning device>
Next, the machine learning device 20 provided in the component mounting system 1S will be described with reference to the block diagram of FIG. 12 in addition to FIG. The machine learning device 20 includes a communication unit 22, an operation unit 23, a storage data storage unit 24, a history storage unit 25, a central processing unit 26, and a teacher data storage unit 27, and each of these units is included. Is connected via bus 21.

The communication unit 22 is an interface circuit for performing data communication with each of the component mounting device 1 and the inspection device 28 via the network NW. The operation unit 23 receives input operations of various information by the operator.

The storage data storage unit 24 is a feature amount related to the component posture calculated by the first image pickup image G1 and the second image pickup image G2 acquired by the first image pickup unit 91 and the second image pickup unit 92, or the feature amount calculation unit 141. Is a storage unit that sequentially stores and stores. Further, the storage data storage unit 24 also sequentially stores and stores the inspection result information J2 output from the inspection device 28. However, the inspection result information J2 stored in the accumulated data storage unit 24 is not limited to the information output from the inspection device 28, and may be the information input by the operation on the operation unit 23.

The central processing unit 26 is composed of, for example, a microprocessor and peripheral circuits thereof, and includes a teacher data generation unit 261, a learning unit 262, and a correction unit 263.

The teacher data generation unit 261 generates teacher data used by the learning unit 262 when executing machine learning. The teacher data generation unit 261 generates the first teacher data D1 shown in FIG. 13 or the second teacher data D1A shown in FIG. 14 according to the machine learning method executed by the learning unit 262.

When generating the first teacher data D1 shown in FIG. 13, the teacher data generation unit 261 displays the inspection result information J2 in the first captured image G1 and the second captured image G2 stored in the accumulated data storage unit 24. By giving, the first teacher data D1 is generated. As shown in FIG. 13, the first teacher data D1 includes, for example, a part shape information D11 (part shape) representing the shape of the part E, a part identification information D12 (part ID) for identifying the part E, and the like. The component type information D13 (component type) representing the type of the component E, the mounting number information D14 (mounting number) representing the mounting number of the component E on the substrate P, and the first image information D15 (first image information D15) representing the first captured image G1. 1 captured image), the second image information D16 (second captured image) representing the second captured image G2, and the inspection result D17 included in the inspection result information J2 are associated with the data.

In the example shown in FIG. 13, the mounting number information D14 corresponds to "M001", the part shape information D11 is indicated by "PS01", the part identification information D12 is indicated by "PID01", and the part type information D13 is indicated by "P001". The first captured image G1 and the second captured image G2 itself are associated with the component E indicated by "", and "good mounting" is associated with the inspection result D17. Similarly, the mounting number information D14 corresponds to "M002", the component shape information D11 is indicated by "PS01", the component identification information D12 is indicated by "PID01", and the component type information D13 is indicated by "P001". The first captured image G1 and the second captured image G2 itself are associated with the component E, and "improper mounting" is associated with the inspection result D17.

When the second teacher data D1A shown in FIG. 14 is generated, the teacher data generation unit 261 inspects the feature amount related to the component posture calculated by the feature amount calculation unit 141 stored in the storage data storage unit 24. By adding the result information J2, the second teacher data D1A is generated. As shown in FIG. 14, the second teacher data D1A includes, for example, part shape information D11A (part shape) representing the shape of the part E, part identification information D12A (part ID) for identifying the part E, and the like. Part type information D13A (part type) indicating the type of part E, mounting number information D14A (mounting number) indicating the mounting number of component E with respect to the substrate P, feature amount D16A for each judgment index D15A, and inspection result information J2. It is the data associated with the inspection result D17A included in.

In the example shown in FIG. 14, the mounting number information D14A corresponds to "M001", the component shape information D11A is indicated by "PS01", the component identification information D12A is indicated by "PID01", and the component type information D13A is indicated by "P001". The judgment index D15A is associated with "T001" and "T002" as the feature amount D16A for each "DA01" and "DA02", and "good mounting" is associated with the inspection result D17A. ing. Similarly, the mounting number information D14A corresponds to "M002", the component shape information D11A is indicated by "PS01", the component identification information D12A is indicated by "PID01", and the component type information D13A is indicated by "P001". With respect to the component E, the determination index D15A is associated with "T003" and "T004" as the feature amount D16A for each "DA01" and "DA02", and the inspection result D17A is associated with "mounting defect".

When generating the teacher data D1 and D1A to which the inspection result information J2 is added, the teacher data generation unit 261 is not limited to using the inspection result information J2 output from the inspection device 28, and the operation unit 23 is not limited to using the inspection result information J2. The inspection result information input by the operation may be used. The teacher data D1 and D1A generated by the teacher data generation unit 261 are sequentially accumulated and stored in the teacher data storage unit 27. In other words, the teacher data storage unit 27 sequentially accumulates and stores the teacher data D1 and D1A generated by the teacher data generation unit 261. Further, the teacher data storage unit 27 stores the teacher data D1 and D1A separately for each shape of the component E.

The learning unit 262 is used when the determination unit 142 of the component posture determination unit 14 determines the posture of the component E by executing machine learning using the teacher data D1 and D1A stored in the teacher data storage unit 27. Generate a judgment condition. The learning unit 262 periodically executes machine learning. The machine learning method executed by the learning unit 262 is not particularly limited, and examples thereof include a method using a neural network (Neural Network). A neural network has a structure that imitates the structure of the human brain, and is constructed by stacking logic circuits that imitate the functions of neurons (nerve cells) in the human brain in multiple layers. The neural network includes an input layer, a hidden layer, and an output layer as logic circuit layers.

The learning unit 262 outputs the determination condition information D2 (see FIG. 15) including the generated determination condition. The determination condition information D2 output from the learning unit 262 is transmitted to the component mounting device 1 via the network NW. In the component mounting device 1, when the determination condition information D2 is received via the communication unit 11, the determination condition storage unit 144 stores the received determination condition information D2.

As shown in FIG. 15, the determination condition information D2 includes, for example, the part identification information D21 (part ID) for identifying the part E, the part type information D22 (part type) indicating the type of the part E, and the learning unit. This is information associated with the determination condition D23 generated by 262. The determination condition D23 includes the determination index D231 and the determination reference value D232. In the determination condition D23, the determination reference value D232 is set for each determination index D231.

In the example shown in FIG. 15, the determination index D231 is "DA01", "DA02", "DA03" with respect to the part E in which the part identification information D21 is indicated by "PID01" and the part type information D22 is indicated by "P001". , "DA04", and "DB11", "DB21", "DB31", and "DB41" are associated with each other as the determination reference value D232. Similarly, for the part E in which the part identification information D21 is indicated by "PID02" and the part type information D22 is indicated by "P002", the determination index D231 is "DA01", "DA02", "DA04", "DA05". As the determination reference value D232 for each of "", "DB12", "DB22", "DB42", and "DB51" are associated with each other.

The determination unit 142 of the component posture determination unit 14 described above refers to the determination condition information D2 output from the learning unit 262 and stored in the determination condition storage unit 144, so that the attitude of the component E held by the suction nozzle 63 To judge. Specifically, the determination unit 142 compares the feature amount calculated by the feature amount calculation unit 141 with the determination reference value D232 included in the determination condition D23 of the determination condition information D2 for each determination index D231. It is determined whether the posture of the component E held by the suction nozzle 63 is the mounting permitted posture or the mounting disallowed posture.

The teacher data D1 and D1A used by the learning unit 262 when executing machine learning are based on the first captured image G1 and the second captured image G2, or the feature quantity related to the component posture based on the captured images G1 and G2. This is the data to which the information J2 is added. Since the posture of the component E held by the suction nozzle 63 is determined based on the determination condition D23 generated by executing machine learning using the teacher data D1 and D1A, mounting on the substrate P is permitted. It is possible to preferably determine whether or not the vehicle is in the mounting permitted posture. Therefore, it is possible to reduce the disposal of the undesired component E and to reduce the occurrence of mounting failure of the component E on the substrate P.

Further, the learning unit 262 generates the determination condition D23 by executing machine learning using the teacher data D1 and D1A stored in the teacher data storage unit 27 and divided according to the shape of the component E. As a result, the range of machine learning executed by the learning unit 262 can be limited for each shape of the component E, and the time required for the learning unit 262 to generate the determination condition D23 can be shortened. Further, the determination condition D23 generated by the learning unit 262 is set for each shape of the component E, and the same determination condition D23 can be applied to the component E having the same shape.

Further, when the first teacher data D1 shown in FIG. 13 is used, the learning unit 262 executes deep learning (deep learning) as machine learning. The first teacher data D1 includes image information D15 and D16 representing the first captured image G1 and the second captured image G2 captured from different directions, and the inspection result D17. When the learning unit 262 executes deep learning, the neural network has a multi-layer structure in which a large number of hidden layers exist. When the learning unit 262 executes deep learning, learning is repeated in each layer while the first teacher data D1 is transmitted from the input layer to the deeper layer (hidden layer), and a judgment index suitable for the test result D17. The determination condition D23 is generated by extracting D231 and the determination reference value D232 from the first captured image G1 and the second captured image G2 classified for each component. Based on the judgment condition D23 generated by the learning unit 262 that executed the deep learning, the judgment unit 142 determines the component posture to allow the mounting to the board P, and the mounting to the board P is not permitted. It is possible to distinguish the mounting disallowed posture with higher accuracy and reliability.

Further, when the second teacher data D1A shown in FIG. 14 is used, the learning unit 262 executes supervised learning as machine learning. The second teacher data D1A includes a feature amount D16A for each determination index D15A and a test result D17A. When the learning unit 262 executes supervised learning that is distinguished from deep learning, the neural network has, for example, a layered structure in which a hidden layer is one layer. When the learning unit 262 executes supervised learning that is distinguished from deep learning, the second teacher data D1A is learned by being transmitted from the input layer to the hidden layer, and the judgment reference value D232 that matches the test result D17A. Is extracted for each determination index D15A, and the determination condition D23 is generated. When the learning unit 262 executes supervised learning that is distinguished from deep learning, the time required for the learning unit 262 to generate the determination condition D23 can be shortened as compared with the deep learning, and the learning unit The load of the machine learning process according to 262 can be reduced.

Further, as described above, the inspection result information J2 included in the teacher data D1 and D1A used when the learning unit 262 executes machine learning is in a state in which the component E is mounted on the substrate P before the reflow processing is performed. This is the information output from the inspection device 28 for inspecting.

When the reflow process is performed after the component E is mounted on the substrate P, the component E is pulled to a predetermined position on the substrate P by the self-alignment effect due to the agglutination action of the solder. That is, when the inspection of the mounted state of the component E on the substrate P is performed after the reflow process, it is not possible to grasp the true mounted state immediately after the component is mounted. Therefore, when the learning unit 262 uses the teacher data to which the inspection result information after the reflow processing is added when executing the machine learning, the component posture directly related to the mounting state of the component E on the substrate P is obtained. There is a possibility that the judgment condition for judgment cannot be generated. Therefore, the inspection device 28 is configured to inspect the mounted state of the component E on the substrate P before the reflow process. As a result, the learning unit 262 can generate the determination condition D23 for determining the component posture directly associated with the mounting state of the component E on the substrate P.

Regarding the determination of the component posture using the determination condition D23 generated by the learning unit 262, the distinction between the mounting permitted posture and the mounting disallowed posture will be described with reference to FIG. In the example shown in FIG. 16, in the first captured image G1 and the second captured image G2, the component postures indicated by the first component posture pattern PA1, the second component posture pattern PA2, and the third component posture pattern PA3 are the mounting permitted postures. Belongs to. On the other hand, the component postures indicated by the 4th component posture pattern PA4, the 5th component posture pattern PA5, the 6th component posture pattern PA6, the 7th component posture pattern PA7, and the 8th component posture pattern PA8 belong to the mounting disallowed posture. There is.

In the first to third component posture patterns PA1, PA2, and PA3, the feature quantities related to the "edge strength", "side orthogonality", and "component angle" of the determination index D231 in the first captured image G1 and the second captured image G2 All the feature amounts of the judgment index D231, including the feature amounts related to the "part thickness" and the "part lower surface inclination degree" of the judgment index D231 in the above, satisfy the judgment reference value D232. Therefore, the determination unit 142 determines that the component postures indicated by the first to third component posture patterns PA1, PA2, and PA3 are the mounting permitted postures.

In the fourth component posture pattern PA4, the feature quantities relating to the "edge strength", "side orthogonality" and "component angle" of the determination index D231 in the first captured image G1 and the "component" of the determination index D231 in the second captured image G2. The feature amount relating to the "bottom inclination" satisfies the determination reference value D232. However, since the component E is held in an upright posture in which the side surface of the suction nozzle 63 is in contact with the component holding surface, the feature amount related to the "component thickness" of the determination index D231 in the second captured image G2 is larger than the determination reference value D232. It is large and does not satisfy the judgment reference value D232. Therefore, the determination unit 142 determines that the component posture indicated by the fourth component posture pattern PA4 is the mounting disallowed posture.

In the fifth component posture pattern PA5, the feature quantities related to the "edge strength" and "side orthogonality" of the determination index D231 in the first captured image G1 and the "component thickness" and "component" of the determination index D231 in the second captured image G2. The feature amount related to "bottom inclination" satisfies the determination reference value D232. However, since the component E is held in a rotational posture rotated at a predetermined rotational angle with respect to the suction nozzle 63, the feature amount related to the "component angle" of the determination index D231 in the first captured image G1 is based on the determination reference value D232. Is also large and does not satisfy the determination reference value D232. Therefore, the determination unit 142 determines that the component posture indicated by the fifth component posture pattern PA5 is the mounting disallowed posture.

In the sixth component posture pattern PA6, the feature amount relating to the "side orthogonality" and the "part angle" of the determination index D231 in the first captured image G1 and the feature amount relating to the "part thickness" of the determination index D231 in the second captured image G2. Satisfies the determination reference value D232. However, the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, and a shadow representing the shadow of the component E is formed at the boundary portion between the background region G11 and the component region G12 in the first captured image G1. Since the region is formed, the feature amount related to the "edge strength" is lower than the judgment reference value D232, and the feature amount related to the "part bottom surface inclination" of the judgment index D231 in the second captured image G2 is lower than the judgment reference value D232. It is large and does not satisfy the judgment reference value D232. Therefore, the determination unit 142 determines that the component posture indicated by the sixth component posture pattern PA6 is the mounting disallowed posture.

In the seventh component posture pattern PA7, the feature amount related to the "part angle" of the determination index D231 in the first captured image G1 and the feature amount related to the "part thickness" of the determination index D231 in the second captured image G2 are the judgment reference values. It satisfies D232. However, the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, and a shadow representing the shadow of the component E is formed at the boundary portion between the background region G11 and the component region G12 in the first captured image G1. Since the region is formed and the adjacent sides of the component region G12 in the first captured image G1 are not orthogonal to each other, the "edge strength" and "side orthogonality" of the determination index D231 in the first captured image G1 are related. The feature amount is lower than the judgment reference value D232, and the feature amount related to the "part lower surface inclination" of the judgment index D231 in the second captured image G2 is larger than the judgment reference value D232 and does not satisfy the judgment reference value D232. Therefore, the determination unit 142 determines that the component posture indicated by the seventh component posture pattern PA7 is the mounting disallowed posture.

In the eighth component posture pattern PA8, the feature quantities relating to the "side orthogonality" and the "part angle" of the determination index D231 in the first captured image G1 and the "part lower surface inclination" of the determination index D231 in the second captured image G2 are related. The feature amount satisfies the determination reference value D232. However, the component E is held in an inclined posture inclined with respect to the component holding surface of the suction nozzle 63, and a shadow representing the shadow of the component E is formed at the boundary portion between the background region G11 and the component region G12 in the first captured image G1. Since the region is formed, the feature amount related to the "edge strength" of the judgment index D231 in the first captured image G1 is lower than the judgment reference value D232, and the feature related to the "part thickness" of the judgment index D231 in the second captured image G2. The amount is larger than the judgment reference value D232 and does not satisfy the judgment reference value D232. Therefore, the determination unit 142 determines that the component posture indicated by the eighth component posture pattern PA8 is the mounting disallowed posture.

With reference to FIG. 12, the history storage unit 25 of the machine learning device 20 stores the history information D3 (see FIG. 17). The history information D3 includes mounting disapproval information J142 included in the judgment result information J1 output from the determination unit 142 and mounting defect information J242 included in the inspection result information J2 output from the inspection device 28 every predetermined period. This is information indicating the history of the number of outputs of.

Machine learning by the learning unit 262 is periodically executed, and as the machine learning progresses, the allowable range of the mounting permission posture that allows mounting on the substrate P is excessively narrowed with respect to the posture of the component E held by the suction nozzle 63. There is a risk that the strict determination condition D23 will be generated. In the example shown in FIG. 17, in order from the left, the first judgment condition update (first judgment condition update), the second judgment condition update (second judgment condition update), and the third judgment condition update (third judgment condition update). (Update), machine learning is executed sequentially at each timing of the 4th judgment condition update (4th judgment condition update), and generated by executing machine learning at the timing of the 4th judgment condition update (4th judgment condition update). The determined determination condition D23 is a determination condition that excessively narrows the allowable range of the mounting permission posture. In this case, although it is possible to significantly reduce the occurrence of mounting defects of the component E on the substrate P and reduce the number of outputs of the mounting defect information J242, even if the component mounting operation is executed, the mounting defects do not actually occur. The component posture is also determined to be the mounting disapproval posture, and the number of times the mounting disapproval information J142 is output tends to increase excessively. As described above, if it is determined that the component posture that does not actually cause mounting failure is the mounting disapproval posture, the component E may be undesirably discarded.

Therefore, the correction unit 263 monitors the transition of the output number of each of the mounting disapproval information J142 and the mounting defect information J242 based on the history information D3. Then, when the correction unit 263 detects an excessive upward tendency of the number of times of output of the mounting disapproval information J142, the correction unit 263 corrects the determination condition D23 so that the upward tendency is alleviated.

For example, the correction unit 263 is generated at the timing (hereinafter, referred to as “excessive increase timing”) in which the number of times the mounting disapproval information J142 is output shows an excessive upward tendency with respect to the determination condition D23 generated by the learning unit 262. The corrected determination condition is generated by referring to the determination condition D23 and the determination condition D23 generated at the timing immediately before the excessive rise timing (hereinafter, referred to as “previous timing”). More specifically, the correction unit 263 sets an intermediate value between the judgment reference value D232 included in the judgment condition D23 generated at the excessive rise timing and the judgment reference value D232 included in the judgment condition D23 generated at the previous timing. A corrected judgment condition is generated as a judgment reference value after correction.

When the strict determination condition D23 that excessively narrows the allowable range of the mounting permission posture is generated, the correction unit 263 can suitably correct the determination condition D23. The determination unit 142 preferably determines whether or not the mounting permission posture that permits mounting on the substrate P is taken by determining the posture of the component E held by the suction nozzle 63 based on the corrected determination condition. Can be done.

<Control operation of machine learning device>
Next, the control operation of the machine learning device 20 will be described with reference to the flowchart of FIG. In the following description, it is assumed that the learning unit 262 executes deep learning as machine learning.

The machine learning device 20 communicates between the first captured image G1 and the second captured image G2 acquired by the first imaging unit 91 and the second imaging unit 92 and the inspection result information J2 output from the inspection device 28. Upon receiving via 22, the machine learning process is started. The storage data storage unit 24 sequentially stores and stores the first captured image G1 and the second captured image G2 each time it is received via the communication unit 22 (step a1), and sequentially stores the inspection result information J2. And store it (step a2).

Next, the teacher data generation unit 261 generates the first teacher data D1 (see FIG. 13) by adding the inspection result information J2 to the first captured image G1 and the second captured image G2 (step a3). ). The first teacher data D1 generated by the teacher data generation unit 261 is sequentially accumulated and stored in the teacher data storage unit 27 in a state of being divided according to the shape of the component E (step a4).

Next, the learning unit 262 executes machine learning (deep learning) using the first teacher data D1 stored in the teacher data storage unit 27 (step a5), and the determination unit 142 determines the posture of the component E. The determination condition D23 to be used is generated (step a6). The learning unit 262 outputs the determination condition information D2 (see FIG. 15) including the generated determination condition D23. When the determination condition information D2 is output from the learning unit 262, the determination condition information D2 is transmitted to the component mounting device 1 via the communication unit 22 (step a7).

In the component mounting device 1, the determination condition storage unit 144 stores the determination condition information D2 transmitted from the machine learning device 20, and the determination unit 142 refers to the determination condition information D2 and holds the component E in the suction nozzle 63. Judge the posture of. Then, the determination unit 142 outputs the mounting permission information J141 or the mounting disapproval information J142 indicating the determination result. The history storage unit 25 of the machine learning device 20 stores the history information D3 (see FIG. 17) including the history of the number of outputs of the mounting disapproval information J142.

Then, the correction unit 263 monitors the transition of the output number of the mounting disapproval information J142 based on the history information D3 (step a8), and determines whether or not the output number of the mounting disapproval information J142 is excessively increased. (Step a9). When the correction unit 263 detects an excessive upward tendency of the number of outputs of the mounting disapproval information J142, the correction unit 263 corrects the determination condition D23 so that the upward tendency is alleviated (step a10). The correction unit 263 outputs the corrected determination condition. When the correction unit 263 outputs the corrected determination condition, the determination condition is transmitted to the component mounting device 1 via the communication unit 22 (step a11).

In the component mounting device 1, the determination condition storage unit 144 stores the corrected determination condition transmitted from the machine learning device 20, and the determination unit 142 is held by the suction nozzle 63 with reference to the corrected determination condition. The posture of the component E is determined. Thereby, it is possible to preferably determine whether or not the mounting permission posture that permits mounting on the substrate P is taken.

<About the component mounting operation of the component mounting device>
Next, the component mounting operation of the component mounting device 1 will be described with reference to the flowchart of FIG.

When the command signal for starting the component mounting operation of the component E on the substrate P is input by the operator's operation, the component mounting device 1 starts the component mounting operation. First, the substrate P is conveyed on the conveyor 3 and positioned at a predetermined component mounting position. Then, the component supply control unit 12 controls the component supply device 5. The component supply device 5 supplies the component E to the component supply position P1 by intermittently sending out the component storage tape 100.

Next, the head drive control unit 13 controls the movement of the head unit 6 by the first drive mechanism 7 and the second drive mechanism 8 on the horizontal plane with respect to the X direction and the Y direction. The first drive mechanism 7 and the second drive mechanism 8 move the head unit 6 so that the suction nozzle 63 is located directly above the component supply position P1. Next, the head drive control unit 13 controls the elevating operation of the suction nozzle 63 by the nozzle elevating drive mechanism 67. The nozzle elevating drive mechanism 67 lowers the suction nozzle 63. The lowered suction nozzle 63 sucks and holds the component E supplied to the component supply position P1 (step b1). When the suction nozzle 63 holds the component E, the head drive control unit 13 raises the suction nozzle 63. At this time, the second imaging unit 92 images the component E held by the suction nozzle 63 from the side to acquire the second captured image G2 (step b2).

Next, the head drive control unit 13 controls the movement of the head unit 6 by the first drive mechanism 7 and the second drive mechanism 8 on the horizontal plane with respect to the X direction and the Y direction. The first drive mechanism 7 and the second drive mechanism 8 move the head unit 6 so that the suction nozzle 63 is located directly above the component mounting position on the substrate P on the conveyor 3. At this time, the first imaging unit 91 images the component E held by the suction nozzle 63 from below to acquire the first captured image G1 (step b2).

When the first imaging unit 91 acquires the first captured image G1 and the second imaging unit 92 acquires the second captured image G2, the feature amount calculation unit 141 is based on the first captured image G1 and the second captured image G2. , The feature amount related to the component posture is calculated for each determination index (step b3).

Next, the determination unit 142 reads out the determination condition D23 included in the determination condition information D2 stored in the determination condition storage unit 144 (step b4). The determination unit 142 reads out the determination condition D23 corresponding to the component E held by the suction nozzle 63. Then, the determination unit 142 compares the feature amount calculated by the feature amount calculation unit 141 with the determination reference value D232 included in the determination condition D23 for each determination index D231, so that the component held by the suction nozzle 63 It is determined whether the posture of E is the mounting permitted posture or the mounting disallowed posture (step b5, step b6). When the feature amount satisfies the determination reference value D232 in all the determination indexes D231, the determination unit 142 determines that the posture of the component E held by the suction nozzle 63 is the mounting permission posture, and outputs the mounting permission information J141. (Step b7). On the other hand, when the feature amount does not satisfy the determination reference value D232 in at least one determination index D231, the determination unit 142 determines that the posture of the component E held by the suction nozzle 63 is the mounting disallowed posture. The mounting disapproval information J142 is output (step b9).

When the mounting permission information J141 is output from the determination unit 142, the head drive control unit 13 is arranged directly above the component mounting position, lowers the suction nozzle 63 holding the component E, and lowers the component E to the substrate P. (Step b8). In this way, the component E can be mounted on the substrate P. On the other hand, when the mounting disapproval information J142 is output from the determination unit 142, the head drive control unit 13 operates the head unit 6 so as to execute a component disposal operation for discarding the component E held by the suction nozzle 63. (Step b10). In this way, by discarding the component E that has taken the mounting disapproval posture, it is possible to reduce the occurrence of mounting failure of the component E on the substrate P.

1 Parts mounting device 1S Parts mounting system 6 Head unit 63 Suction nozzle (holder)
9 Parts recognition imaging device 91 1st imaging unit 92 2nd imaging unit 10 Control unit 14 Parts attitude determination unit 20 Machine learning device 22 Communication unit 23 Operation unit 24 Accumulated data storage unit 25 History storage unit 26 Central processing unit 261 Teacher data generation Part 262 Learning part 263 Correction part 27 Teacher data storage part 28 Inspection device D1 First teacher data D1A Second teacher data D2 Judgment condition information D3 History information G1 First captured image G2 Second captured image J1 Judgment result information J141 installed Permission information J142 Installation disapproval information J2 Inspection result information J241 Installation good information J242 Installation failure information

Claims (9)

A component mounting device that mounts components on a board,
A machine learning device connected to the component mounting device so as to be capable of data communication is provided.
The component mounting device is
A head unit that has a holder for holding parts and mounts the parts held by the holder on a board.
An imaging device that captures an image of a component held by the holder and acquires an image before the component is mounted on the substrate by the head unit.
Based on the captured image, the posture of the component held by the holder is determined, and in the case of the component posture that allows mounting on the board, the mounting permission information is output, and in the case of the component posture that disallows mounting on the board. Including a component posture determination unit that outputs mounting disapproval information,
The machine learning device
A teacher data generation unit that generates teacher data by adding inspection result information indicating the inspection result of the mounting state of the component to the substrate to the captured image or the feature amount related to the component posture based on the captured image.
A teacher data storage unit that sequentially accumulates and stores the teacher data generated by the teacher data generation unit, and a teacher data storage unit.
By executing machine learning using each of the teacher data stored in the teacher data storage unit, a learning unit that generates a determination condition used by the component attitude determination unit to determine the attitude of the component. Including component mounting system. The teacher data generation unit generates, as the teacher data, first teacher data in which the inspection result information is added to the captured image.
The component mounting system according to claim 1, wherein the learning unit executes deep learning using the first teacher data as the machine learning. The teacher data generation unit generates, as the teacher data, second teacher data in which the test result information is added to the feature amount.
The component mounting system according to claim 1, wherein the learning unit executes supervised learning using the second teacher data as the machine learning. The machine learning device
A history storage unit that stores history information representing the history of the number of times of output of the mounting disapproval information by the component posture determination unit for each predetermined period, and a history storage unit.
Further includes a correction unit that monitors the transition of the number of outputs based on the history information and corrects the determination condition so that the upward tendency is alleviated when the excessive upward tendency of the number of outputs is detected. , The component mounting system according to any one of claims 1 to 3. The teacher data storage unit stores the teacher data separately for each shape of the part, and stores the teacher data.
The component mounting system according to any one of claims 1 to 4, wherein the learning unit executes the machine learning for each component shape by using the teacher data classified for each component shape. The image pickup device
A first imaging unit that captures a component held by the holder from a first direction and acquires a first captured image.
Any of claims 1 to 5, including a second imaging unit that captures a component held by the holder from a second direction intersecting the first direction to acquire a second captured image. The component mounting system according to item 1. Further equipped with an inspection device which is connected to the machine learning device so as to be capable of data communication and inspects the mounted state of the components mounted on the board by the head unit.
The component mounting system according to any one of claims 1 to 6, wherein the teacher data generation unit acquires the inspection result information from the inspection device and generates the teacher data. The inspection device inspects the mounting state of the component on the board after the head unit mounts the component on the board and before the reflow process of fixing the component to the board using solder is performed. , The component mounting system according to claim 7. The machine learning device includes an operation unit that receives an operation for inputting the inspection result information.
The component mounting system according to any one of claims 1 to 6, wherein the teacher data generation unit acquires the inspection result information input by an operation on the operation unit and generates the teacher data.

Images