WO2020012588A1

WO2020012588A1 - Image processing method and component mounting machine

Info

Publication number: WO2020012588A1
Application number: PCT/JP2018/026262
Authority: WO
Inventors: 天野雅史; 横井勇太
Original assignee: 株式会社Ｆｕｊｉ
Priority date: 2018-07-12
Filing date: 2018-07-12
Publication date: 2020-01-16
Also published as: CN112369135A; JPWO2020012588A1; EP3823427B1; EP3823427A1; US11546528B2; US20210274104A1; EP3823427A4; JP6903823B2

Abstract

The present invention relates to an image processing method which makes it easy to see an image obtained by imaging a plurality of components at once, the method including: an imaging step for capturing, in one image, component holding states of a plurality of suction nozzles mounted on a mounting head; an image division step for dividing an area of a predetermined component holding state from image data on the plurality of component holding states, the image data being obtained through the imaging step; a direction conversion step for converting the direction of the component holding state in the divided image data obtained through the image division step; and a display step for displaying an image based on the divided image data subjected to the direction conversion step.

Description

Image processing method and component mounting machine

The present invention relates to an image processing method for a plurality of components suction-held by a plurality of suction nozzles of a rotary head, and a component mounter that executes the layer processing.

(4) In a component mounter, a mounting head provided with a suction nozzle is configured to move inside the device, sucks and holds a component supplied from a component supply device, and mounts the component on a substrate transported by a transport device. The component mounter of Patent Document 1 below uses a rotary head that rotates a plurality of nozzles arranged in a circumferential direction as a mounting head. The rotary head includes a side camera for capturing an image of the tip of the suction nozzle, and an optical system for forming a plurality of images on the image sensor by a plurality of mirrors. Therefore, in the first conventional example, an operator can simultaneously check a plurality of situations from the monitor by simultaneously capturing images at a plurality of places.

Patent Document 2 below describes an inspection method using an image for processing in each step in a mounting production system for mounting components on a printed circuit board. In this inspection method, for example, when the substrate is sent in the order of the printing process, the mounting process, and the reflow process, the substrate is imaged by the camera of the inspection device each time the substrate passes through each process, and the state inspection after each process becomes possible. ing. At that time, the magnification and the direction of the verification image data may be different, and the size and the direction of the inspection image displayed on the monitor may vary. Therefore, in the second conventional example, the inspection images are aligned by enlarging or reducing the verification image data or by performing a direction conversion process.

JP-A-2017-220544 JP 2004-361145 A

(4) In the component mounting machine of the first conventional example, images of a plurality of suction nozzles captured simultaneously are displayed at a time. At this time, since images are taken via a plurality of mirrors, an inverted image may be displayed on the monitor for some of the suction nozzles. In the component mounter, a plurality of components held by the rotary head are imaged at once by the part camera, and the status of each component is checked, but images of components with different orientations are displayed on the monitor. Will be done. If the same object is displayed on the monitor in different directions, it is not possible to cope with the method of aligning the directions of the respective image data as in the second conventional example.

Therefore, an object of the present invention is to provide an image processing method and a component mounter that make it easier to view images obtained by capturing a plurality of components at one time, in order to solve such a problem.

An image processing method according to an aspect of the present invention includes an imaging process of imaging each component holding state of a plurality of suction nozzles mounted on a mounting head as one image, and a plurality of component holdings obtained by the imaging process. Image division processing for dividing a region relating to a predetermined component holding state with respect to image data in a state; direction conversion processing for converting the direction of the component holding state with respect to the divided image data divided by the image division processing; Display processing for displaying an image based on the divided image data on which the conversion processing has been performed.

According to another aspect of the present invention, there is provided a component mounter configured to move a substrate transport device that transports a substrate into the device, a component supply device that supplies components to be mounted on the substrate, and a mounting head including a plurality of suction nozzles in the device. A component mounting device that mounts components on the substrate by the suction nozzle; an imaging device that captures, as one image, a component holding state of the plurality of suction nozzles mounted on the mounting head; and the imaging device. An image data processing device that divides the image data obtained by each component holding state image of the suction nozzle and creates converted image data in which the direction of the component holding state is converted for each of the divided image data, A display device for displaying an image based on the converted image data created by the image data processing device.

According to the above configuration, one image obtained by capturing the component holding state of each of the plurality of suction nozzles is divided into predetermined regions for each component holding state of the suction nozzle, and the direction of each divided image data is converted. Thereby, an easy-to-view image in which the directions of the components are aligned is obtained.

FIG. 1 is an external perspective view illustrating an embodiment of a component mounter. FIG. 4 is a diagram showing a part of the mounting head in a simplified manner, as viewed from the lower side in the rotation axis direction. FIG. 3 is a diagram conceptually showing an optical system mounted on a mounting head. It is the block diagram which showed the control system of the component mounting machine simply. FIG. 8 is a diagram showing a display screen of a component holding state of a suction nozzle captured by a side camera. FIG. 8 is a diagram showing a display screen of a component holding state of a suction nozzle captured by a part camera. FIG. 7 is a diagram illustrating an image obtained by dividing a component holding state of a suction nozzle captured by a part camera for each component.

Next, an embodiment of an image processing method and a component mounter according to the present invention will be described below with reference to the drawings. FIG. 1 is an external perspective view of a component mounter shown partially through. The component mounter 1 of the present embodiment is a so-called mounter for mounting components on a board to be transported. A large opening is formed on the side surface in the width direction of the component mounter 1 so that the board can be transported between adjacent component mounters, and the board manufacturing line can be arranged with a screen printer or another component mounter. Is configured. FIG. 1 shows a state in which two component mounters 1 are mounted side by side on a base 9. The description will be made on the assumption that the body width direction, which is the direction in which the substrate is transported, is the X-axis direction, the body longitudinal direction orthogonal thereto is the Y-axis direction, and the height direction is the Z-axis direction.

The component mounter 1 includes a board transfer device 2 for transferring a board, a component supply device 3 for supplying components, a component mounting device 4 for mounting components taken out from the component supply device 3 onto a board, and the like. . In the substrate transfer apparatus 2,

transfer units

201 and 202, which are belt conveyors having the same configuration according to the size of the substrate, are provided side by side, and the transfer of the substrate and the mounting of components to the substrate are performed at two locations. . Each of the

transfer units

201 and 202 is provided with a clamp mechanism, and can position the transferred substrate in the apparatus.

The component feeder 3 has a tape feeder 18 holding components. An opening is formed in the component mounter 1 at a front portion of the main body cover 5, a device table 19 is installed therein, and a plurality of tape feeders 18 are detachably attached. Next, the component mounting device 4 is a working device that takes out a component from the tape feeder 18 and mounts it on a substrate. The mounting head 6 (see FIG. 2) having the suction nozzle 21 can move on a horizontal XY plane. Configuration.

には Two Y-axis rails 11 are fixed on the ceiling side of the main body cover 5 in the front-rear direction of the machine body, and a Y-axis slide 12 is slidably mounted on the Y-axis rail 11. A nut member 13 is fixed to the Y-axis slide 12, and a ball screw mechanism is formed in which a screw shaft 15 connected to a Y-axis servomotor 14 passes through the nut member 13 and is screwed. An X-axis rail is fixed to the Y-axis slide 12, and the mounting head 6 mounted on the X-axis slide is slidably assembled in the X-axis direction. A ball screw mechanism that converts the rotation of the X-axis servo motor into a linear motion is formed on the X-axis slide.

パーツ A parts camera 17 is provided in the moving range of the mounting head 6 moving in the machine. In the present embodiment, it is provided between a component supply device 3 that supplies components and a substrate transport device 2 that transports and holds a substrate. The part camera 17 captures an image of the component held by the mounting head 6 from below, and can detect the damage of the component and the holding position and orientation of the held component from the image data. The mounting head 6 has a plurality of suction nozzles 21 mounted thereon, and the parts camera 17 can image all the components held and held therein at one time.

FIG. 2 is a simplified view of a part of the mounting head 6, and is shown from the lower side in the rotation axis direction. FIG. 3 is a view conceptually showing an optical system mounted on the mounting head 6. The mounting head 6 has a nozzle holder 16 connected to a turning motor, and a plurality of suction nozzles 21 are attached at equal intervals in the circumferential direction. In the mounting head 6, the two suction nozzles 21 are arranged at the P1 position and the P2 position, which are the component pick-up positions, respectively, by rotating indexing where the nozzle holder 16 rotates. It is configured to move up and down by an elevating mechanism that converts the data to

In such a mounting head 6, the components are sucked and held by the evacuation accompanying the elevating operation of the suction nozzle 21, and the components are mounted on the substrate by applying a positive pressure. The component mounter 1 is an imaging device for confirming a suction holding state of a component at a nozzle tip portion, a non-holding state after mounting, and the like (collectively referred to as a “component holding state”) with respect to the suction nozzle 21 that performs work. Is provided. One of them is the above-described parts camera 17, which captures images of a plurality of components sucked and held by the mounting head 6 passing above.

In addition, the mounting head 6 has an imaging device including a side camera 23 for imaging the tip of the suction nozzle from the side, and an optical system 25 for forming images at a plurality of locations on the imaging device by a plurality of mirrors. Is provided. FIGS. 2 and 3 show an optical system 25 relating to the side camera 23 of the mounting head 6. In the mounting head 6, each time the nozzle holder 16 rotates by a certain angle, the suction nozzle 21 is sequentially turned to the P1 position or the P2 position. The side camera 23 captures images of two suction nozzles 21 (21a, 21b, 21c, 21d) positioned before and after the P1 position and the P2 position, respectively.

That is, in this imaging apparatus, the nozzle tips of the

suction nozzles

21a and 21c immediately before sucking and holding the component at the P1 position or the P2 position and the nozzle tips of the

suction nozzles

21b and 21d immediately after sucking and holding the component can be imaged at a time. ing. Therefore, as shown by the four lines, the optical system 25 reflects the light reflected from the four suction nozzles 21 (21a, 21b, 21c, 21d) in the direction opposite to the direction shown by the arrow, and The image sensor of the camera 23 is configured to receive light.

When viewed from the side camera 23, the optical system 25 changes the optical path from a vertical direction to a horizontal direction by a plate-like mirror 28 and is divided into a P1 side and a P2 side by a triangular prism block mirror 29. On the separated sides, the direction is similarly reflected by the plate-

like mirrors

31 and 32 and the next plate-

like mirrors

33 and 34 to change the direction. Further, two triangular prism block mirrors 35 and 36 are provided on the P1 side and the P2 side, respectively. Divide into directions. Then, light is reflected from each of the plate mirrors 37, 38, 39, and 40 and reaches the

suction nozzles

21a, 21b, 21c, and 21d.

FIG. 4 is a block diagram showing a control system of the component mounter 1. In the control device 10, a microprocessor (CPU) 41, a ROM 42, a RAM 43, and a nonvolatile memory 44 are connected via a bus line. The CPU 41 controls the entire control device. The ROM 42 stores system programs and control parameters executed by the CPU 41, and the RAM 43 stores temporary calculation data and image data. The non-volatile memory 44 stores information necessary for processing performed by the CPU 41, and stores a sequence program of the component mounter 1 and the like.

The non-volatile memory 44 also stores a production job (production program) for performing a predetermined operation on the board by the mounting head 6, an image processing program for displaying a component holding state regarding the suction nozzle 21, and the like. Have been. The image processing program according to the present embodiment processes image data of the component holding state of the suction nozzle 21 captured by the part camera 17 and the side camera 23 by a predetermined image processing method and displays the processed image data on a monitor.

The component mounter 1 is provided with an operation display device 7 provided with a touch panel type monitor and a button type input unit at the front of the body, and is connected to the control device 10. A general-purpose computer 50 having a monitor is connected to the control device 10 as a management device for managing the entire system of the component mounter 1. The computer 50 displays an image processed by the control device 10 on a monitor, acquires image data of a component holding state relating to the suction nozzle 21, and stores an image processing program to perform image processing by itself. Good. Therefore, the display on the monitor described below corresponds to both the operation display device 7 and the computer 50.

Next, the operation of the component mounter 1 will be described. In the component mounter 1, the mounting head 6 moves on the XY plane by drive control in accordance with the program, and the component supplied from the component supply device 3 is sucked and held by the suction nozzle 21. At this time, in the mounting head 6, the plurality of suction nozzles 21 are sequentially moved to the P1 position or the P2 position by the rotation indexing of the rotation of the nozzle holder 16, the suction nozzles 21 are lowered at each position, and the evacuation is performed. Then, the components sent by the tape feeder 18 are sucked and held. As shown in FIG. 2, 20 suction nozzles 21 are mounted on the mounting head 6, and all the suction nozzles 21 hold components by suction.

(4) Thereafter, the mounting head 6 moves from the component supply device 3 to the substrate transport device 2, and the components are mounted on the transported substrate. In such a series of operations, while the component is being taken out by the component supply device 3, the component holding state of the suction nozzle 21 is imaged by the side camera 23. In the course of moving from the component supply device 3 to the board transfer device 2 for mounting components, the component camera 17 captures an image of the component holding state of the suction nozzle 21.

In the image processing program, when the operator selects the confirmation mode from the operation display device 7 or the computer 50, the image of the component holding state captured by the side camera 23 or the part camera 17 is displayed on the monitor. ing. Although it may be displayed in a normal production process, it is generally used for a test run when a production program is created, a first product inspection at the start of production, and the like. Therefore, first, image processing of an image captured by the side camera 23 will be described. FIG. 5 is a diagram showing a display screen of the component holding state of the suction nozzle 21 captured by the side camera 23.

In the mounting head 6, the components 80 are sequentially suction-held by the suction nozzles 21 at the P1 position and the P2 position. Then, two

suction nozzles

21a and 21b and two

suction nozzles

21c and 21d positioned at the front and rear are imaged by the side camera 23 in accordance with the suction operation. The side camera 23 captures images of the tips of the four suction nozzles 21 as the objects, that is, images of the component holding state, as one image by the optical system 25 shown in FIG. 3, and as shown in FIG. It is displayed on the monitor 60 at the same time. In particular, in this embodiment, image processing is performed on the normal pattern on the upper side of the drawing by the image processing program, and an image of the conversion pattern on the lower side of the drawing is displayed on the monitor 60.

像 In the optical system 25 of the side camera 23, the image is inverted by the reflection mirror. Therefore, for example, even if the image on the P1 side is turned upside down like a normal pattern image, the image as it is has been displayed on the monitor 60 until now. However, the operator who has seen the image of the normal pattern will check the original state of the suction nozzle 21 while imagining the original state of the suction nozzle 21 from the vertically inverted

images

61A and 61B. . Therefore, in the image processing program of the present embodiment, a direction conversion process for changing the direction of the component holding state to the original direction of the nozzle imagined by the operator is performed.

First, of the image data captured by the side camera 23, an area that needs to be changed in direction is divided, and an image that is turned upside down as shown in the converted pattern image is created by converting the pixel values. In the case of the side camera 23, the regions of the

images

61A and 61B are objects of the conversion, and it is also known in advance that the images are inverted by 180 °. Therefore, based on the image data captured by the side camera 23, the regions of the

images

61A and 61B are divided and pixel values are converted to create inverted images 61Ax and 61Bx shown in the converted pattern images. The converted images 61Ax and 61Bx are replaced with the

images

61A and 61B, and are displayed on the monitor 60 as one conversion pattern image together with the

images

61C and 61D.

変換 In this conversion pattern image, the tips of the

suction nozzles

21a, 21b, 21c, 21d are all unified downward, so that the operator can intuitively understand the holding state of each suction component. The conversion pattern image displayed on the monitor 60 is, as shown in FIG. 5, a part holding state of the

suction nozzles

21 a, 21 b, 21 c, and 21 d displayed on the monitor 60 as one image. The

images

61C and 61D of the suction nozzles 21 may also be divided, and the images 61Ax, 61Bx, 61C and 61D of the component holding state of each suction nozzle 21 may be displayed on the monitor 60 one screen at a time.

Next, image processing of an image captured by the part camera 17 will be described. FIG. 6 is a diagram showing a display screen of the component holding state of the suction nozzle 21 captured by the part camera 17, and particularly shows the state of the component 80 displayed on the monitor 60. FIG. 7 is a diagram showing an image obtained by dividing the component holding state of the suction nozzle 21 captured by the part camera 17 for each component 80.

(4) The mounting head 6 holding the component 80 by suction passes over the part camera 17 before being mounted on the board. At this time, the component 80 held by the suction nozzle 21 is imaged, and component determination and positional deviation determination are performed based on the image data. A plurality of parts 80 imaged by the part camera 17 are displayed on the monitor 60 as shown in FIG. However, in the mounting head 6 which is a rotary head, the direction of the component 80 is scattered as shown in the normal pattern of FIG. 6 because the suction nozzle 21 turns. Therefore, in the image processing program of the present embodiment, a direction conversion process is performed so that the directions of all the components 80 are aligned.

領域 Among the image data captured by the parts camera 17, the area requiring the direction conversion is divided, and further, all the components 80 are horizontally converted by the conversion of the pixel values, and the image shown in the conversion pattern is created. When an image is taken by the parts camera 17, the position of the component 80 in the image can be specified from the center position information on each suction nozzle 21 and the direction of the component 80 sucked and held (from the turning angle information on each suction nozzle 21). Angle) can also be specified. Therefore, based on the image data captured by the parts camera 17, the regions of the

images

63A, 63B, 63C, 63D,... Shown on the screen of the normal pattern in FIG.

Then, the divided images 63A and the like are rotated by a predetermined angle by the conversion of the pixel values so that all the components 80 are directed sideways with respect to the screen, and the converted images 63Ax, 63Bx, 63Cx, 63Dx,. A single conversion pattern image is created, and an image in a component holding state in which a plurality of components 80 are arranged on the circumference is displayed on the monitor 60 in the same manner as before conversion. Therefore, in the conversion pattern image shown in FIG. 6, the components 80 having different directions for all the suction nozzles 21 are all unified in the same horizontal direction, and the operator can intuitively understand the holding state of each suction component. it can.

(7) Image data captured by the part camera 17 may be displayed together with positional deviation information as another display form as shown in FIG. Heretofore, as shown as a normal pattern, the captured image has been displayed as it is for each area of the suction nozzle 21 as it is. However, since the direction of the component 80 is scattered due to the swirling suction nozzle 21, it is difficult to understand the numerical value of the shift amount corresponding to the image of the component 80.

Here, based on the center of the suction nozzle 21, how much error occurs in the center position of the component 80 held by suction in the X-axis direction, the Y-axis direction, and the rotation direction of the component mounter 1 shown in FIG. Is calculated, and the numerical value is displayed. However, when the displayed image of the component 80 is displayed in various directions, the operator matches the deviation amount in the direction indicated by the numerical value with the image of the component 80 displayed on the monitor 60. It was difficult to understand.

Therefore, in the image processing program of the present embodiment, a direction conversion process is performed so that the directions of all the components 80 are aligned. First, in the normal pattern shown in FIG. 7,

images

65A, 65B, 65C, 65D... Are divided for each region of all components 80 based on the information of each suction nozzle 21 as in the case of FIG. Then, images 65Ax, 65Bx, 65Cx, 65Dx,... Rotated by a predetermined angle for each divided image are created so that the component 80 is oriented vertically by the conversion of the pixel values. After that, the processed images 65Ax, 65Bx, 65Cx, 65Dx,... Are displayed on the monitor 60 as sequentially arranged conversion pattern images.

Therefore, in the conversion pattern image shown in FIG. 7, the components 80 having different directions for each of the suction nozzles 21 are all displayed as an image in which the longitudinal direction is unified in the Y-axis direction. The displacement amount in the axial direction and the displacement amount in the rotation direction can be intuitively understood. In particular, this direction change is performed in the vertical direction in accordance with the direction in which the component 80 is mounted, that is, the Y-axis direction which is the front-back direction of the component mounter 1. Therefore, the operator can easily imagine the amount of displacement when the component 80 is mounted on the board.

By the way, in the above-described embodiment, the components 80 are all expressed as the same component, but the components handled by the mounting head 6 are not only the same type but also a case where different types of components 80 are included in one image. It may be. In FIGS. 6 and 7, in the conversion pattern images, the longitudinal directions of all the components 80 are unified in the X-axis direction or the Y-axis direction, but the operator operates the operation display device 7 or the computer 50. Then, by instructing the control device, a method of unifying the direction of the component 80 may be selected. For example, various types of components 80 are displayed in a direction based on when component shape data for determining positional deviation of the center and deviation of the turning angle is created. In this case, one conversion pattern image may include an image whose longitudinal direction is in the X-axis direction and an image whose longitudinal direction is in the Y-axis direction, depending on the type of the component 80. In addition, the display may be made according to the direction of the various components 80 supplied in the component supply device 3.

Although the case where the image-processed conversion pattern image is displayed on each monitor 60 without distinguishing the operation display device 7 and the computer 50 has been described, the conversion pattern image may be mainly displayed on the computer 50. At this time, divided image data such as a normal pattern captured by the side camera 23 shown in FIG. 5 is stored in the RAM 43 of the control device 10 and transmitted to the computer 50 as it is. Then, the computer 50 creates an image of the conversion pattern based on the image data of the normal pattern as described above, and causes the monitor 60 to display the image. In this case, when the user wants to view the image of the conversion pattern on the operation display device 7 of the component mounter 1, the image of the conversion pattern created by the computer 50 may be transmitted to the control device 10 and displayed.

As mentioned above, although one Embodiment of this invention was described, this invention is not limited to these, A various change is possible in the range which does not deviate from the meaning.
For example, in the example of FIG. 6, the regions of the

images

63A, 63B, 63C, 63D... Are divided for all the components 80. Is also good. The same applies to the example of FIG.
In addition, for example, in the above-described embodiment, the rotary head is described as an example of the mounting head 6 on which the plurality of suction nozzles 21 are mounted. However, another type of mounting head may be used. The imaging device including the side camera 23 and the optical system 25 may have a different configuration.

DESCRIPTION OF SYMBOLS 1 ... Component mounting machine 2 ... Substrate conveyance device 3 ... Component supply device 4 ... Component mounting device 6 ... Mounting head 7 ... Operation display device 10 ... Control device 16 ... Nozzle holder 17 ... Part camera 21 (21a, 21b, 21c, 21d) ... Suction nozzle 23 ... Side camera 25 ... Optical system 50 ... Computer 60 ... Monitor ...

Claims

An imaging process of imaging each component holding state of the plurality of suction nozzles mounted on the mounting head as one image,
For a plurality of image data of the component holding state obtained by the imaging process, an image dividing process of dividing an area related to a predetermined component holding state,
For the divided image data divided by the image division processing, a direction conversion processing for converting the direction of the component holding state,
A display process of displaying an image based on the divided image data on which the direction conversion process has been performed;
An image processing method comprising:
The display processing replaces the plurality of divided images in the component holding state, the directions of which have been changed by the direction conversion processing, and displays the plurality of images in the component holding state obtained by the imaging processing as one image. The image processing method according to claim 1.
A division data storage process for storing divided image data divided for each component holding state image of the suction nozzle by the image division process; and the direction conversion process includes a plurality of divisions stored by the division data storage process. 2. The image processing method according to claim 1, wherein the conversion is performed on the image data.
The said component holding state is what image | photographed the front-end | tip part of these suction nozzles from the side surface side, and the said direction conversion process matches the direction of the said suction nozzles by changing the direction of divided image data. Item 4. The image processing method according to any one of Items 3.
The component holding state is obtained by capturing an image of the component suction-held by the plurality of suction nozzles from the opposite side of the holding surface, and the direction conversion process matches the direction of the component by changing the direction of the divided image data. The image processing method according to claim 1, wherein:
6. The image processing method according to claim 5, wherein in the display processing, the plurality of divided image data, the directions of which have been converted by the direction conversion processing, are rearranged vertically and horizontally and displayed as one image.
A substrate transfer device for transferring the substrate into the machine,
A component supply device that supplies a component to be mounted on the board,
A component mounting device that moves a mounting head having a plurality of suction nozzles in the machine, and mounts components on the substrate by the suction nozzle;
An imaging device for imaging each component holding state of the plurality of suction nozzles mounted on the mounting head as one image,
Image data processing for dividing the image data obtained by the imaging device for each component holding state image of the suction nozzle, and creating converted image data in which the direction of the component holding state is converted for each of the divided image data. Equipment and
A display device that displays an image based on the converted image data created by the image data processing device,
Component mounting machine.
8. The component mounter according to claim 7, wherein the imaging device is configured to simultaneously image the side surfaces of the tip ends of the plurality of suction nozzles by a single camera via an optical system using a plurality of mirrors.
8. The component mounter according to claim 7, wherein the imaging device is configured to image all components sucked and held by the plurality of suction nozzles mounted downward from below.
The component according to any one of claims 7 to 9, wherein the image data processing device converts a direction of the component holding state for the divided image data based on angle information related to position information of the suction nozzle. Mounting machine.