JP2023110671A - Image processing device and image processing method - Google Patents

Abstract

To provide a technology with which, when an error occurs in verifying a component of interest, it is possible to generate a new template easily.SOLUTION: Disclosed is an image processing device for processing the image data of components. The image processing device comprises: an imaging unit for capturing an image of the component of interest; a storage unit for storing an original template that is generated in advance and indicates a plurality of features of the component of interest; and a display unit which, when the rate of match of the image data of the component of interest acquired by the imaging unit with the plurality of features of the original template stored in the storage unit is lower than a prescribed threshold, displays a new template in which the plurality of features of the component of interest are extracted from the image data having been acquired by the imaging unit.SELECTED DRAWING: Figure 5

Description

The technology disclosed in this specification relates to an image processing device and an image processing method for processing image data of a component.

Patent Literature 1 discloses an apparatus that creates a template indicating the characteristic portion of a component based on the image data of the captured component. When creating a template with this device, an operator sets an area including the outer shape of the component for the image data of the component rotated at an angle to be mounted on the board. Then, a template is created from the images of the parts in the region.

Japanese Patent Application Laid-Open No. 2004-119408

When mounting a component on a board, the image data of the target component to be actually mounted on the board is acquired, and by matching the acquired image data with the created template, the target component is confirmed to be the correct component. is identified. If the matching of the image data fails (if the target part cannot be recognized), it is necessary to correct the created template. This specification provides a technique that can easily generate a new template when an error occurs in matching target parts.

This specification discloses an image processing device that processes image data of a component. The image processing apparatus includes an imaging unit for imaging a target part, a storage unit for storing an original template generated in advance and indicating a plurality of characteristic parts of the target part, and the target part acquired by the imaging unit. and the plurality of characteristic portions of the original template stored in the storage unit are lower than a predetermined threshold value, the target part is extracted from the image data acquired by the imaging unit and a display unit for displaying a new template obtained by extracting a plurality of characteristic portions of.

In this image processing device, when it is determined that the matching rate between the feature part of the original template and the image data of the target part is lower than a predetermined threshold value, a new template in which the feature part is extracted from the image data of the target part is displayed. be done. In other words, in this image processing apparatus, when an error occurs in matching the target component, a new template based on the image data of the target component to be actually mounted is displayed on the display unit. Therefore, in the image processing apparatus described above, since the operator does not need to correct the original template, the working efficiency is improved.

This specification also discloses an image processing method for processing image data of a component. The image processing method includes an image capturing step of capturing an image of a target component, image data of the target component acquired by the image capturing step, and an original template, which is prepared in advance and indicates a plurality of characteristic portions of the target component. a display step of displaying a new template obtained by extracting the plurality of characteristic portions of the target part from the acquired image data when a matching rate of the plurality of characteristic portions is lower than a predetermined threshold.

Schematic which shows the structure of a component mounting line. Schematic which shows the structure of a component mounter. FIG. 2 is a block diagram showing the configuration of a management device and a control device of a mounter; The figure which shows an example of an original template. 4 is a flowchart showing processing executed by a control device of the mounter; 4 is a flowchart showing the contents of new template generation processing; FIG. 4 is a diagram for explaining processing for generating a new template; FIG. 4 is a diagram for explaining processing for generating a new template; FIG. 4 is a diagram for explaining processing for generating a new template;

The main features of the embodiments described below are listed. It should be noted that the technical elements described below are independent technical elements, exhibiting technical usefulness alone or in various combinations, and are limited to the combinations described in the claims as filed. not a thing

In one embodiment of the present technology, the image processing device has a match rate between the image data of the target part acquired by the imaging unit and the plurality of feature parts of the original template stored in the storage unit. You may further have a control part which detects. The control unit may generate a new template by extracting a plurality of characteristic parts of the target part from the image data acquired by the imaging unit when the matching rate is lower than the predetermined threshold.

With such a configuration, a new template can be generated in the image processing device.

In an embodiment of the present technology, the control unit may set an existence region in which the target part exists in the image data acquired by the imaging unit, and the The plurality of characteristic portions of the target part may be extracted.

In such a configuration, since the presence area where the target part exists is set for the image data, a specific part in the area where the target part does not exist (for example, the background of the target part) is mistakenly regarded as the characteristic part. Extraction can be suppressed.

In an embodiment of the present technology, the control unit may extract an outer shape of the target part from the image data acquired by the imaging unit, and set the existing region based on the extracted outer shape. good too.

With such a configuration, since the outline of the target component is extracted from the image data, the existing region can be set with high accuracy.

In one embodiment of the present technology, the control unit may set the existing region for the image data acquired by the imaging unit based on the outline of the target part included in the original template. .

In such a configuration, the existence area of the target part is set for the image data based on the existence area set in the original template. Since the existing region in the original template is used, the efficiency of the new template generation processing can be improved. Note that the target component is generally of the same type as that of the original template, and the existence area does not change significantly between the original template and the image data of the target component.

In an embodiment of the present technology, the plurality of features may include a plurality of seeklines that intersect a contour of the target part.

With such a configuration, the matching rate is calculated based on the optical characteristics on the seek line, so the matching between the original template and the image data of the target part can be executed in a short time.

In an embodiment of the present technology, the image processing device may further include an operation unit capable of accepting an operation from an operator to correct the new template displayed on the display unit.

With such a configuration, for example, when the operator determines that the new template displayed on the display unit is not appropriate, the operator can correct the new template.

(Example)
A component mounter according to an embodiment will be described below with reference to the drawings. As shown in FIG. 1, the component mounter 10 is communicably connected to the management device 50 . The management device 50 is communicatively connected to the multiple component mounters 10 and manages the multiple component mounters 10 . A component mounting line 100 is configured by a management device 50 and a plurality of component mounters 10 . The management device 50 controls the entire component mounting line 100 by controlling the operation of each component mounter 10 . That is, the management device 50 identifies the type and position of the component 4 to be mounted on the board 2 by the component mounter 10 for each of the component mounters 10, and instructs the component mounter 10 accordingly. . A plurality of component mounters 10 operate based on instructions from the management device 50 to mount the necessary components 4 on the board 2 .

A component mounter 10 is a device that mounts a component 4 on a board 2 . The component mounter 10 is also called an electronic component mounting device or a chip mounter. As shown in FIG. 2, the component mounter 10 includes a component mounting section 22, an interface device 24, and a control device 30. The component mounting section 22 includes a plurality of component feeders 12 , a feeder holding section 14 , a mounting head 16 , a head moving device 18 , an imaging device 19 and a board conveyor 20 .

Each component feeder 12 accommodates multiple components 4 . The component feeder 12 is detachably attached to the feeder holding portion 14 and supplies the component 4 to the mounting head 16 . A specific configuration of the component feeder 12 is not particularly limited. Each component feeder 12 may be, for example, a tape-type feeder that accommodates a plurality of components 4 on a winding tape, a tray-type feeder that accommodates a plurality of components 4 on a tray, or the like.

The feeder holding portion 14 has a plurality of slots, and the component feeder 12 can be detachably installed in each slot. The feeder holding section 14 may be fixed to the component mounter 10 or may be detachable from the component mounter 10 . The mounting head 16 is fixed to the moving base 17 . The mounting head 16 detachably holds one or more nozzles 6 , picks up the component 4 supplied by the component feeder 12 using the nozzle 6 , and mounts the component 4 on the substrate 2 . At this time, the head moving device 18 moves the moving base 17 in the X direction and the Y direction with respect to the component feeder 12 and the substrate 2 . As a result, the mounting head 16 moves together with the moving base 17 , picks up the component 4 from a specific component feeder 12 among the plurality of component feeders 12 , and mounts the component 4 at a predetermined position on the board 2 . The board conveyor 20 carries in, supports and carries out the board 2 .

The imaging device 19 is attached to the moving base 17 . Therefore, when the mounting head 16 moves, the imaging device 19 also moves together. The imaging device 19 includes a support portion 19a and a camera 19b. The camera 19b is arranged on the side of the mounting head 16, and arranged so that the imaging optical axis faces downward. The camera 19b images the component 4 supplied from the component feeder 12 . That is, the camera 19b captures an image of the upper surface of the component 4 before being mounted (before being sucked by the nozzle 6). Image data of the component 4 acquired by the camera 19 b is transmitted to the control device 30 .

The interface device 24 is a device that provides various types of information about the mounter 10 to the operator, and is a device that receives instructions and information from the operator. In this embodiment, the interface device 24 is a touch panel. However, the interface device 24 may be configured by, for example, a display or the like, a keyboard, a mouse, or the like.

The control device 30 is configured to be communicable with the management device 50 . As shown in FIG. 3, the control device 30 is composed of a computer including a memory 32 and a CPU . A component mounting section 22 and an interface device 24 are communicably connected to the control device 30 . The control device 30 mounts the component 4 on the board 2 by controlling the operation of each part of the component mounting section 22 according to the instructions from the management device 50 .

The memory 32 stores an original template 40 (described later) transmitted from the management device 50 . The CPU 34 includes a matching rate detection section 42 and a new template generation section 44 . The matching rate detection unit 42 detects the matching rate between the image data of the component 4 acquired by the imaging device 19 and the original template 40 . The new template generation unit 44 generates a new template for the part 4 based on the image data of the part 4 when the matching rate detected by the matching rate detection unit 42 is lower than a predetermined threshold. Specific processing executed by the CPU 34 (matching rate detection unit 42 and new template generation unit 44) will be described later.

As shown in FIG. 3, the management device 50 has an interface device 52 and a control device 60 . The interface device 52 is a device that provides various types of information about the component mounting line 100 to the operator, and is a device that receives instructions and information from the operator. In this embodiment, the interface device 52 is a touch panel. However, the interface device 52 may be configured by, for example, a display or the like, a keyboard, a mouse, or the like.

The control device 60 is composed of a computer including a memory 62 and a CPU 64 . A production program 66 is stored in the memory 62 . The production program 66 describes the operation of the entire component mounting line 100, and the CPU 64 instructs each device incorporated in the component mounting line 100 according to the production program 66 to perform the operation to be performed by each device. do. A plurality of original templates 40 are stored in advance in the memory 62 . The original template 40 is image data of the upper surface of the component 4, and image data used to check whether the component 4 mounted on the board 2 produced by executing the production program 66 is a correct component. (so-called master data).

FIG. 4 shows the original template 40 displayed on the interface device 52 . As shown in FIG. 4, the original template 40 has a caliper 70, a plurality of seek lines 72, and coordinate axes 74 set therein. A caliper 70 indicates an existence area where the part 4 exists in the original template 40 . The caliper 70 can be set by the operator for the image data of the part 4 when creating the original template 40, for example. Each seek line 72 is arranged at a location where optical characteristics (for example, brightness) change greatly within the range of the set caliper 70 . In the example shown in FIG. 4, within the range of the caliper 70, the difference in optical properties between the part 4a of the part 4 (the part indicated by the dotted hatching) and the background of the part 4 is relatively small. On the other hand, the difference in optical properties between the portion 4b of the component 4 and the background and portion 4a is relatively large. Therefore, each seek line 72 is arranged at a position including the boundary between the portion 4a and the portion 4b and the boundary between the portion 4b and the background, where the optical characteristics change greatly. The memory 62 stores the coordinates of the point where the optical characteristic changes on the seek line 72 as the edge position of the component 4 (coordinates of the outer shape of the component 4). The coordinate axes 74 are set so that the center of the part 4 is the reference position based on each edge position detected by the plurality of seek lines 72 .

Based on the production program 66 , the management device 50 sends the original template 40 corresponding to the component 4 to be mounted by each component mounter 10 to the controller 30 of the component mounter 10 . Send to

In the process of mounting the component 4 , the component 4 supplied from the component feeder 12 is mounted on the board 2 in each component mounter 10 . At this time, in order to determine whether or not the target component 4 accommodated in the component feeder 12 is a correct component, the image data of the target component 4 accommodated in the component feeder 12 is acquired and compared with the original template 40. . In general, if the target part 4 and the part indicated by the original template 40 are of the same type, they are successfully matched. However, for example, the outline of the target component 4 and the relative position of the character string printed on the target component 4 (“2RO” in the example shown in FIG. 4) may differ depending on the lot. In this case, the CPU 34 cannot recognize the target part 4 because the collation fails even though both parts are of the same type. In the mounter 10 of this embodiment, when such a situation occurs, the control device 30 generates a new template for the target component 4 (hereinafter referred to as a new template). Processing executed by the CPU 34 of the control device 30 when starting to mount the target component 4 on the component mounting line 100 will be described below with reference to FIGS. 5 and 6 . The processing in FIG. 5 is started, for example, when an operator inputs an instruction to start mounting. As described above, when the production program 66 is input to the memory of the management device 50, the management device 50 transmits the corresponding original template 40 to each mounter 10 based on the production program 66. . Therefore, the original template 40 is stored in the memory 32 of the control device 30 when the process of FIG. 5 is started.

In S<b>10 , the CPU 34 images the target component 4 using the imaging device 19 . As described above, before starting the mounting process, it is necessary to determine whether the target component 4 is a correct component. Therefore, in S10, the upper surface of the target component 4 supplied from the component feeder 12 (before it is sucked by the nozzle 6) is imaged.

In S<b>12 , the CPU 34 compares the image data of the upper surface of the target part 4 acquired by the imaging device 19 with the original template 40 . This collation is performed, for example, by superimposing the original template 40 on the image data of the target part 4 and moving and rotating the original template 40 relative to the image data. The CPU 34 moves and rotates the original template 40 relative to the image data so that the match rate between the image data and the plurality of seeklines 72 of the original template 40 increases. After specifying the maximum value of the matching rate, the CPU 34 proceeds to S14.

In S14, the CPU 34 determines whether or not the match rate specified in S12 is equal to or greater than a predetermined threshold. The CPU 34 determines that the collation is successful when the identified match rate is equal to or higher than the threshold. That is, the CPU 34 determines that the target part 4 is a correct part when the matching rate is equal to or higher than the threshold. The predetermined threshold is not particularly limited, but can be, for example, 50% to 100%. The predetermined threshold can be appropriately set by the operator. If the matching rate is equal to or greater than the predetermined threshold (YES in S14), the CPU 34 starts mounting processing based on the production program 66 in S16.

On the other hand, when the matching rate is less than the predetermined threshold (NO in S14), the CPU 34 determines that collation has failed. That is, the CPU 34 determines that the target component 4 cannot be recognized, and proceeds to S18 without starting the mounting process. Note that the processes of S12 and S14 (matching process and comparison process between matching rate and threshold value) may be performed at the same time. In other words, the CPU 34 may determine YES in S14 when it is determined that the match rate is equal to or higher than a predetermined threshold value in the process of increasing the match rate. That is, if the threshold is equal to or greater than a predetermined value, there is no need to specify the maximum match rate.

In S18, the CPU 34 executes new template generation processing. The new template generation process is performed on the image data of the target part 4 acquired in S10. S18 is performed by executing a subroutine shown in FIG.

In S40 of FIG. 6, the CPU 34 sets the caliper 80 in the image data of the target part 4. As shown in FIG. As shown in FIG. 7, the CPU 34 extracts the outline of the target part 4, and sets the caliper 80 in a range including a region outside the extracted outline by a predetermined distance. The outer shape of the target part 4 is obtained, for example, by extracting points where the optical characteristics change in the image data of the target part 4, and selecting the largest area from among a plurality of areas defined by the extracted point group. determined by specifying Note that in S40, a portion where the change in optical characteristics is relatively small (in the example shown in FIG. 7, the boundary between the portion 4a of the target component 4 and the background of the target component 4) is also extracted.

In S<b>42 , the CPU 34 arranges a plurality of seek lines 82 with respect to the image data of the target part 4 . As shown in FIG. 8, the CPU 34 places a seek line 82 at a point where the optical characteristics change significantly within the set caliper 80 area. Specifically, the CPU 34 arranges each seek line 82 at a position including the boundary between the portion 4a and the portion 4b and the boundary between the portion 4b and the background. Then, in S44, the CPU 34 detects the edge position of the target part 4 based on the placed seek line 82 as shown in FIG. 9, and sets the coordinate axis 84 with the center of the target part 4 as the reference position. As a result, a new template 90 shown in FIG. 9 is generated. After executing the process of S42, the CPU 34 terminates the new template generation process subroutine, and proceeds to S20 of FIG.

In S<b>20 , the CPU 34 displays an image on the interface device 24 indicating that a collation error has occurred in the target part 4 . Thereby, the worker can grasp that collation of the target part 4 has failed.

In S<b>22 , the CPU 34 determines whether or not an operation input for displaying the edit screen of the new template 90 has been received from the operator. The edit screen is a screen for the operator to confirm the generated new template 90 and a screen for the operator to correct the new template 90 . When determining that the operation input has been received (YES in S22), the CPU 34 displays the generated new template 90 on the interface device 24 in S24.

In S26, the CPU 34 determines whether or not an operation input for correcting the new template 90 has been received from the operator. As described above, the new template 90 is automatically generated based on the image data of the target part 4. FIG. Therefore, as a result of confirming the generated new template 90 by the worker, it may be determined that the new template 90 is not appropriate. In such cases, the operator can manually modify the new template 90 . For example, the operator can correct the position of the caliper 80 or the like by operating the interface device 24 (touch panel). If the CPU 34 accepts an operation input to modify the new template 90 (YES in S26), in S28, the new template 90 is modified according to the content of the operation input.

In S30, the CPU 34 determines whether or not an operation input reflecting the new template 90 has been received from the operator. When the operator determines that the displayed (or corrected) new template 90 is appropriate, the operator performs an operation input that reflects the new template 90 . When the CPU 34 receives an operation input to reflect the new template 90 (YES in S30), the CPU 34 overwrites the original template 40 stored in the memory 32 with the new template 90 and stores it in the memory 32 in S32. Thereafter, in S34, the CPU 34 starts mounting processing based on the production program 66. FIG.

As described above, in this embodiment, when the CPU 34 determines that the matching rate between the original template 40 and the image data of the target part 4 is lower than a predetermined threshold (NO in S14 of FIG. 5), A new template 90 in which a plurality of seeklines 82 are arranged in the image data of the target part 4 is displayed on the interface device 24 . That is, in this embodiment, when the target component 4 does not match the original template 40, the new template 90 is automatically displayed based on the target component 4 to be actually mounted. Therefore, in the mounter 10 of the present embodiment, since the operator does not need to correct the original template 40, work efficiency is improved.

In addition, in this embodiment, the CPU 34 sets the calipath 80 indicating the presence area where the target component 4 exists in the image data of the target component 4, so that the area where the target component 4 does not exist (for example, It is possible to prevent the seek line 82 from being placed on a specific portion of the background. Further, when setting the caliper 80, the CPU 34 extracts the outline of the target part 4 from the image data of the target part 4, so that the caliper 80 can be set with high accuracy.

Further, in this embodiment, since the match rate is calculated based on the optical characteristics of the seek line 72 of the original template 40, the comparison between the original template 40 and the image data of the target part 4 can be executed in a short time. can be done.

Also, in this embodiment, the new template 90 displayed on the interface device 24 can be modified by the operator. Therefore, if the automatically generated new template 90 is not appropriate, the operator can correct the new template 90 .

In the above-described embodiment, the caliper 80 is set by extracting the outline of the target part 4 in S40 of FIG. However, in the process of S<b>40 , the caliper 80 may be set by using the caliper 70 set in the original template 40 . With this configuration, the process of extracting the outline of the target part 4 can be omitted, so the efficiency of the process of generating the new template 90 can be improved. Note that the target part 4 is generally of the same type as the part indicated by the original template 40, and the existence area of the part greatly changes between the image data of the original template 40 and the target part 4. never.

(correspondence relationship)
The component mounter 10 is an example of an "image processing apparatus". The memory 32 is an example of a "storage unit". The interface device 24 is an example of the 'display section' and the 'operation section'. The CPU 34 is an example of a "controller".

Although specific examples of the technology disclosed in this specification have been described above in detail, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above. Modifications of the above embodiment are listed below.

Although the original template 40 is stored in the memory 62 of the management device 50 in the above embodiment, it may be stored in advance in the memory 32 of the control device 30 .

Further, in the above-described embodiment, the component mounter 10 functions as an image processing device, but the component mounter 10 does not have to have the function of an image processing device. That is, each component (imaging unit, storage unit, display unit) functioning as an image processing device may be mounted in a device separate from the component mounter 10 . Also, although the controller 30 of the mounter 10 functions as a controller (matching rate detector 42 and new template generator 44), a device separate from the mounter 10 may perform the functions of the controller. good.

In the above-described embodiment, the edit screen for the new template 90 was displayed in response to the operator's operation input (YES in S22). It may be automatically displayed on the interface device 24 . That is, the processing of S20 and S22 in FIG. 5 can be omitted.

Further, in the above-described embodiment, an operation input for correcting the new template 90 is accepted. That is, S26 and S28 in FIG. 5 can be omitted.

Moreover, in the embodiment described above, the CPU 34 may verify the generated new template 90 . This verification can be performed, for example, in the reverse order of the process of S12 in FIG. That is, the verification may be performed by superimposing the image data of the target part 4 on the generated new template 90 and moving and rotating the image data relative to the new template 90 . The CPU 34 moves and rotates the image data relative to the new template 90 so that the match rate between the plurality of seeklines 82 of the new template 90 and the image data increases, and when the match rate is equal to or higher than a predetermined value. It may be judged that the verification is successful depending on the result.

The technical elements described in this specification or in the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims as of the filing. In addition, the techniques exemplified in this specification or drawings achieve multiple purposes at the same time, and achieving one of them has technical utility in itself.

4: Target Component 10: Component Mounting Machine 12: Component Feeder 14: Feeder Holding Unit 16: Mounting Head 19: Imaging Device 20: Board Conveyor 22: Component Mounting Unit 24: Interface Device 30: Control Device 40: Original Template 50: Management Device 60: Control Device 90: New Template

Claims (8)

An image processing device for processing image data of parts,
an imaging unit that images a target component;
a storage unit that stores an original template generated in advance and indicating a plurality of characteristic portions of the target part;
When the matching rate between the image data of the target part acquired by the imaging unit and the plurality of characteristic parts of the original template stored in the storage unit is lower than a predetermined threshold, the imaging unit a display unit that displays a new template obtained by extracting a plurality of characteristic portions of the target part from the acquired image data;
An image processing device comprising: further comprising a control unit that detects a matching rate between the image data of the target part acquired by the imaging unit and the plurality of characteristic portions of the original template stored in the storage unit;
2. The control unit generates a new template by extracting a plurality of characteristic parts of the target part from the image data acquired by the imaging unit when the matching rate is lower than the predetermined threshold. The image processing device according to . The control unit
setting an existing region in which the target part exists for the image data acquired by the imaging unit;
3. The image processing apparatus according to claim 2, wherein said plurality of characteristic portions of said target part are extracted within said set existing region. 4. The image processing apparatus according to claim 3, wherein said control section extracts an outer shape of said target component from said image data acquired by said imaging section, and sets said existing region based on said extracted outer shape. 4. The image processing apparatus according to claim 3, wherein said control section sets said existing region for said image data acquired by said imaging section based on the outer shape of said target component included in said original template. The image processing apparatus according to any one of claims 1 to 5, wherein the plurality of characteristic portions include a plurality of seek lines that intersect the outer shape of the target part. The image processing apparatus according to any one of claims 1 to 6, further comprising an operation unit capable of accepting an operation from an operator to modify said new template displayed on said display unit. An image processing method for processing image data of a part,
an imaging step of imaging the target component;
A matching rate between the image data of the target part acquired by the imaging step and the plurality of feature parts of the original template prepared in advance and indicating the plurality of feature parts of the target part is higher than a predetermined threshold. a display step of displaying a new template obtained by extracting a plurality of characteristic portions of the target part from the acquired image data when the
An image processing method comprising:

Images