JPH10222670A - Method for processing picture and method for controlling work transfer device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely recognize the picture of a subject. SOLUTION: When the three pictures m1, m2 and m3 are detected in a last time proceeding picture processing and the three pictures n1, n2 and n3 are detected in this time picture processing, the feature quantities of the pictures m1-m3 and n1-n3 are extracted and the feature quantities of the last time pictures m1-m3 are compared with those of this time pictures n1-n3 concerning whole combinations. When the last time pictures and this time pictures satisfy a prescribed condition, they are synthesized so as to obtain the pictures m1, n2, n3 and m3 for the portion of four cylinders.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device for picking up a work on a pallet.
The present invention relates to an image processing method for recognizing a work position by performing binarization processing at a predetermined threshold and a control method for a work transfer device.

[0002]

2. Description of the Related Art Conventionally, when recognizing an image of a work position, the image is recognized by calculating a feature amount such as an area and a length from a picked-up work.

[0003] For example, Japanese Patent Application Laid-Open No. 1-284799 discloses that when a shape of a work is imaged and binarized with a predetermined threshold to recognize an image, the binarization threshold is changed for each region on the screen. An image processing method is disclosed. This conventional technique can deal only with a case where an area for which the binarization threshold should be changed is known in advance.

[0004]

However, in the case of performing image processing while keeping the binarization threshold constant, for example, if a plurality of works are continuously arranged on a pallet, the image processing is performed for each work. There is a possibility that the image recognition of each work is erroneously performed because the optimum binarization threshold value is different.

Further, as shown in FIG. 16, when image processing is performed on the cylinder bore 3a of an automobile, the shaded portion h becomes a bright portion and the work is divided by adjusting the brightness of sunlight or illumination or reflecting a pallet inside the bore. There is a risk of being recognized.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to recognize an image of a subject regardless of changes in the external environment such as adjustment of the brightness of sunlight or illumination and reflection of a pallet in a bore. Is to provide an image processing method that can reliably perform the image processing.

[0007]

SUMMARY OF THE INVENTION To solve the above problems,
In order to achieve the object, an image processing method according to the first invention has the following features. That is, in an image processing method in which a subject is imaged, the image is binarized at a predetermined threshold, and the image is recognized, the first feature data of the subject is converted from the first image data binarized at the first threshold. Extracting, extracting second feature data of the subject from the second image data binarized by a second threshold, synthesizing the first feature data and the second feature data, Extract the image data of the subject.

[0008] Preferably, the first and second feature data are extracted based on a predetermined reference value after changing the predetermined threshold value.

Preferably, the subject is a cylinder bore of an automobile engine.

Preferably, binarized noise is removed from the image data of the subject, and edge portions of the image data are made continuous.

Preferably, when the binarization noise is removed, the image data of the subject is grouped based on a predetermined reference value.

Preferably, the grouped image data is returned to the original scale after the edge portion is continued after being enlarged.

Preferably, the image data is subjected to a matching process based on a template of the subject.

Preferably, the subject is a cylinder bore of an automobile engine.

[0015] Preferably, the predetermined reference value is numerical data corresponding to original characteristic data of a subject.

[0016] Preferably, the first and second feature data are image-recognizable shape portions of a subject.

The image processing method according to the second invention has the following features. That is, in an image processing method in which a subject is imaged, the image is binarized at a predetermined threshold, and the image is recognized, the first feature data of the subject is converted from the first image data binarized at the first threshold. The second feature data of the subject is extracted from the second image data that has been extracted and binarized with a second threshold value, and the first feature data and the second feature data are extracted.
To extract image data of the subject, detect a relative positional relationship of a predetermined portion of the subject from the image data of the subject, and position the subject based on the positional relationship. Detect posture.

Further, the control method of the work transfer device according to the present invention has the following features. That is, a method of controlling a work transfer device for transferring a plurality of works on a pallet to a predetermined position by a work holding member, wherein the work to be transferred is imaged at a predetermined threshold, and the image is taken at a first threshold. Extracting first feature data of the work from the first image data that has been binarized, and extracting second feature data of the work from the second image data that has been binarized with a second threshold; Synthesizing the first feature data and the second feature data, extracting image data of the work, detecting a relative positional relationship of a predetermined portion of the work from the image data of the work,
The position and orientation of the work are detected based on the positional relationship, and the gripping operation of the work holding member is controlled based on the position and orientation of the work.

Preferably, the work gripping member has a positional shift absorbing means for gripping the work while absorbing a shift amount of the position and orientation of the work.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[Overall Configuration of Work Transfer Device] As shown in FIG. 1, the work transfer device WA automatically transfers the work 3 placed on the upper surface of the pace pallet 1 or the upper surface of the intermediate pallet 2 to the conveyor 4. And the empty intermediate pallet 2 is automatically moved onto the carriage 5.

A work 3 (FIG. 1 illustrates a cylinder block of an automobile engine, for example) uses one base pallet 1 and a plurality of intermediate pallets 2 and a work transfer device by a forklift (not shown) or the like. It is designed to be carried into a predetermined position in the WA in the directions of arrows A1 and A2. Here, each work 3 is loaded on both pallets 1 and 2 in the following state. That is, a plurality of (for example, eight) works 3 are placed on the base pallet 1, one intermediate pallet 2 is placed on these works 3, and a plurality of works 3 are placed on the intermediate pallet 2. Has been posted. A plurality of the work pieces 3 placed on the intermediate pallet 2 are further stacked.

Here, the intermediate pallets 2 from which all the works 3 have been paid out and become empty are automatically transferred to the carriage 5 automatically as will be described later.
Is transported by moving the cart 5 in the direction of arrow B when the predetermined number of sheets has been accumulated. In addition, unloading conveyor 4
Is transported in the direction shown by arrow C.

The work transfer device WA is supported by a plurality of columns 6 extending in the vertical direction.
A first and second horizontal frames 7a and 7b extending in the width direction of A (hereinafter, this direction is referred to as a lateral direction),
The longitudinal direction of the work transfer device WA supported by the second horizontal frames 7a and 7b (hereinafter, this direction is referred to as the vertical direction).
And a second vertical frame 8b supported only by the first and second horizontal frames 7a and 7b and extending in the vertical direction.

First and second rails 9a and 9b are provided on the upper end surfaces of the first and second frames 8a and 8b, respectively, and extend in the vertical direction. A movable frame 10 extending in the horizontal direction is provided across both the vertical frames 8a and 8b, and the movable frame 10 is engaged with the first and second rails 9a and 9b, and the first and second rails 9a are provided. , 9b in the vertical direction.

A movable frame driving mechanism 11 for driving the movable frame 10 is provided. The movable frame driving mechanism 11 freely reciprocates the movable frame 10 in the vertical direction via the first and second chains 12a and 12b. It can be moved.

As shown in FIG. 2, a pallet arm 13 extending vertically is attached to the side of the movable frame 10 on the first horizontal frame 7a side.
A pallet clamp member 19 capable of clamping (gripping) the intermediate pallet 2 is attached to a lower end of the pallet 3. The pallet arm 13 is provided with a pallet arm elevating mechanism 17 that can freely reciprocate the pallet arm 13 in the vertical direction.

A work arm 14 extending in the vertical direction is attached to the side of the second horizontal frame 7b of the movable frame 10 via a work arm connecting portion 14a. A work clamp member 20 capable of clamping (gripping) is provided. An image processing camera 22, which will be described later, is attached to a vertically intermediate portion of the work arm 14.

Here, the work arm 14 can be moved laterally along the movable frame 10. The movable frame 10 is provided with a work arm drive mechanism 15 for driving the work arm 14, and the work arm drive mechanism 15 can freely reciprocate the work arm 14 in the lateral direction via a drive chain 16. It has become. The work arm 14 is provided with a work arm elevating mechanism 18 that can reciprocate the work arm 14 freely in the up and down direction.

Further, at the upper end of the work arm 14,
A turning drive mechanism 21 that can turn the work arm 14 around its axis is provided.

As shown in FIGS. 3 and 4A and 4B, in the work clamp member 20, a connecting member 31 connected and fixed to the lower end of the work arm 14 and a main body 32 are provided. It is connected via a ball joint 33 and a coil spring 34. A proximity switch 3 that detects an approach to the workpiece 3 (a cylinder block is illustrated in FIGS. 3 and 4) is provided in the main body 32.
5 is provided, and a clamp arm 38 driven by an air cylinder 37 is provided. Here, the contact member 36 comes into contact with the upper surface of the work 3 when the work clamp member 20 is clamping the work 3 to stabilize its position. Further, the clamp arm 38 engages with a predetermined portion of the work 3 so that the work 3 can be reliably clamped. In FIG. 3, the left half shows a state where the work 3 is not clamped (unclamping), and the right half shows a state where the work 3 is clamped.

As shown in an enlarged view in FIG. 5, the ball joint 33 includes a first link ball 33b swingably supported by a first ball receiving portion 33a fixed to the connecting member 31 and a main body portion 32. Fixed second ball receiving portion 33
The second link ball 33 swingably supported by c.
d, and a cylindrical member 33e that grips and connects the first and second shaft portions 33f and 33g extending from the first and second link balls 33b and 33d, respectively. 31 (work arm 14) and the main body 32 can be relatively displaced (shifted) from each other in the horizontal direction.

As described above, the connecting member 31 and the main body 32 are connected via the ball joint 33 and the coil spring 34 so as to be relatively displaceable.
Is basically suspended by the work arm 14, and the material body 32 can be displaced to some extent in the horizontal direction. FIG. 4B shows a state in which the main body 32 is relatively displaced (shifted) in the horizontal direction.

For this reason, even when the position of the work 3 recognized by the image processing described later is slightly shifted, when the work 3 is clamped, the body 32 absorbs the shift amount. Can reliably clamp the work 3.

It should be noted that the work clamp member 20 is
Is moved in a clamped state, the work arm 14 and the main body 32 are inserted by inserting the insertion member 39 attached to the connecting member 31 (work arm 14) into the receiving portion 40 attached to the main body 32. And can be combined rigidly. Therefore, the main body 32 does not swing in the horizontal direction during such movement.

Here, the work transfer device WA has a control device for detecting the position and orientation of the work 3 to be transferred and controlling the gripping and transfer operations of the work 3 and the intermediate pallet 2 by each drive mechanism. The control device will be described below.

As shown in FIG. 6, the control device S of the work transfer device WA includes a first control unit 51 including an upper link unit, a basic unit, an A / D conversion unit, and a plurality of I / O units. And a second control unit 52 including a sequencer, a positioning unit, and the like.

Here, the upper link unit of the first control unit 51 links the first control unit 51 and the second control unit 52, and causes the two units 51 and 52 to communicate with each other. Various output signals of the touch panel 41 operated by the operator are input to the upper link unit.

The basic unit of the first control unit 51 is an image processing camera 22 (hereinafter referred to as a camera 22).
The image captured by the camera amplifier 4
7 and the result of processing by the image processing device 48 are accepted.
Here, as described later, the image processing device 48 performs a binarization process on each pixel of the multi-value image of the work 3 captured by the camera 22 based on a predetermined binarization threshold.
The position and orientation of the work 3 are detected (calculated) from the binary image obtained by the binarization process. The processing result of the image processing device 48 is displayed on the CRT 49.

The distance between the pallet clamp member 19 and the intermediate pallet 2 below the pallet clamp member 19 or the work clamp member 20 and the lower portion thereof below the pallet clamp member 19 are detected by the A / D conversion unit of the first control unit 51. Is input via the sensor amplifier 50. The I / O unit of the first control unit 51 includes a valve 43, a limit switch 44, a push button 4
5, lamp 46, etc. are connected.

The sequencer of the second control unit 52 mainly includes a movable frame drive mechanism 11, a work arm drive mechanism 15, a pallet arm elevating mechanism 17, a work arm elevating mechanism 18, a turning drive mechanism 21, a pallet clamp member 19, a work clamp The operation order of the members 20 and the like is controlled.

The positioning unit of the second control unit 52 is connected to the first to fifth amplifiers 53 to 57 through the first to fifth amplifiers 53 to 57.
The seventh motors 58 to 62 are controlled to position the pallet arm 13 and the work arm 14. Here, the first motor 58 is a drive source of the movable frame drive mechanism 11,
The second motor 59 is a drive source of the work arm drive mechanism 15, the third motor 60 is a drive source of the work arm elevating mechanism 18, the fourth motor 61 is a drive source of the turning drive mechanism 21, and the fifth motor 62 is a pallet arm lifting mechanism 1
7 is a driving source. That is, the first motor 58 moves or positions the pallet arm 13 and the work arm 14 in the vertical direction, and the second motor 59 operates the work arm 14
Of the third motor 6
0 moves or positions the work arm 14 in the vertical direction, the fourth motor 61 moves or positions the work arm 14 in the rotational direction about its axis, and the fifth motor 62 moves the pallet arm 13 in the vertical direction. Or perform positioning. The pallet arm 13 (pallet clamp member 19) is controlled by controlling the first to fifth motors 58 to 62 by the positioning unit.
Further, the work arm 14 (work clamp member 20) can be moved to an arbitrary position or positioned.

Hereinafter, a method of controlling the work transfer device WA by the control device S will be described with reference to FIGS. 1 to 6 as appropriate according to the flowchart shown in FIG. FIG. 7 shows a main routine of such control.

As shown in FIG. 7, in the main routine, the next cycle operation is determined in steps S1 to S2, that is, the next cycle operation is transfer of the work 3 or transfer of the intermediate pallet 2. Is determined, and according to the determination result, steps S3 to S
One of a work transfer routine of No. 12 and a pallet transfer routine of Steps S13 to S20 is executed. This next cycle operation is determined by determining whether or not an instruction to execute the next cycle operation has been issued from the operator via the operation panel, or whether or not the camera 22 has run out of work on the intermediate pallet 2.

If it is determined that the next cycle operation is the transfer of the workpiece 3, first, in step S3, the first and second motors 58 and 59 are driven to move the movable frame 10 by a predetermined distance in the vertical direction. The work arm 14a (work arm 14) is moved in the horizontal direction by a predetermined distance, and the work arm 14 (camera 22) is arranged at the image reading position in plan view.

In step S4, the third motor 60 is driven, the work arm 14 is lowered by a predetermined distance, and the work arm 14 (camera 22) is located at the image reading position (height).

Subsequently, in step S5, an image processing routine (see FIGS. 8 to 10), which is a subroutine, is executed to calculate (detect) the position and orientation of the work 3 to be transferred. The work arm 14 (the work clamp member 2)
The position where 0) is to be arranged is calculated, and the corresponding movement amount of the work arm 14 (work clamp member 20), that is, the first to fourth motor 58 to 61 drive amount is calculated. The image processing routine will be described later.

In step S6, the first, second, and fourth motors 58, 5
The work arms 9 and 61 are driven, and the work arm 14 is arranged at a position where the work 3 to be transferred can be accurately clamped in plan view. Subsequently, in step S7, the third
The motor 60 is driven, and the work arm 14 (work clamp member 20) is lowered to a position (height) where the work 3 to be transferred can be accurately clamped.

In step S8, the work clamp member 2
0 air cylinder 37 is driven, and clamp arm 38
The workpiece 3 to be transferred is clamped (gripped). Subsequently, in step S9, the third motor 60 is driven, and the work arm 14 is raised to the traveling position.

In step S10, the first, second, and fourth motors 58, 59, and 61 are driven, and the work arm 14 that has clamped the work 3 moves the work 3 to a predetermined position on the conveyor 4 in plan view. It is moved to a position where it can be transferred. Next, in step S11, the third motor 60 is driven, and the work arm 14 is lowered to a position (height) at which the work 3 can be transferred onto the transport conveyor 4, and subsequently, in step S12, air is supplied. The cylinder 37 is driven, and the work 3
Is released (unclamped), and the work 3 is transferred onto the transport conveyor 4.

If it is determined in steps S1 and S2 that the next cycle operation is transfer of the intermediate pallet 2, the first motor 58 is driven in step S13, and the movable frame 10 is vertically moved. The pallet arm 13 is moved in the direction by a predetermined distance, and the pallet arm 13 is arranged at a position corresponding to the intermediate pallet 2 to be transferred in plan view.

In step S14, the fifth motor 62 is driven, the pallet arm 13 is lowered by a predetermined distance, and the pallet arm 13 (the pallet clamp member 1) is moved.
9) is arranged at a position (height) where the intermediate pallet 2 to be transferred can be clamped accurately.

In step S15, the intermediate pallet 2 to be transferred is clamped (gripped) by the pallet clamp member 19. Subsequently, in step S16, the fifth motor 62 is driven, and the pallet arm 13 is raised to the traveling position.

In step S17, the distance data of the lowering position of the intermediate pallet 2 is fetched. That is, the number of intermediate pallets 2 stacked on the cart 5 is calculated, and the height of the intermediate pallet layer is calculated from this number,
The intermediate pallet 2 currently clamped based on this
Is calculated.

In step S18, the first motor 58 is driven, and the pallet arm 13 clamping the intermediate pallet 2 is moved to a position where the intermediate pallet 2 can be transferred to a predetermined position on the carriage 5 in plan view. Moved. Next, in step S19, the fifth motor 62 is driven, and the pallet arm 13 is lowered to near the upper end surface of the intermediate pallet layer on the carriage 5 based on the calculation result in step S17. Subsequently, in step S20, the clamp of the intermediate pallet 2 by the pallet clamp 19 is released (unclamped), and the intermediate pallet 2 is transferred onto the intermediate pallet layer on the carriage 5.

In this manner, both the work 3 and the intermediate pallet 2 are automatically transferred to predetermined positions.

[Image Processing Method] The image processing method will be described below.

The outline of this image processing method is as follows. That is, the image photographed by the camera 22 is binarized with a predetermined binarization threshold, and the feature amount (area, circular shape coefficient, etc.) of the binarized image data is extracted and stored.
Next, the same processing is executed by changing the binarization threshold to extract the current feature amount. Then, the position and orientation of the work 3 are detected by comparing the previous characteristic value with the current characteristic value and adopting a value closer to the reference value as the characteristic value. If the position and orientation of the work 3 cannot be detected by the image processing,
The value threshold is changed and the image processing is performed again to detect the position and orientation of the work 3.

Next, with reference to FIGS. 12, 14, and 15, a case in which the workpiece imaged by the camera 22 is a cylinder bore of a cylinder block of four cylinders will be schematically described.

First, the cylinder bore is photographed by the camera 22 at a predetermined threshold for binarization and binarized. At this time, as shown in FIG. 14, if the area of the binarized bore image Bi is about 40% or more of the area of the original bore Bref, it is recognized as a bore image. Alternatively, the bore image Bi is recognized as a bore image if the length lx and the length ly in the y direction of the bore image Bi are about 40% or more of the length lx in the x direction and the length ly of the original bore Bref in the y direction. Is done.

Next, as shown in FIG. 12, three bore images m1, m2, and m3 are detected in the previous image processing in the cylinder bores of the four cylinders by the above-described bore recognition, and in the current image processing. Assuming that three bore images n1, n2, and n3 are detected, the feature amounts of the images m1 to m3 and n1 to n3 are extracted, and the feature amounts of the previous images m1 to m3 and the current image n1 to n3 are extracted.
The feature amount of n3 is compared for all combinations, and when the previous image and the current image satisfy a predetermined condition, the previous image and the current image are combined to generate images m1 and n for four cylinders.
2, n3 and m3 are obtained. In the image synthesis, for example, a combination in which the center of gravity interval between the previous image and the current image is smaller than a predetermined reference value is extracted, and further, the length of the previous image and the current image in the x direction and the length in the y direction are extracted. By taking the difference from the predetermined reference value and taking the value of the smaller difference as the feature amount, if the feature amount of the previous image is adopted, the image data is stored as it is (image m1), and the current image When the feature amount is adopted, the previous image data is updated to the current image data (image n3), and images that cannot be combined are added as new image data (images n2 and m3).

If the bore images B1 to B4 are captured by the image processing shown in FIG. 14, the distances f1, f between the centers of gravity of the bore images B1 to B4 are obtained as shown in FIG.
2 and f3 are equal to a predetermined reference value (for example, the original bore Bre
straight lines g1-g2, g2- connecting the centers of gravity of the respective bore images B1 to B4.
If the angles formed by g3 and g3-g4 are within a predetermined reference angle, the bore images B1 to B4 are recognized as four bores of the workpiece alone.

Finally, the center position of the work 3 is calculated from the center of gravity data g1 to g4 of each of the bore images B1 to B4, and the work clamp member 20 is applied to the bores B1 and B4 on both sides, and the center two bores B2 and B3 are applied. The workpiece is gripped by inserting the contact member 36. Here, the center of gravity data g of each of the bore images B1 to B4
If 1 to g4 deviate from the original position of the center of gravity, the center position of the work will also deviate from the original position. However, when the work 3 is clamped, the main body 32 absorbs the deviation amount. Numeral 38 can reliably clamp the work 3.

Specifically, as shown in FIG. 8, first, at step S30, a predetermined binarization threshold is set. Next, in step S31, the binarization threshold set in the previous image processing is changed. In step S32, the image captured by the camera 22 is subjected to a binarization process at a predetermined threshold, and in step S34, the binarized image is subjected to binarization image removal to perform a labeling process to perform image data processing. Generate In step S36, a predetermined feature amount is extracted from the image data. Here, the binarization processing is performed by the camera 2
The multi-valued image captured by the method 2 is performed by an ordinary method such that pixels brighter than a predetermined threshold are set to white and pixels darker than the threshold are set to black. Note that such binarization processing or other image processing may be performed by selecting light of a specific wavelength. Labeling is performed by first attaching a label (for example, A) to one black pixel. If a pixel adjacent to this pixel is black, the pixel is also labeled A, and the label A is already attached. When pixels adjacent to the assigned pixel are black, the method of attaching a label A to all these pixels is repeated, and when the labeling converges, a group of black pixels assigned the label A is set to 1 It is performed by an ordinary method such as recognizing the image as one shape part image. The shape portion is a component of the work 3 and has a predetermined simple shape (for example, a circle) and can be recognized as a group of isolated black pixels by binarization processing. . For example, when the work 3 is a cylinder block, the cylinder pore is a typical shape portion.

As shown in FIG. 11, for example, the following values are applied to the feature values extracted in step S36.

(1) Area S of each shape part (2) Perimeter R of each shape part (3) Circular shape coefficient (ratio of area to circumference) S / R of each shape part (4) Two adjacent two A distance d between the centers of gravity (centers) g of the shape parts d (5) a length lx in the x direction of each shape part and a length ly in the y direction (6) a straight line connecting the centers of gravity g of two adjacent shape parts; Angle between a straight line connecting the centers of gravity of the other two adjacent shape portions. Next, in step S38, the feature amount of the previous shape portion mi and the feature amount of the shape portion nj extracted this time are compared, and each shape is compared. (Step S3)
8 will be described later with reference to FIG. 9). After that, in step S40, a grouping process of each shape part is performed. In this grouping process, as shown in FIG. 13, when one shape part mi has two image data i1 and i2, these two parts are determined as image data to be grouped into one.

In step S42, a combining process for combining and integrating the image data grouped in step S40 is executed. In this combination processing, after enlarging the grouped image data to fill the gap between the two image data, the enlargement / reduction processing for returning to the original scale, edge processing for combining the edge portions of the two image data, For example, a template matching process or the like for matching in correspondence with a predetermined work template is employed.

In step S44, the same processing as in steps S30 to S34 is performed, and a predetermined feature amount is extracted from the image data on which the grouping processing and the combining processing have been performed. In step S46, each feature is detected by comparing the feature of the previous shape pi after the combining process with the feature of the shape qj extracted this time (detailed processing of step S46 will be described later with reference to FIG. 10). ). In step S48,
From the feature amount of the shape part detected in step S46, FIG.
As described in 5, the distance between the centers of gravity of the bore image data and the angle formed by the straight line connecting the centers of gravity of the bore image data are detected. In step S50, it is determined whether or not the image data of the work 3 detected in step S48 is within a predetermined reference value (for example, a value of about 80% of the original distance between the centers of gravity of the bores Bref shown in FIG. 15). Is determined. If it is within the predetermined reference value in step S50 (Y in step S50)
es) If the bore image data is recognized as four bores of the workpiece alone, the process proceeds to step S6 in FIG. 7, and if the bore image data is not within the predetermined reference value (No in step S50), the binarization process is not properly performed. Therefore, the process proceeds to step S31, the binarization threshold is changed, and step S3 is performed again.
The processing is executed from step 2.

<Shape Part Candidate Extraction Processing> Next, the shape part candidate extraction processing described in step S38 of FIG. 8 will be described in detail.

In this shape part candidate extraction processing, a shape part is roughly detected, and a candidate having a high possibility of being a shape part is extracted.

As shown in FIG. 9, in step S60,
The distance d between the centers of gravity of the respective shape portions is calculated from the feature amount of the previous shape portion mi and the feature amount of the current shape portion nj. Center of gravity distance d
Is calculated from Equation 1 below. Here, “↑” represents a multiplier, i represents the cumulative detection number up to the previous time, and j represents the current detection number.

D = (gxn [j] −gxm [i]) ↑ 2 + (gyn [j] −gym [i]) ↑ 2 (1) where gxnj: x-coordinate data of the centroid extracted this time gynj: this time The extracted y-coordinate data of the center of gravity gxmi: the x-coordinate data of the previously extracted center of gravity gymi: the y-coordinate data of the previously extracted center of gravity Next, in step S62, the center-of-gravity interval d is set to a predetermined value d0
It is determined whether or not it is smaller. Here, the predetermined value d0 is
For example, when the shape portion is a cylinder bore, the bore radius is used. If the center-of-gravity interval d is smaller than the predetermined value d0 in step S62 (Yes in step S62), step S64
If the center distance d is equal to or greater than the predetermined value d0 in step S62 (No in step S62),
Go to 80.

In step S64, the absolute value of the difference between the length lxmi in the x direction of each shape portion and a predetermined first reference value X1 from the feature value of the last shape portion mi, and the length lym in the y direction
sum lmi of the absolute value of the difference between i and a predetermined first reference value Y1
Is calculated. The sum lmi is calculated from Equation 2 below.

Lmi = | lxm [i] −X1 | + | lym [i] −Y1 | (2) where lxmi: the length of the previously extracted x direction lymi: the length of the previously extracted y direction Step S66 Then, the absolute value of the difference between the length lxnj in the x direction of each shape portion and the predetermined reference value X1 and the absolute value of the difference between the length lynj in the y direction and the predetermined reference value Y1 are calculated from the feature amount of the current shape portion nj. The sum lnj of the values is calculated. The sum Inj is calculated from the following Equation 3.

Lnj = | lxn [j] -X1 | + | lyn [j] -Y1 | (3) where lxnj: the length of the currently extracted x direction lynj: the length of the currently extracted y direction Step S68 Then, the sum lnj of the difference in the x and y directions this time
Is smaller than the previous sum lmi of the difference in the x and y directions. If the current value is smaller than the previous value in step S68 (Yes in step S68), step S70
If the current value is equal to or greater than the previous value in step S68 (No in step S68), the process proceeds to step S72 described below.
Proceed to.

In step S70, the feature quantity extracted last time is updated to the feature quantity extracted this time. Step S72
Then, the detection number i of the previously extracted shape portion is decremented by one, and the processing from step S64 is repeated until the detection number i becomes zero in step S74. That is, when the number of detections i becomes zero, step S is performed for all of the previously extracted shape parts (m1, m2,..., Mi).
This means that the processing from step 64 has been executed.

Next, in step S76, the detected number j of the shape part extracted this time is decremented by one, and the processing from step S64 is repeated until the detected number j becomes zero in step S78. That is, when the number of detections j becomes zero, the shape parts (n1, n2,.
The processing from step S64 has been executed for all of j). Then, when the number j of detections becomes zero in step S78 (Yes in step S78), the process proceeds to step S40 shown in FIG. 8, and each shape part is subjected to a grouping process.

In step S80, the feature quantity extracted this time is added as a feature quantity of a new shape portion, and step S8
After adding 1 to the previous detection number i in 2, the process proceeds to step S72.

<Shape Part Detection Processing> Next, the shape part candidate extraction processing described in step S38 of FIG. 8 will be described in detail.

In the shape part detection processing, the shape part image data used for detecting the position and orientation of the work is compared by comparing the shape part candidates roughly detected in the shape part candidate extraction processing with more accurate reference values. To detect.

As shown in FIG. 10, in step S84, the distance e between the centers of gravity of the respective shape portions is calculated from the feature amount of the previous shape portion pi and the feature amount of the current shape portion qj. The center-of-gravity interval e is calculated from Equation 1 above.

Subsequently, in step S86, the center of gravity e
Is smaller than a predetermined value e0. Here, the predetermined value e0 is a value smaller than the center-of-gravity interval d in step S62 (e0 <d0). When the center-of-gravity interval e is smaller than the predetermined value e0 in step S86 (Ye in step S86)
s), the process proceeds to step S88, and if the center-of-gravity interval e is equal to or larger than the predetermined value e0 in step S86 (N in step S86)
o), and proceed to step S104 described later.

In step S88, the absolute value of the difference between the length lxpi in the x direction of each shape portion and a predetermined second reference value X2 from the feature value of the previous shape portion pi, and the length lyp in the y direction
sum lpi of the absolute value of the difference between i and a predetermined second reference value Y2
Is calculated. The sum lpi is calculated from the above equation (2).

In step S90, the absolute value of the difference between the length lxqj in the x direction of each shape portion and the predetermined reference value X2, the length lyqj in the y direction and the predetermined reference value Y2 And the sum lqj of the absolute values of the differences from Sum l
qj is calculated from Equation 3 above. Here, a predetermined second
Is smaller than the first reference value X1 used in steps S64 and S66 (X2 <X
1) It is set so that the error between the previous feature and the current feature can be detected more accurately.

In step S92, it is determined whether or not the current sum lqj of the differences in the x and y directions is smaller than the previous sum lpi of the differences in the x and y directions. If the current value is smaller than the previous value in step S92 (Yes in step S92), the process proceeds to step S94. If the current value is equal to or more than the previous value in step S92 (No in step S92), the process proceeds to step S96 described later. move on.

In step S94, the previously extracted feature quantity is updated to the currently extracted feature quantity. Step S96
Then, the detection number i of the previously extracted shape portion is decremented by one, and the processing from step S88 is repeated until the detection number i becomes zero in step S98. That is, when the number of detections i becomes zero, step S is performed for all of the previously extracted shape parts (p1, p2,... Pi).
This means that the processing from step 88 has been executed.

Next, in step S100, the detected number j of the shape part extracted this time is decremented by one, and in step S102, until the detected number j becomes zero in step S8.
The processing from step 8 is repeated. That is, when the number of detections j becomes zero, the shape parts (q1, q2,...
The processing from step S88 is executed for all of qj). Then, when the number of detections j becomes zero in step S102 (Yes in step S102), the process proceeds to step S48 shown in FIG. 8, and the position and orientation of the workpiece are detected from the shape part candidates roughly detected in the shape part candidate extraction processing. The image data of the used shape part is detected.

In step S104, the feature quantity qj extracted this time is added as a feature quantity of a new shape portion. In step S82, 1 is added to the number of detections i in the previous time.
Proceed to 06.

According to this image processing, a shape portion is roughly detected from an image captured by the camera 22 based on a predetermined first reference value, and a candidate having a high possibility of being a shape portion is extracted. Further, based on a predetermined second reference value that is more accurate than the first reference value, the shape part used for detecting the position and orientation of the workpiece from the shape part candidate roughly detected in the shape part candidate extraction process. Since image data is detected, even when a plurality of works 3 are included, a shape part belonging to the work 3 to be transferred can be selected. The position of the center of gravity of the work 3, that is, the position and orientation of the work 3 can be accurately detected, and the work 3 can be accurately clamped.

Further, even when the external environment changes, such as adjustment of the brightness of sunlight or illumination or the reflection of a pallet in the bore, image recognition of the work can be executed accurately, and the position and orientation of the work 3 to be transferred can be accurately determined. Can be detected.

The present invention can be applied to a modification or change of the above embodiment without departing from the spirit thereof.

[0092]

As described above, according to the present invention, the first feature data of a subject is extracted from the first image data binarized by the first threshold, and the binary data is extracted by the second threshold. The second feature data of the subject is extracted from the second image data subjected to the conversion processing, and the first feature data and the second feature data are combined to extract the image data of the subject. The image recognition of the object can be reliably performed regardless of changes in the external environment such as the adjustment of the brightness of the image and the reflection of the palette inside the bore.

[0093]

[Brief description of the drawings]

FIG. 1 is an external perspective view of a work transfer device according to an embodiment of the present invention.

FIG. 2 is a view showing a pallet arm and a work arm of the work transfer device according to the embodiment of the present invention.

FIG. 3 is a front view of a work clamp member according to the embodiment of the present invention.

FIG. 4 is a side view of the work clamp member shown in FIG. 3;

FIG. 5 is a sectional view of a ball joint.

FIG. 6 is a system block diagram of a control device of the work transfer device.

FIG. 7 is a flowchart showing main control of the work transfer device.

FIG. 8 is a flowchart illustrating main control of image processing.

FIG. 9 is a flowchart illustrating subroutine control in image processing.

FIG. 10 is a flowchart illustrating subroutine control in image processing.

FIG. 11 is a diagram illustrating a feature amount of a shape portion that has been subjected to image processing.

FIG. 12 is a diagram illustrating an image processing method according to the embodiment.

FIG. 13 is a diagram illustrating an image processing method according to the embodiment.

FIG. 14 is a diagram illustrating an image processing method according to the embodiment.

FIG. 15 is a diagram illustrating an image processing method according to the embodiment.

FIG. 16 is a diagram illustrating a problem in image processing of the related art.

[Explanation of symbols]

 WA Work transfer device S Control device 2 Intermediate pallet 3 Work 20 Work clamp member 22 Image processing camera 48 Image processing device 51 First control unit 52 Second control unit

Claims (13)

[Claims] 1. An image processing method for capturing an image of a subject, binarizing the image with a predetermined threshold value, and recognizing the image, comprising the steps of: Extracting characteristic data, extracting second characteristic data of the subject from the second image data binarized by a second threshold, and synthesizing the first characteristic data and the second characteristic data And extracting image data of the subject. 2. The image processing method according to claim 1, wherein the first and second feature data are extracted based on a predetermined reference value after changing the predetermined threshold value. 3. The image processing method according to claim 1, wherein the subject is a cylinder bore of an automobile engine. 4. The image processing method according to claim 1, wherein binarized noise is removed from the image data of the subject, and edge portions of the image data are made continuous. 5. The image processing method according to claim 4, wherein the image data of the subject is grouped based on a predetermined reference value when the binarization noise is removed. 6. The image processing method according to claim 5, wherein the grouped image data is restored to its original scale after the edge portion is continued after being enlarged. 7. The image processing method according to claim 1, wherein the image data is subjected to a matching process based on a template of the subject. 8. The vehicle according to claim 5, wherein the subject is a cylinder bore of an automobile engine.
The image processing method according to any one of the above. 9. The image processing method according to claim 2, wherein the predetermined reference value is numerical data corresponding to original characteristic data of the subject. 10. The image processing method according to claim 2, wherein the first and second feature data are a shape portion of an image of a subject that can be recognized. 11. An image processing method for capturing an image of a subject, binarizing the image with a predetermined threshold, and recognizing the image, comprising: Extracting characteristic data, extracting second characteristic data of the subject from the second image data binarized by a second threshold, and synthesizing the first characteristic data and the second characteristic data Extracting image data of the subject, detecting a relative positional relationship of a predetermined portion of the subject from the image data of the subject, and detecting a position and orientation of the subject based on the positional relationship. Image processing method. 12. A method of controlling a work transfer device for transferring a plurality of works on a pallet to a predetermined position by a work gripping member, wherein the work to be transferred is imaged at a predetermined threshold value. Extracting the first feature data of the work from the first image data binarized by the threshold value, and the second feature data of the work from the second image data binarized by the second threshold value Is extracted, and the first feature data and the second feature data are combined to extract image data of the work. From the image data of the work, a relative positional relationship of a predetermined portion of the work is determined. Detecting the position and orientation of the work based on the positional relationship, and controlling a gripping operation of a workpiece gripping member based on the position and orientation of the work. 13. The work transfer device according to claim 12, wherein the work gripping member has a position shift absorbing unit that grips the work while absorbing a shift amount of the position and orientation of the work. Control method.