JP5769559B2 - Image processing apparatus, image processing program, robot apparatus, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing program, a robot apparatus, and an image processing method for detecting an edge by image processing.

  In the field of image processing, edge extraction is well known as a method for knowing an end face of an object from an image. As a method for extracting an edge (image processing method), a Sobel filter, a Laplacian filter, or the like is generally known. As this technique, an image processing method using the following edge extraction apparatus is known. Note that the edge means a portion where the luminance (pixel value) changes rapidly in the grayscale image.

  The edge extraction device (image processing device) includes an edge detection unit that calculates edge strength from an image and detects an edge, and a labeling processing unit that performs a labeling process on the edge detected by the edge detection unit and calculates an edge length And. Further, the edge extraction apparatus includes an edge enhancement processing unit that performs edge enhancement processing using a value in which the edge length obtained by the labeling processing unit is associated with the edge strength calculated by the edge detection unit. Then, the edge extraction device performs binarization processing on the value of the image emphasized by the edge enhancement processing unit using a threshold value (threshold value) that can be adjusted, and extracts a predetermined edge. (See Patent Document 1).

  That is, in the edge detection device described in Patent Document 1, when detecting an edge, the edge strength (steepness of shading change) and length are used as a reference, and an edge having a weak edge strength and a short edge is determined as a noise edge. To remove fine noise edges.

  Another image processing method using an edge extraction system is known. The edge extraction system includes an image input unit that inputs an image, an edge extraction unit that extracts an edge of an image from an input image output from the image input unit, and an edge extraction that captures an edge extraction image extracted by the edge extraction unit Image output means. The edge extraction means learns the edge extraction processing of the image using the data of the area cut out from the input image output from the image input means as the input data and the data indicating whether or not the specific pixel of the area is an edge. An edge extraction learning unit is provided. Furthermore, an edge extraction unit that extracts an edge of an image from an input image output from the image input unit based on a learning result learned by the edge extraction learning unit is provided (see Patent Document 2).

  That is, in the edge extraction system described in Patent Document 2, as shown in FIG. 15A, an area is set from a reference image 33 of a workpiece to be compared with an input image. Further, the data of the detection target region 34 including the detection target edge and the data of the non-detection target non-detection region 35 that does not include the detection target edge are stored in advance. The detection target area 34 is associated with a true detection target (for example, 1), and the non-detection target area 35 is associated with a detection target for a negative (for example, 0) to learn the detection area in advance.

  Then, as shown in FIG. 15B, the area is scanned with respect to the input image 36 of the work imaged separately from the reference image 33. From the learning result, the area 37 in the input image 36 is a non-detection area. Yes, it is determined that the region 38 in the input image 36 is a detection target region. That is, by learning the edge extraction process of the image using the data indicating whether or not the specific pixel of the area data cut out from the reference image 33 is an edge as shown in FIG. Detect edges.

JP 2009-282929 A Japanese Patent Laid-Open No. 5-46764

  The result of performing conventional general edge extraction on the image of FIG. 16A is shown below. That is, since only the general edge extraction performed on the image of FIG. 16A causes a lot of noise due to background and object scratches, uneven brightness of illumination, and the like as shown in FIG. This causes an error when performing position measurement using edges.

  Here, according to the image processing method using the edge detection apparatus described in Patent Document 1, it is possible to remove fine noise edges as shown in FIG. However, when edge extraction is performed on an image having a double circular portion as shown in FIG. 16C, an edge is formed by the inner and outer diameters of the annular portion as shown in FIG. Extracted to each. Note that reference numeral 31 in FIG. 16D indicates an inner diameter edge that is a detection target edge to be detected, and reference numeral 32 indicates an outer diameter edge that is a non-detection target edge that is not a detection target.

  In the case of such an image, both the inner diameter and the outer diameter are similarly strong and have long edges, so that it is difficult to detect only an arbitrary inner diameter edge. If the non-detection target edge has a shape difference with respect to the detection target edge, it is easy to separate the two based on the shape characteristics. However, when the non-detection target edge and the detection target edge show similar shapes, it is particularly difficult to detect only the detection target edge. That is, as shown in FIG. 16D, since both the detection target edge 31 and the non-detection target edge 32 have a circular shape, it is difficult to separate by shape features.

  Further, according to the image processing method using the edge extraction system described in Patent Document 2, it is learned in advance whether or not the edge is the detection target edge based on the reference image even for the image as shown in FIG. By doing so, it is possible to detect only the detection target edge 31. However, in this image processing method, when the illumination conditions change due to deterioration of the illumination device used for imaging by the camera, the luminance variation of the input image becomes large. Then, a difference due to luminance fluctuation occurs between the teacher data indicating whether the edge has been learned in advance and the edge area data of the input image. For this reason, the detection of the detection target edge 31 becomes unstable.

  The present invention relates to an image processing apparatus capable of stably detecting only a detection target edge even when a change in illumination occurs in a situation where the detection target edge has a non-detection target edge having a similar shape to the detection target edge, An object is to provide an image processing program, a robot apparatus, and an image processing method.

  The present invention stores a magnitude relationship of feature amounts composed of luminance values of a sampling area having a fixed positional relationship with respect to an edge, calculated for each of a plurality of edges on a reference image obtained by imaging a workpiece in advance. An edge extraction unit that extracts an edge of a workpiece on the acquired input image, a feature amount calculation unit that calculates the feature amount for each edge extracted by the edge extraction unit, and a calculated on the input image A histogram showing the frequency distribution of luminance values and frequencies is created from the feature values of each edge, and extracted by the edge extraction step based on the feature value range divided by the threshold value obtained based on the distribution state of the histogram. An edge classifying unit that classifies the detected edge, a magnitude relationship between the range of the feature quantity segmented by the threshold value, and a magnitude relation of the feature quantity stored in the storage unit And an edge relationship setting unit that associates an edge on the input image within the range of the corresponding feature quantity with an edge on the reference image, and an image processing comprising: In the device.

  The present invention resides in an image processing program that causes a computer to function as the image processing apparatus.

  The present invention relates to an illumination device that illuminates a workpiece, a camera that captures a workpiece illuminated by the illumination device, the image processing device, and positional information calculated by the image processing device based on an input image captured by the camera. And a working arm that performs a predetermined process on the workpiece.

  According to the present invention, the magnitude relationship of the feature amount including the luminance value of the sampling area having a fixed positional relationship with respect to the edge, which is calculated for each of the plurality of edges on the reference image obtained by capturing an image of the workpiece in advance by the image processing apparatus. A storage step for storing the image, an edge extraction step in which the image processing device extracts an edge of a work on the acquired input image, and the image processing device performs the feature for each edge extracted in the edge extraction step. A feature amount calculating step for calculating the amount, and the image processing apparatus creates a histogram indicating the frequency distribution of the luminance value and the frequency from the calculated feature amount of each edge on the input image. An edge segmentation step for segmenting the edge extracted by the edge extraction unit according to a range of feature quantities segmented by the threshold value obtained based on the threshold value; The image processing apparatus compares the size relationship between the feature amount ranges classified by the threshold and the feature relationship stored in the storing step, and determines whether the feature amount range is in the corresponding feature amount range. An image processing method comprising an edge relationship setting step for associating an edge on an input image with an edge on the reference image.

  According to the present invention, the magnitude relationship between the ranges of the feature quantities classified by the threshold value is compared with the magnitude relation of the feature quantities stored in the storage unit, and the input within the corresponding feature quantity range is compared. An edge on the image can be associated with an edge on the reference image. Therefore, even when a change in illumination occurs in a situation where the detection target edge has a non-detection target edge having a similar shape, it is possible to stably detect only the detection target edge. .

(A), (b) is a figure for demonstrating the robot apparatus provided with the image processing apparatus of embodiment concerning this invention. It is a block diagram which shows the hardware constitutions in this Embodiment. It is a block diagram which shows in detail the structure of the image processing part of the image processing apparatus in this Embodiment. (A), (b) is a figure for demonstrating this Embodiment. (A), (b), (c), (d) is a figure for demonstrating this Embodiment. (A), (b), (c), (d) is a figure for demonstrating this Embodiment. (A), (b) is a figure for demonstrating this Embodiment. (A), (b) is a figure for demonstrating this Embodiment. (A), (b) is a figure for demonstrating this Embodiment. It is a figure for demonstrating this Embodiment. (A), (b), (c) is a figure for demonstrating this Embodiment. (A), (b) is a figure for demonstrating this Embodiment. It is a flowchart figure for demonstrating this Embodiment. It is a flowchart figure for demonstrating this Embodiment. (A), (b), (c) is a figure for demonstrating a prior art. (A), (b), (c), (d) is a figure for demonstrating a prior art.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. First, an image processing apparatus according to the present invention and a robot apparatus including the image processing apparatus will be described with reference to FIGS. 1A and 1B. The image processing method according to the present invention is described below. And the explanation of the robot apparatus. FIG. 1A is a schematic configuration diagram when an edge is detected from the workpiece 3 to be detected on the placement unit 4 using the camera 1 and the image processing unit 2 in the robot apparatus 100 including the image processing apparatus. FIG. 1B is a perspective view showing the entire robot apparatus 100.

  As shown in FIGS. 1A and 1B, the robot apparatus 100 includes a lighting device 105 that illuminates a detection target work (work) 3, and a camera 1 that images the detection target work 3 illuminated by the lighting device 105. It has. Furthermore, the robot apparatus 100 performs predetermined processing on the detection target workpiece 3 based on an image processing apparatus configured by the computer 19 and position information calculated by the image processing apparatus based on an input image captured by the camera 1. And an actuating arm 102 for applying

  In other words, the robot apparatus 100 includes a booth 103 formed by a plurality of frame members 104 and a gantry 101 disposed at the lower center of the booth 103. Furthermore, an operating arm 102 disposed on the top of the gantry 101 and a mounting unit 4 for mounting the workpiece 3 to be detected in a state of being disposed at a position facing the operating arm 102 are provided. A computer 19 having an image processing unit 2 and the like is disposed in the gantry 101. Above the operation arm 102 in the booth 103, a camera 1 as an imaging unit and an illumination device 105 arranged on both sides of the camera 1 are disposed. The operating arm 102 is composed of an articulated serial link robot, and the position and posture of the arm tip 102 a are controlled by a motor drive based on the computer 19.

  The camera 1 forms an optical image with a lens (not shown), converts the optical image into an imaging signal with an imaging element (not shown), and provides the imaging signal to the image processing unit 2 as an input image that is an edge detection target. The imaging means to perform is comprised.

  As shown in FIG. 1A, the hand portion 102b attached to the arm tip portion 102a is configured to be able to grip and release the workpiece 3 to be detected (workpiece). The hand unit 102b includes a control mechanism such as a rotation mechanism at a joint corresponding to the wrist or a multi-joint finger mechanism within a range sufficient to correct the position / posture of the gripped detection target workpiece 3. Such drive control of the operating arm 102 having the hand unit 102b is realized by the image processing unit 2 based on the imaging data from the camera 1 and the computer 19 using the data from the image processing unit 2.

  As shown in FIG. 2, a computer main body 19a of a computer 19 houses an image processing unit 2 and the like that mainly function as a CPU 5. In addition to the image processing unit 2, a RAM 29 and a ROM 30 are connected to the CPU 5 via the bus 7. A working area for the CPU 5 is secured in the RAM 29. The ROM 30 stores a program necessary for basic control of the computer 19. The image processing unit 2 has a function of displaying a predetermined image by controlling the display 40 such as a liquid crystal according to a drawing instruction from the CPU 5 in addition to an image processing function described later. The ROM 30 in the computer 19 stores an image processing program for causing the computer 19 to function as an image processing apparatus.

  A keyboard 42 and a mouse 43 as input devices are connected to the CPU 5 via an input interface (interface) 41 connected to the bus 7. Specification information necessary for edge detection and the like, menu selection instructions, and others It is possible to input instructions. In addition, the camera 1 is connected to the CPU 5 via an input interface 47 connected to the bus 7. A recording disk reader 44 is connected to the bus 7 so that a recording medium 45 on which various information is recorded can be read and stored in, for example, the ROM 30. In addition, a communication device 46 is connected to the bus 7 so that various information distributed from the Internet or the like via the communication device 46 can be downloaded without using the recording medium 45 as described above. ing.

  Here, the details of the image processing section 2 in the computer 19 will be described with reference to FIG. That is, as shown in FIG. 3, the image processing unit 2 includes a feature amount relative relationship storage unit 8, an input image storage unit 9, an edge image storage unit 10, an edge feature amount storage unit 51, a feature amount histogram storage unit 11, and An edge-feature quantity correspondence storage unit 12 is provided. The edge-feature amount correspondence storage unit 12 includes a selection unit 12a and a position calculation unit 12b.

  In such an image processing unit 2, the input image provided from the camera 1 is stored in the input image storage unit 9, and the CPU 5 accesses each storage unit 8 to 12, 51 via the bus 7 in a timely manner and Execute the process.

  The CPU 5 and the feature amount relative relationship storage unit 8 constitute a storage unit. The storage unit is calculated for each of the plurality of edges (14, 15) on the reference image obtained by capturing the detection target workpiece 3 in advance, and sampling regions 20 to 22 having a fixed positional relationship with respect to the edges (see FIG. 5). ) Is stored in the magnitude relationship between the feature values K, N1, and N2.

  In other words, the feature amount relative relationship storage unit 8 follows the instruction of the CPU 5 from the reference image 13 to the sampling region 20 having a fixed positional relationship with respect to the detection target edge 14 as the detection target and the non-detection target edge 15 other than the detection target. , 21 and 22 are calculated. And it memorize | stores as the feature-value K calculated by the feature-value calculation part mentioned later, and feature-value N1, N2. At that time, the magnitude relationship between the feature value K of the detection target edge 14 and the feature values N1 and N2 of the non-detection target edge 15 is stored. That is, the feature amount relative relationship storage unit 8 is based on the sampling region 20 to the sampling region 20 to a certain positional relationship with respect to the edges 14 and 15 from the reference image 13 of the detection target work 3 imaged in the illumination state according to the instruction of the CPU 5. An average luminance value (k) for every 22 is calculated. Then, the feature quantity K corresponding to the detection target edge 14 and the feature quantities N1 and N2 corresponding to the non-detection target edge 15 are set, and the magnitude relation between the feature quantities K, N1 and N2 is stored. The feature amount relative relationship storage unit 8 stores the size relationship as, for example, “1”, “2”, and “3” of a size relationship flag (flag) described later.

  The input image storage unit 9 stores an input image provided from the camera 1 in accordance with an instruction from the CPU 5. The CPU 5 creates a reference image 13 (see FIG. 4), which is a work image to be compared with the detection target image, from the input image provided from the camera 1 and stores the reference image 13 in the input image storage unit 9.

  The CPU 5 and the edge image storage unit 10 constitute an edge extraction unit that extracts the edge of the detection target workpiece 3 on the acquired input image. That is, the edge image storage unit 10 detects an edge from the detection target image or the reference image in accordance with an instruction from the CPU 5 and stores the image.

  The CPU 5 and the edge feature amount storage unit 51 constitute a feature amount calculation unit that calculates the feature amount K and the feature amounts N1 and N2 for each edge extracted by the edge extraction unit (5, 10). That is, the edge feature amount storage unit 51 calculates the feature amount K and the feature amounts N1 and N2 at each extracted edge in accordance with an instruction from the CPU 5.

  The CPU 5 and the feature amount histogram storage unit 11 constitute an edge division unit. The edge segmentation unit creates a feature amount histogram indicating the frequency distribution of the luminance value and frequency from the calculated feature amount of each edge on the input image, and is classified by the threshold value obtained based on the distribution state of the histogram. The edges extracted by the edge extraction unit are classified according to the amount range.

  That is, the feature amount histogram storage unit 11 creates a feature amount histogram according to instructions from the CPU 5 based on the edge feature amounts K, N1, and N2 obtained by the feature amount calculation unit (5, 51). Then, the threshold value is determined (set) from the histogram and stored, and the edge extracted by the edge extraction unit is classified according to the range of the feature amount classified by the threshold value.

  The CPU 5 and the edge-feature amount correspondence storage unit 12 constitute an edge relationship setting unit. The edge relationship setting unit calculates the size relationship between the ranges of the feature amounts classified by the threshold value and the size relationship (feature relative relationship) of the feature amounts K, N1, and N2 stored in the storage unit (5, 8). Compare. Then, the edge on the input image within the range of the corresponding feature quantity in the size relationship is associated with the edge (14, 15) on the reference image 13.

  That is, the edge-feature amount correspondence storage unit 12 stores an edge-feature amount correspondence table (see FIG. 8B) in accordance with an instruction from the CPU 5, and the feature amount range of the detection target edge 14 and the non-detection target edge 15. The feature amount range is separated and distinguished. Then, an edge having a feature amount within the feature amount range of the detection target edge 14 is selected as the detection target edge. That is, the edge-feature amount correspondence storage unit 12 determines the range of the feature amount K of the detection target edge 14 of the detection target workpiece 3 to be detected this time.

  The selection unit 12a selects the detection target edge 14 to be detected on the input image based on the edge correspondence set by the edge relationship setting unit (5, 12). The position calculation unit 12b calculates the position information of the detection target workpiece 3 from the edge selected by the selection unit 12a.

  Hereinafter, details of the present embodiment will be described with reference to FIGS. 13 and 14. FIG. 13 is a flowchart showing a main flow in the present embodiment, and FIG. 14 is a flowchart showing a subroutine of the edge feature quantity magnitude relation processing in step S2.

  First, when the process starts in FIG. 13, teaching is performed offline. That is, the reference image 13 obtained by capturing the detection target workpiece 3 with the camera 1 in advance is input. Then, in the state displayed on the display 40 connected to the image processing unit 2, the operator performs a teaching operation using the mouse 43 to select only the detection target edge 14 desired to be detected (step S1).

  In step S2, the CPU 5 and the feature quantity relative relationship storage unit 8 determine the magnitude relation between the edge feature quantities K, N1, and N2. In step S2-1 in FIG. 14, the edge feature amount magnitude determination process selects an edge to be used for calculating the feature amount from the reference image 13, and in step S2-2, sets the feature amount of the detection target edge 14 to K, The feature amounts of the non-detection target edge 15 are calculated as N1 and N2. That is, the feature quantity calculation unit (5, 51) calculates the feature quantity K and the feature quantities N1, N2 for each edge extracted by the edge extraction unit (5, 10).

  That is, as shown in FIGS. 4A and 4B, the edge to be detected in the reference image 13 is set as the detection target edge 14 having the innermost diameter, and the other edge that is not desired to be detected is set as the non-detection target edge 15. Teaching is performed using the mouse 43 while the operator looks at the screen of the display 40, and a specific method thereof will be described below.

  First, in step S2-1, the reference image 13 shown in FIG. 4A displayed on the display 40 is operated as follows. That is, as shown in FIG. 4B, the respective positions of the appropriate point 17 on the detection target edge 14, the appropriate point 18 on the non-detection target edge 15, and the point 16 that is one point near the center are respectively set. Set with the mouse pointer. This setting with the mouse pointer means that coordinate information (x, y) on the reference image 13 pointed to by the mouse pointer is transmitted to the image processing unit 2.

  In step S2-2, the luminance value of a region having a certain positional relationship is sampled from the above-described position (points 16, 17, and 18) set with respect to the reference image 13, and the average of the luminance values of the region is calculated. To do. A region having a certain positional relationship is shown below.

  First, as shown in FIG. 5A, a point 16 near the set center is set as a reference point, a straight line connecting the reference point (16) and the point 17 on the set edge, and a reference image 13 An angle θ formed with the y-axis (vertical direction in the figure) is obtained.

  Next, as shown in FIG. 5B, a rectangular sampling region 20 is set using a mouse at a position where the point 17 on the edge is the center of the bottom of the rectangle. Then, as shown in FIG. 5C, the rectangular sampling region 20 is rotated clockwise by the angle θ of the straight line obtained with the point 17 on the edge as a fixed point, and a point 18 on the non-detection target edge 15 (FIG. 4 (b)), sampling areas 21 and 22 are set as shown in FIG. Similarly to the above, the rectangular sampling areas 21 and 22 are rotated in the clockwise direction in the drawing by the angle of the straight line obtained with the point 18 as a fixed point.

  As described above, in this embodiment, the sampling regions 20, 21, and 22 are set so as to be in a positional relationship in contact with the tangent line of the edge (14, 15) on one side of the detection target edge 14 and the non-detection target edge 15. To do. That is, when the workpiece 3 to be detected is configured with a circular edge, the following processing is performed in the rectangular sampling areas 21 and 22. First, an arbitrary one point on each extracted edge is selected, and a straight line connecting the reference point (16) and any one point on each edge with the one point 16 near the center of the workpiece 3 to be detected as a reference point. The formed angle θ is obtained. Furthermore, by rotating an area set at an arbitrary point on each edge by an angle corresponding to the angle θ, the area of the adjacent detection target edge 14 and the area of the non-detection target edge 15 are set so as not to intersect. .

  Up to this point, the case where the detection target edge is circular has been described. However, when the detection target edge is a straight line, an area having a certain positional relationship can be obtained by the following method.

  That is, when the edge of the workpiece to be measured, which is the detection target, is configured with a straight edge as shown in FIG. That is, the positions of the point 23a on the detection target edge 23, the point 24a on the non-detection target edge 24, and the point 25, which is one point near the center, are respectively set with the mouse pointer. Then, the edge of the area centered on each point 23a, 24a on the edge 23, 24 set as shown by the broken line in FIG. 6B is extracted, and the x-axis of the image of the edge (the horizontal direction in the figure) Find the angle to.

  That is, a rectangular sampling region is set at a position where the points 23a and 24a on the edge are the center of the bottom of the rectangle (FIG. 6A), and the size of this region is a rectangle created for the detection target edge. Are set so as not to intersect the adjacent non-detection target edge. For example, when the width between the non-detection target edges adjacent to the detection target edge is about L pixels, the size of one side of the rectangle is set to L / 2. The area to be set is not limited to the rectangular sampling area, but may be a circular area or an elliptical area.

  Further, as shown in FIG. 6 (c), the set rectangular sampling area is rotated by the determined edge angle with the points 23a and 24a on the edge as fixed points. When this rotation is performed, the center position 28 of the upper side of the rectangular sampling area is rotated in a direction away from the reference position 26 as shown in FIG.

  In the case of FIG. 6D, the angle of the edge is 90 degrees with respect to the x-axis, so two rotation directions of plus 90 degrees and minus 90 degrees are conceivable. Assuming that the clockwise direction is plus, the center position (center point) 28 of the upper side of the rectangular sampling area is away from the reference position (reference point) 26 in the plus 90 degree direction, so that it is rotated plus 90 degrees. The actual judgment is made by performing both rotations, calculating the distance between the reference position 26 and the upper side center position 28, and adopting the rotation direction with the longer distance.

  In this way, when the detection target workpiece 3 is composed of straight edges, an arbitrary point on each extracted edge is selected in a rectangular sampling area, and an edge angle near the selected arbitrary point is selected. Ask for. Further, the region set to an arbitrary point on each edge is rotated by an angle corresponding to the edge angle so that the region of the adjacent detection target edge 14 and the region of the non-detection target edge 15 do not cross each other. To do.

  After the area is set, the luminance average value in each area set from the reference image is calculated as described above. The average luminance value can be calculated by dividing the total luminance value in one area by the area of the area. The calculated luminance average value of the detection target edge region and the non-detection target edge region is used as the feature amount of each edge. The value as the feature amount is not limited to the average luminance value of the region, but may be a median value of the luminance of the region or a luminance value of one central pixel of the region (a luminance value of the representative pixel).

  After that, in step S2-3, if feature quantity K> feature quantity N1> feature quantity N2, the magnitude relation flag (flag) = 1 is set and stored in the feature quantity relative relationship storage unit 8 (step S2-3). S2-6).

  In step S2-4, if feature quantity N1> feature quantity K> feature quantity N2, the magnitude relation flag (flag) = 2 is set and stored in the feature quantity relative relationship storage unit 8 (step S2). -7).

  In step S2-5, if feature quantity N1> feature quantity N2> feature quantity K, the magnitude relation flag (flag) = 3 is set and stored in the feature quantity relative relationship storage unit 8 (step S2). -8).

  In the above steps S2-3 to S2-8, the size relationship between the feature amount of the detection target edge 14 and the feature amount of the non-detection target edge 15 is calculated and stored in the feature amount relative relationship storage unit 8. . That is, when the luminance value of the image is large, it is close to white, and when it is small, the image becomes black. For this reason, in the present embodiment, the feature quantity K of the area of the detection target edge 14 shown in FIG. 5D is larger than the feature quantities N1 and N2 of the rectangular sampling areas 21 and 22 of the non-detection target edge 15. And stored in the feature quantity relative relationship storage unit 8. In other words, the feature amount relative relationship storage unit 8 stores that the feature amount K of the detection target edge 14> the feature amount N1 of the non-detection target edge 15> the feature amount N2 of the non-detection target edge 15.

Specifically, the following three first to third states (1) to (3) can be considered as the magnitude relationship of the feature amounts.
(1) When the feature quantity K of the detection target edge 14 is the largest, that is, when the feature quantity K of the detection target edge 14> feature quantity N1 of the non-detection target edge 15> feature quantity N2 of the non-detection target edge 15 .
(2) When the feature value K of the detection target edge 14 is the second largest, that is, the feature value N1 of the non-detection target edge 15> the feature value K of the detection target edge 14> the feature value N2 of the non-detection target edge 15 One day.
(3) When the feature quantity K of the detection target edge 14 is the smallest, that is, when the feature quantity N1 of the non-detection target edge 15> feature quantity N2 of the non-detection target edge 15> feature quantity K of the detection target edge 14 .

  One of the above three states is stored. Therefore, the feature quantity relative relationship storage unit 8 stores the flag 1 in the first state (1) (S2-3, S2-6), and the flag 2 in the second state (2) (S2). -4, S2-7), and in the third state (3), it is only necessary to store the flag 3 in (S2-5, S2-8). That is, the feature quantity relative relationship storage unit 8 calculates a feature quantity using the reference image 13 as an input, and represents a magnitude relationship between the feature quantities K, N1, and N2 (any one of flags 1, 2, and 3). Is output.

  After the edge feature amount magnitude relation processing as described above, an image to be detected is input in step S3. That is, step S3 is a detection process that is actually performed online with respect to the input image of the detection target work 3 that is the detection target. In step S3, the image of the detection target work 3 that is the detection target by the camera 1 Is stored in the input image storage unit 9.

  In step S4, an edge is extracted by an edge extraction unit including the CPU 5 and the edge image storage unit 10. That is, in this step S4, the edge extraction unit extracts edges by using, for example, a Sobel (SobeL) filter which is a typical technique for the input image of the workpiece 3 to be detected input in step S3. Do. Various edge extraction filters are known, but filters other than the SobeL filter may be used.

  In edge extraction, the contour of an object can be extracted. That is, when the edge of the input image shown in FIG. 7A is extracted, an image as shown in FIG. 7B is obtained. In an actual image, fine noise edges appear due to various causes such as scratches on the background and work, uneven illumination, and the like. However, the image after edge extraction shown in FIG. 7B is stored in the edge image storage unit 10 as an edge image after removing a short edge as a noise edge on the basis of the length of the edge.

  Since various methods for removing fine noise edges are known, they are not limited to removal by length. In addition, the extracted edges are subjected to a labeling process in which those having connectivity in the vicinity are combined as one edge so that numbers can be assigned to each connected edge so as to be identified (edges in FIG. 8B). No. 1, 2,...) And the edge-feature amount correspondence storage unit 12.

  Further, in step S5, the feature amount calculation unit including the CPU 5 and the edge feature amount storage unit 51 calculates (calculates) the feature amounts of all the edges. That is, in this step S5, the feature amount calculation unit selects a position of a certain point on each edge from the edge image acquired by the edge extraction unit. This certain point may be a point located at the center of the edge or an arbitrary point.

  Then, as shown in FIG. 8A, a rectangular sampling region is set in a fixed positional relationship, like the feature amount relative relationship storage unit 8, with the position of each point as a point on the edge. At this time, the center position of the work serving as the reference point is the reference position used in the reference image 13 of the feature amount relative relationship storage unit 8 when the detection target work 3 comes to a certain position with respect to the camera 1 each time. It may be used as it is. On the other hand, when the center position of the detection target workpiece 3 is shifted every time, the approximate center position of the detection target workpiece 3 needs to be obtained in advance by pattern matching or the like.

  Then, the luminance value of the input image located in each sampling region (rectangular region) 20, 21, 22 is acquired, an average luminance value is calculated for each rectangular sampling region, and the calculated average luminance value is calculated for each edge. Feature amount. In addition, since the feature values of each edge are stored, the edge number (see FIG. 8B) obtained by the edge extraction unit including the CPU 5 and the edge image storage unit 10 and the edge that indicates the correspondence between the feature values of the edge- A feature quantity correspondence table is created and stored in the edge-feature quantity correspondence storage unit 12.

  Furthermore, in step S6, the edge segmentation unit composed of the CPU 5 and the feature amount histogram storage unit 11 is converted into the feature amounts K, N1, N2 of all edges obtained by the feature amount calculation unit composed of the CPU 5 and the edge feature amount storage unit 51. Based on this, a feature amount histogram is created. Further, in step S7, the edge classification unit determines and stores a threshold value from the feature amount histogram.

  The processing steps from Step S6 to Step S7 are executed by the edge segmentation unit. The edge segmentation unit first creates a histogram of the calculated feature value (average luminance value) with the average luminance value on the horizontal axis and the frequency on the vertical axis as shown in FIG. 9B. And stored in the feature amount histogram storage unit 11. Actually, the histogram is created by preparing 256 tables of 0 to 255 as shown in FIG. 9A and storing how many average luminance values match the table values. . For example, when there are two average luminance values 30, “2” is input to the table value 30. Then, a threshold value (threshold value) is obtained from the created histogram. The method for obtaining the threshold is shown below.

Assuming that the frequency at the luminance value k is H [k] from the feature amount histogram, first, H _WMA [k] is obtained by the weighted moving average method shown in the following equation (1).

  In the above formula (1), n represents an interval to be averaged, and if the value of n is large, smoothing can be strengthened, and if it is small, smoothing can be weakened. A histogram obtained by smoothing the histogram of FIG. 9B with n = 3 is shown as a broken line 29 in FIG.

Next, H _WMA [k] that satisfies the conditional expression shown in the following expression (2) is searched. However, in equation (2), the threshold with the smaller value of k is threshold 1 and the other is threshold 2. In the present embodiment, since two types of non-detection target edges 15 appear in the case of an image as shown in FIG. 4, the two minimum values are set as the threshold values 1 and 2 in this way.

A position where the histogram 29 after smoothing becomes a minimum value is found, and that value is set as a threshold value. In the minimum value search method, k of H _WMA [k] is changed from 0 to 255. Then, for the current _{H WMA} [k], the value _H WMA before and after the histogram (frequency) _[k-1], if _H WMA [k + 1] is greater than _{H WMA} [k], the minimum value of the histogram that the created It is considered.

  In step S8, based on the value (any one of flags 1, 2 and 3) representing the magnitude relationship between the feature quantities K, N1, and N2 stored in the feature quantity relative relationship storage unit 8 in step S2, It is determined whether or not flag = 1. If the magnitude relation flag = 1, the process proceeds to step S11. Otherwise, the process proceeds to step S9. In step S11, an edge satisfying the feature amount K of the detection target edge 14> the threshold 2 (N1 of the non-detection target edge 15) is selected, and the process proceeds to step S14. In step S14, it is determined whether or not there is a next image. If there is, the process returns to step S3 to input an image to be detected. If there is no next image, the process ends.

  In step S9, it is determined whether or not the magnitude relation flag = 2. If the magnitude relation flag = 2, the process proceeds to step S12. Otherwise, the process proceeds to step S10. In step S12, an edge that satisfies threshold value 2 (N1)> feature value K> threshold value 1 (N2) is selected, and the process proceeds to step S14.

  In step S10, it is determined whether or not the magnitude relation flag = 3. If the magnitude relation flag = 3, the process proceeds to step S13. In step S13, an edge satisfying threshold 1 (N2)> feature quantity K is selected, and the process proceeds to step S14.

  The processing steps in steps S8 to S13 are executed by the edge relationship setting unit including the CPU 5 and the edge-feature amount correspondence storage unit 12. That is, if the value (flag) stored in the feature amount relative relationship storage unit 8 is “1”, the edge relationship setting unit sets a feature amount K equal to or greater than the threshold value 2 (N1 of the non-detection target edge 15). It is determined that the held edge is the detection target (S8, S11).

  Further, when the value of the feature quantity relative relationship storage unit 8 is “2”, the edge relationship setting unit is equal to or greater than the threshold 1 (N2 of the non-detection target edge 15) and equal to or less than the threshold 2 (N1 of the non-detection target edge 15). It is determined that the edge having the feature quantity K is a detection target (S9, S12). Further, if the value in the feature quantity relative relationship storage unit 8 is “3”, it is determined that an edge having a feature quantity equal to or less than the threshold value 1 (N2 of the non-detection target edge 15) is a detection target (S10, S13). .

  As described above, in the present embodiment, the threshold value 2 is determined based on the two threshold values 1 and 2 and the magnitude relation flag 1 (feature quantity K> feature quantity N1> feature quantity N2) stored in advance. It can be determined that an edge having the above feature amount K is a detection target. Then, the edge relationship setting unit selects only edges having a feature quantity K equal to or greater than the threshold value 2 based on the created edge-feature quantity correspondence table (FIG. 8B). The edge relationship setting unit generates and outputs an edge image in which only the edge determined as the detection target edge is left out of all the edges extracted by the edge extraction unit including the CPU 5 and the edge image storage unit 10.

  Also, as in the case of the reference image 48 shown in FIGS. 11A, 11B, and 11C, if only one type of non-detection target edge 50 appears, the created histogram (FIG. 12A) is used. The threshold value can be obtained by discriminant analysis. The method is shown below.

That is, the frequency at the luminance value k is H [k], and the temporary threshold is t. Here, reference numeral 30 in FIG. 12B indicates a threshold value t. When the histogram is divided at this threshold t, the left side is class 1 and the right side is class 2. The number of edges (total frequency) in class 1 is ω ₁ , the average of luminance values is m ₁ , and the variance is σ ₁ . To do. On the other hand, assuming that the number of edges in class 2 (the sum of frequencies) is ω ₂ , the average luminance value is m ₂ , and the variance is σ ₂ , the intra-class variance σ _W in class 1 and class 2 according to the following equation (3) Is obtained.

Next, when the average brightness value of the entire histogram and m _t, the following equation (4) Class 1 and Class 2 interclass variance sigma _b is obtained.

  Then, the following equation (5) is used as the degree of separation, the temporary threshold t is changed from 0 to 255, and t when the degree of separation is maximum is set as the threshold.

  The edge relationship setting unit compares the size relationship between the ranges of the feature amounts classified by the threshold and the size relationship between the feature amounts K, N1, and N2 stored in the storage units (5, 8). Then, the edge on the input image and the edge (14, 15) on the reference image 13 in the range of the corresponding feature quantity range are associated with each other. That is, the edge relationship setting unit has a feature amount equal to or greater than the threshold value based on the relative relationship between the threshold value and the previously stored luminance value (brightness value near the detection target edge> brightness value near the non-detection target edge). It can be determined that the edge is a detection target.

  The above processing flow is the image processing method (edge detection method) of the present invention for one input image. When the next image is continuously input and this image processing method is performed, the process returns to step S3 as described above, and an image of the next detection target workpiece 3 to be newly detected is input (step S14). ).

  As described above, the relative relationship between the luminance value around the detection target edge 14 calculated in advance from the reference image 13 and the luminance value around the non-detection target edge 15, and the detection target workpiece 3 sent sequentially. Only necessary detection target edges can be selected using a threshold value calculated for each input image. That is, when a change occurs in the lighting state of the lighting device 105, the threshold value can be dynamically changed in accordance with this change. Therefore, as long as the relative relationship between the feature amounts calculated in advance does not change, the lighting state of the lighting device 105 is affected. Instead, only the detection target edge can be detected robustly.

  In the present embodiment described above, the luminance value k of a region having a certain positional relationship with respect to the detection target edge 14 and the non-detection target edge 15 is calculated from the reference image 13 as the feature amount K in advance. Then, the magnitudes of the feature quantity K of the detection target edge 14 and the feature quantities N1 and N2 of the non-detection target edge 15 are compared, and the magnitude relationship is stored. Then, an edge is extracted from the input image to be detected, feature amounts K, N1, and N2 of all the extracted edges are calculated, and a histogram of the feature amount is created.

  Further, a threshold value is obtained from the created histogram, and the feature amount range of the detection target edge 14 and the non-detection target are determined based on the obtained threshold value and the magnitude relationship between the feature amounts K, N1, and N2 calculated in advance from the reference image 13. The feature amount range of the edge 15 can be separated and distinguished. Then, by selecting only edges having a feature amount within the feature amount range of the detection target edge 14, it is possible to accurately detect only the detection target edge 14 from the input image regardless of changes in the brightness of the illumination device 105. Can do.

  As described above, the edge feature amount is calculated every time for the input image of the detection target workpiece 3 to be detected, and the threshold value of the feature amount for separating the detection target edge 14 and the non-detection target edge 15 is dynamically set. Can be calculated. The feature amount is calculated based on the luminance value of the sampling area. For this reason, even when the luminance fluctuation in the input image increases due to deterioration of the lighting device 105 or the like, the magnitude relationship between the feature amounts of the detection target edge 14 and the non-detection target edge 15 obtained in advance from the reference image 13. As long as does not change, both edges 14 and 15 can be stably separated.

  Further, when obtaining a region having a certain positional relationship with respect to the edge for calculating the feature amount, one point on each edge is selected. Then, when the detection target object (detection target workpiece 3) is circular, the angle between the reference point (16) and the points 17, 21, 22 on each edge is obtained with one point 16 near the center as the reference point. On the other hand, when the detection target object is linear, the angles of the edges 23 and 24 on the selected point are obtained.

  Then, as shown in FIGS. 5B and 6A, the region set at the point on the edge is rotated by the calculated angle. This eliminates the inconvenience of crossing each edge with the set area by following the edge tangent in the case of a circle and along the length of the edge in the case of a straight line. Therefore, it is possible to reliably prevent an erroneous feature amount from being calculated when the edge feature amount is obtained based on the luminance value of the region.

  The operator uses the reference image 13 to teach the magnitude relationship between the feature quantity K of the detection target edge 14 and the feature quantities N1 and N2 of the non-detection target edge 15. At that time, an arbitrary point 17 on the detection target edge 14, an arbitrary point (21, 22) on each edge other than the target, and a point 16 near the center of the detection target object are taught using a mouse pointer or the like. Just do it. The calculation of the angle, feature amount, the storage of the magnitude relationship, etc. can be automatically set by a program as long as the position of each point 16, 17 (21, 22) is known, so simple teaching as described above is sufficient. become.

DESCRIPTION OF SYMBOLS 1 ... Camera, 3 ... Work (detection object work), 5, 8 ... Memory | storage part (CPU, feature-value relative relationship memory | storage part), 5, 10 ... Edge extraction part (CPU, edge image memory | storage part), 5, 11 ... Edge classification unit (CPU, feature amount histogram storage unit), 5, 12... Edge relationship setting unit (CPU, edge-feature amount correspondence storage unit), 5, 51... Feature amount calculation unit (CPU, edge feature amount storage unit) , 13 ... Reference image, 14 ... Edge (detection target edge), 15 ... Edge (non-detection target edge), 19 ... Computer, 20, 21, 22 ... Sampling area (rectangular area), 105 ... Illumination device, K, N1 , N2 ... features

Claims (6)

A storage unit that stores a magnitude relationship of feature amounts including brightness values of a sampling area that is calculated with respect to each of the plurality of edges on a reference image obtained by imaging a workpiece in advance and in a fixed positional relationship with the edges;
An edge extraction unit for extracting the edge of the workpiece on the acquired input image;
A feature amount calculation unit that calculates the feature amount for each edge extracted by the edge extraction unit;
A histogram showing the frequency distribution of the luminance value and frequency is created from the calculated feature value of each edge on the input image, and the feature value range divided by the threshold obtained based on the distribution state of this histogram, An edge segmentation unit that segments the edges extracted by the edge extraction unit;
By comparing the magnitude relation of the range of the feature quantity divided by the threshold and the magnitude relation of the feature quantity stored in the storage unit, an edge on the input image within the corresponding feature quantity range and An edge relationship setting unit that associates an edge on the reference image,
An image processing apparatus. The sampling region is set so as to be in a positional relationship in contact with the tangent of the edge on one side of the edge.
The image processing apparatus according to claim 1. A selection unit that selects an edge to be detected on the input image based on the correspondence between the edges set by the edge relationship setting unit;
A position calculation unit that calculates position information of the workpiece from the edge selected by the selection unit,
The image processing apparatus according to claim 1 or 2. Allowing a computer to function as the image processing apparatus according to any one of claims 1 to 3;
An image processing program characterized by that. A lighting device for illuminating the workpiece;
A camera for imaging a workpiece illuminated by the illumination device;
An image processing apparatus according to claim 3,
An actuating arm that performs a predetermined process on the workpiece based on position information calculated by the image processing device based on an input image captured by the camera.
A robot apparatus characterized by that. A memory for storing a magnitude relationship of feature amounts composed of luminance values of a sampling area having a fixed positional relationship with respect to an edge, which is calculated for each of a plurality of edges on a reference image obtained by imaging a workpiece in advance. Process,
An edge extraction step in which the image processing apparatus extracts an edge of a workpiece on the acquired input image;
A feature amount calculating step in which the image processing apparatus calculates the feature amount for each edge extracted in the edge extraction step;
The image processing apparatus creates a histogram indicating a frequency distribution of luminance values and frequencies from the calculated feature values of each edge on the input image, and is classified by threshold values obtained based on the distribution state of the histogram. An edge division step of dividing the edge extracted in the edge extraction step according to a range of feature amounts;
The image processing apparatus compares the size relationship of the feature amount range classified by the threshold value with the feature amount stored in the storing step, and within the corresponding feature amount range. An edge relationship setting step for associating an edge on the input image with an edge on the reference image,
An image processing method.

Images