JP2002230562A - Image processing method and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely extract rectilinear portions of the contour of a subject and make measurements of the position and direction of the subject and the process of extracting defects in the contour based on the rectilinear portions extracted. SOLUTION: A control part 8 extracts edge pixels on a gray image of the subject of processing and forms a histogram showing the degree of edge pixels on the image for each edge direction. The control part 8 then extracts peaks on the histogram, and after a respective label is set for an angle corresponding to each peak, labels set for the corresponding angles are allocated to the edge pixels having the directions corresponding to the angles. Further, the control part 8 allocates a respective label to each assembly of consecutive edge pixels for the same label on a labeled image so that the assembly of edge pixels constituting a line on the image is divided into sub-assemblies. Thereafter, the control part extracts a line matching extraction requirements, using the characteristic quantities of the sub-assemblies.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the field of processing gray-scale image data by a computer, and more particularly, to an object having a straight line portion in its contour as an observation target and extracting a straight line portion of the contour of the object. The present invention relates to a method and an apparatus for performing processing such as measuring the position and orientation of an object using the information, and inspecting whether a defect has occurred in a contour shape.

[0002]

2. Description of the Related Art In observing the position and orientation of an object on a grayscale image and inspecting a defect on the outline, it is sometimes necessary to extract a straight line portion of the outline of the object. Conventionally, when extracting the contour of an object, a method based on a binarization process, a method based on the magnitude of a density gradient, a method using an expanded image and a contracted image, and the like are used.

In the binarization method, a grayscale image is binarized into a white pixel region and a black pixel region by a predetermined threshold value, and an outline is formed by pixels located at the boundary between these regions. In the method based on the magnitude of the density gradient, when the magnitude of the density gradient at each pixel forming the grayscale image exceeds a predetermined threshold value, the pixel is set as a pixel forming the contour line. In the method using the dilated image and the contracted image, a dilated image in which a bright region of the original image is expanded and a contracted image in which the bright region of the original image is contracted are created, and a difference image between the dilated image and the contracted image is created. To extract a contour line.

[0004]

In any of these methods, the whole contour is extracted, and it is not possible to extract only the straight portion of the contour. In order to extract a straight line portion of the contour, it is conceivable to separately evaluate a connection pattern of pixels constituting the contour and specify a portion that can be regarded as a straight line. For example, there is a method in which attention is paid to pixels constituting the contour in order, and a range in which the connection direction is constant is regarded as a straight line. However, in order to increase the resolution in the connection direction, it is necessary to average the connection direction over a range of several pixels, so that even a minute defect that interrupts the straight line tends to be ignored. In addition, when the contour has a line width of a plurality of pixels, it is difficult to set an appropriate algorithm for straight line extraction.

[0005] Furthermore, the state of the image (for example, the density of the background portion and the object portion of the image) can be determined by any of a binarization method, a method based on the magnitude of a density gradient, and a method using an expanded image and a contracted image. The state of the outline (line width, position, size of minute unevenness, etc.) of the outline fluctuates due to differences in the difference, uniformity of illumination, etc. It is difficult to extract a straight line portion of a line by a method based on evaluation of a pixel connection pattern.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to accurately extract a straight line portion of a contour of an object in a grayscale image. Another object of the present invention is to accurately extract a straight line portion of a contour of an object regardless of the state of a grayscale image, particularly when the density difference between the background portion and the object portion is small or when illumination is not uniform. That is. Still another object of the present invention is to perform processing for measuring the position and orientation of an object based on a straight line portion of the extracted outline of the object and extracting defects on the outline of the object.

[0007]

According to the method of processing a grayscale image according to the present invention, when the outline of an object image represented by the grayscale image includes a straight line portion, an individual label is provided for each direction of the straight line. A label setting step to be set, and assigning a label corresponding to the coincident direction to an edge pixel included in the grayscale image, the edge pixel direction of which coincides with one of the directions of the straight line. A pixel labeling step and a line segment extracting step of extracting a set of edge pixels to which the same label is assigned and which can be regarded as being continuous on the grayscale image as a line segment are sequentially executed. .

An edge pixel is a pixel that forms the contour of an object in a grayscale image. The line width of the outline is not limited to one pixel, but may be a line width of a plurality of pixels. As a preferred specific process for extracting an edge pixel, a process in which a pixel whose density gradient is larger than a predetermined value is set as an edge pixel can be employed. In addition, all pixels extracted by this processing are not limited to edge pixels, and pixels remaining after executing processing for narrowing down target pixels, such as thinning processing, following this processing are assumed to be edge pixels. Can also. On the other hand, it is also possible not to pay attention to the magnitude of the density gradient in a specific processing step. For example, a pixel corresponding to a contour line of a binarized image can be an edge pixel. However, in any case, a pixel having a relatively large density gradient is extracted as a result.

According to a general definition, the direction of the edge pixel is a direction orthogonal to the direction of the density gradient at the edge pixel. The definition of the edge pixel direction in this way is
This is because, when a certain edge pixel is included in the straight line portion of the contour of the target object image, it is necessary to indicate the direction of the edge pixel and the direction of the straight line as the same direction. Therefore, in the actual processing, for example, the direction of the straight line is defined by the normal direction of the straight line unlike the ordinary meaning, and the direction of the edge pixel is defined by the direction of the density gradient at the edge pixel. The present invention can also be implemented by adopting the definition of the direction of the straight line and the direction of the edge pixel such that the relationship between the direction of the straight line and the direction of the edge pixel becomes equivalent to the case according to the usual definition. The coincidence between the direction of the edge pixel and the direction of the straight line is not limited to the case of strict coincidence, but also includes the case where the direction difference is within a predetermined range set according to the purpose of image processing.

The fact that two edge pixels can be regarded as being continuous is not necessarily limited to the case where no other pixel is interposed between the pixels, but the continuous judgment method adopted therefor is not limited thereto. It suffices if each pixel satisfies the criterion for judging that it has been performed. If there is a pixel to be determined in the vicinity of the pixel of interest in the four neighboring areas (upper, lower, left, and right, or in the eight neighboring areas obtained by adding the oblique direction), a method of determining that the pixels are continuous is often used. It can be set larger.

According to the present invention, attention is paid to the fact that the direction of the edge pixel can be obtained with a small error because the density gradient is relatively large in the edge pixel forming the contour line. Since the line segment is represented by a set of edge pixels extracted on the condition that the labels assigned to the edge pixels are common and that the edge pixels are continuous, the straight line portion of the outline of the object is It can be accurately extracted. Also, regardless of the state of the grayscale image, especially when the density difference between the background portion and the object portion is small or when the illumination is not uniform,
Since the direction of the edge pixel can be obtained stably,
Even in such a case, the straight line portion of the contour of the object can be accurately extracted.

Each step of the present invention may be executed for the entire gray image, or a measurement area may be set in a part of the gray image and executed only for the measurement area.

According to one embodiment of the image processing method according to the present invention, the label setting step includes a histogram creation step of creating a histogram indicating the degree of the number of edge pixels for each direction of the edge pixels, and a histogram creation step. A peak label setting step of extracting a local maximum value and setting an individual label for each direction of an edge pixel corresponding to the local maximum value, thereby obtaining a separate label for each direction of a straight line included in the contour of the target object image. A label can be set. Further, in the pixel labeling step, the label can be assigned by assigning the label set in the peak label setting step to an edge pixel having a direction of an edge pixel corresponding to each label.

As the degree of the number of edge pixels in each direction of the edge pixels, the number of edge pixels in each direction of the edge pixel itself (so-called frequency) can be adopted. Alternatively, the number of edge pixels weighted by the magnitude of the density gradient at each edge pixel (for example, multiplied by the magnitude of the density gradient) may be employed.

As the direction of the edge pixel corresponding to the maximum value, the direction of the edge pixel corresponding to the section of the histogram to which the maximum value belongs can be adopted, and the section adjacent to the section to which the maximum value belongs is also included. Edge pixel directions corresponding to a plurality of sections may be employed.

According to this embodiment, it is possible to extract the straight line portion of the contour line regardless of the direction of the straight line portion of the contour line of the object, even if the direction is not known in advance. it can.

In this embodiment, when extracting the maximum value of the histogram, conditions for the extraction may be set. For example, conditions such as a maximum value from a larger value to a predetermined number, a maximum value larger than a predetermined value, and the like can be set.

Further, among the line segments extracted in the line segment extracting step, a set of line segments whose direction shift amount within the line segment is within a predetermined range and position shift amount within the line segment within a predetermined range is included. At some point, a line segment integration step of integrating these line segments into one line segment may be further executed.
In this way, even if the line segment is divided due to a defect such as a chip or a protrusion in the outline of the object, the divided line segment can be integrated. Also, it is possible to accurately measure the position and orientation.

Further, a line segment selecting step of selecting a line segment satisfying a predetermined condition from the line segments extracted in the line segment extracting step or the line segments integrated in the line segment integrating step is further executed. You may. In this way, it is possible to select a line segment suitable for the specific purpose of the image processing. Conditions for extracting a line segment include a range of the length of the line segment, a range of an angle with respect to the reference direction,
The range of positions, the range of distance between the endpoints of the two line segments,
An angle range formed by two line segments, a rank order of the line segments, and the like can be adopted.

Further, a display step of displaying the position of the line segment selected in the line segment selecting step or the position of the intersection on the extension line of the plurality of selected line segments so as to be identifiable on the grayscale image may be further executed. It may be. In this way, the status of line segment extraction and selection can be presented to the user. By viewing this display, the user can confirm whether the intended portion such as the corner of the object is correctly measured.

According to another embodiment of the image processing method according to the present invention, there is provided a defect determining step for determining the presence or absence of a defect in a straight line portion of the outline of the object using the extracted state of the line segment in the line segment extracting step. More can be done. According to this embodiment, the line segment is extracted based on the continuity of the edge pixels such that the directions of the edge pixels can be considered to be the same. However, the defect of the outline is as small as several pixels. However, since the direction of the edge pixel greatly changes, such a minute defect is detected with high accuracy.

Specifically, in the defect determining step, in the line segment extracted in the line segment extracting step, the direction deviation amount between the line segments is within a predetermined range, and the positional deviation amount between the line segments is within a predetermined range. When there is a set of line segments within the predetermined range, it can be determined that a defect exists between these line segments. In the defect determining step, the number of line segments extracted in the line segment extracting step is compared with a predetermined reference value, and when the two values are different, it is determined that a defect exists in a straight line portion of the contour of the object. You may do it.

According to another embodiment of the image processing method according to the present invention, the label setting step sets an individual label for each direction assumed as a direction of a straight line portion in the outline of the object. Wherein the pixel labeling step assigns the label corresponding to the coincident direction to an edge pixel whose direction of the edge pixel coincides with any of such assumed directions. Can be.

When the direction of the straight line portion of the contour can be assumed in advance, such as when the direction of the object is determined, an individual label is set for each of such directions, so that each gray-scale image to be processed is set. Since it is not necessary to create the above-mentioned histogram, the processing can be sped up.
Further, the method of this embodiment can also be used for pass judgment and classification judgment of the object. That is, the pass / fail or the type of the object is determined based on whether or not the outline of the object has a straight line portion in the assumed direction. The above-described line segment integration step, line segment selection step, and defect determination step can be employed in this embodiment.

According to the present invention, there is provided an apparatus for processing a grayscale image, comprising: means for inputting a grayscale image; means for extracting edge pixels included in the grayscale image; means for determining a direction of an edge pixel in each of the edge pixels; Direction setting means for setting the direction of a straight line portion of the outline of the object image represented in the grayscale image; label setting means for setting an individual label for each direction of the straight line; Pixel labeling means for assigning the label corresponding to the coincident direction to an edge pixel that coincides with any one of the directions, and the same label is assigned, and Line segment extracting means for extracting a set of edge pixels which can be regarded as such as a line segment.

According to the image processing apparatus of the present invention, an edge pixel is extracted and a direction of the edge pixel is determined for the input grayscale image, while a straight line of the contour of the object image represented in the grayscale image is obtained. An individual label is set for each direction of the portion, and a label corresponding to the matching direction is assigned to an edge pixel whose edge pixel direction matches one of the straight line directions. Further, a set of edge pixels to which the same label is assigned and which can be regarded as being continuous on the grayscale image is extracted as a line segment. In this manner, the extracted portion of the outline of the object can be accurately extracted.

The means for inputting the grayscale image is, for example, a means connected to image generating means such as a camera or a scanner, and for taking in the grayscale image generated by these means.
It is composed of an interface circuit, an A / D conversion circuit and the like. However, the configuration for inputting an image is not limited to this, and may be configured by a circuit that receives an image transmitted by communication, a reading device that reads an image stored in a predetermined recording medium, or the like.

Each means from the means for extracting edge pixels to the means for extracting line segments and each additional or more specific means to be described later can be realized as computer hardware and software operating thereon.
ASIC (Applicati
on Specific Integrated Circuit), and cooperative operation of each circuit block is controlled by a computer.

As means for extracting the edge pixels and means for determining the direction of the edge pixels, an edge extraction filter such as a Sobel filter can be used. The means for extracting edge pixels is not limited to this, and the various contour extraction methods described above may be executed on an image memory of a computer.

According to one embodiment of the image processing apparatus according to the present invention, the direction setting means includes a histogram creating means for creating a histogram indicating the degree of the number of edge pixels for each direction of the edge pixels, Peak direction setting means for extracting the maximum value and setting the direction of the edge pixel corresponding to the maximum value as the direction of a straight line. The label setting means sets an individual label for each direction set by the peak direction setting means. The pixel labeling means sets the label set by the label setting means to an edge corresponding to each label. It may be assigned to an edge pixel having a pixel direction.

In this embodiment, in the line segments extracted by the line segment extracting means, the direction shift amount between the line segments is within a predetermined range, and the position shift amount between the line segments is within a predetermined range. When there is a set of minutes, a line segment integrating means for integrating these line segments into one line segment may be further provided.

Further, the apparatus further comprises a line segment selecting unit for selecting a line segment satisfying a predetermined condition from the line segments extracted by the line segment extracting unit or the line segments integrated by the line unit integrating unit. You may. The predetermined condition for selecting the line segment may be input from outside by further providing an input means for inputting the line segment, or further provided by a storage means for storing this in advance. It may be provided from storage means. Further, the image processing apparatus further includes display control means for displaying the position of the line segment selected by the line segment selection means or the position of the intersection on the extension line of the plurality of selected line segments so as to be identifiable on the grayscale image. You may.

According to another embodiment of the image processing apparatus according to the present invention, the presence / absence of a defect in the straight line portion of the contour of the object is determined using the extraction state of the line segment extracted by the line segment extraction means. The apparatus may further include a determination unit. Specifically, the defect discriminating means is such that, among the line segments extracted by the line segment extracting means, the direction deviation amount between the line segments is within a predetermined range, and the positional deviation amount between the line segments is within a predetermined range. When there is a set of line segments, it can be determined that a defect exists between these line segments. The defect determining means compares the number of line segments extracted by the line segment extracting means with a predetermined reference value, and determines that a defect exists in a straight line portion of the contour of the object when the two values are different. You can do it.

According to another embodiment of the image processing apparatus according to the present invention, the direction setting means inputs an assumed direction of a straight line portion in the contour of the object, and the label setting means sets the direction setting. Means for setting an individual label for each direction input by the means, and the pixel labeling means sets an edge pixel whose edge pixel direction matches one of the directions input by the direction setting means. , A label corresponding to the matched direction can be assigned. Furthermore, the line segment integrating means, the line segment selecting means, and the defect determining means described above can be employed in this embodiment.

[0035]

FIG. 1 shows the configuration of an image processing apparatus according to an embodiment of the present invention. This image device 1 is for extracting a linear part of a contour of an object and an intersection of the extensions of the linear part when there are a plurality of linear parts on a digital grayscale image. , Image output unit 4, timing control unit 5, character
It comprises a graphic memory 6, a character memory 7, a control unit 8, a monitor 9, an I / O port 10, and the like.

The image input unit 2 includes an interface circuit for receiving a grayscale image signal from an analog camera (not shown), an A / D converter for digitally converting the grayscale image signal, a filter circuit for noise cut, and the like. You. The camera is not limited to an analog camera, but may be a digital camera.

The image memory 3 takes in and stores digital image grayscale image data (hereinafter simply referred to as "image") which has been taken in by the image input section 2 and A / D converted. The character / graphic memory 6 stores image data necessary for displaying an edge code histogram, a processing result image, and a screen for setting line segment extraction conditions, which will be described later. The character memory 7 stores text data for displaying character information such as an inspection result and a display position thereof. These memories 3, 6, and 7 are connected to a control unit 8 via an address / data bus 11, and output data according to an instruction from the control unit 8 according to a timing signal from the timing control unit 5. Output to the unit 4 or the address / data bus 11.

The control unit 8 includes a CPU 12, a ROM 1
3, a hard disk 15 having a RAM 14 as a main body and a control program indicating a processing procedure for inspection installed therein. The CPU 12 executes a target measurement process while reading and writing information from and to each memory via the address / data bus 11 based on a control program in the hard disk 15.

The image output unit 4 gives the monitor 9 the image of the object to be measured, the histogram, the processing result image, and the character information indicating the inspection result in a single or combined state. To be displayed. I / O
The port 10 is connected to an input unit such as a keyboard and a mouse, and to an output unit such as an external storage device and a transmission unit, and is used for inputting various setting data from the input unit and outputting test results to the outside. Can be

In the image processing apparatus 1 of this embodiment, a measurement area including an object to be measured is set on an input image, and a line segment forming a contour of the object to be measured is extracted. As a condition, input of data indicating the characteristics of the line segment is received from the I / O port 10. The measurement area is set based on preset conditions, but is not limited to this. Each time an image is input, a measurement area of an arbitrary size is provided at an arbitrary position according to a user's setting operation. You may. Alternatively, the entire input image may be used as the measurement area.

FIG. 2 shows a flow of a series of measurement processing in the image processing apparatus 1 (ST1 to ST10). Hereinafter, FIGS.
4 will be described in detail along the flow of FIG. 2 with reference to FIG.

First, in ST1, edge pixels in the measurement area are extracted. In this edge pixel extraction processing, while scanning a 3 × 3 mask as shown in FIG. 3 on an image, the density value I of each pixel in the mask is determined for each scanning position by the following (1).
By applying Equations (3) to (3), the density gradient Ex (x, y), Ey (x, x, y) of each of the central pixel g (pixel at the coordinate position (x, y)) in each of the x and y axial directions is obtained. y) and the magnitude of the density gradient (hereinafter referred to as “edge strength”) Ei (x, y) are calculated. This edge strength Ei
When (x, y) exceeds a predetermined value, the target pixel g is recognized as an edge pixel.

[0043]

(Equation 1)

[0044]

(Equation 2)

[0045]

(Equation 3)

For a simple system suitable for high-speed operation, the expression (3) can be substituted by the following expression (4). Ei (x, y) = Ex (x, y) + Ey (x, y) (4)

In this embodiment, as will be described later, the mask is scanned by software processing to extract the edge pixels. However, the present invention is not limited to this. The image input to the image memory 3 is performed by a dedicated differentiating circuit. The edge pixel extraction processing may be performed in parallel with the above, and the following processing may be performed using the result.

The concentration gradients Ex (x, y) and Ey (x,
y) shows the amount of change in density at the target pixel g for each of the x and y axis directions. Edge strength Ei (x,
y) are these concentration gradients Ex (x, y), Ey (x,
This corresponds to the length of the composite vector of the vector indicated by y). When the edge strength Ei (x, y) exceeds a predetermined value as described above, the target pixel g is recognized as an edge pixel. The direction indicated by the composite vector indicates the direction in which the density changes in the target pixel g (that is, the direction of the density gradient).

In the next ST2, for each edge pixel extracted in ST1, an angle indicating the direction of the edge pixel (hereinafter, numerical data indicating this angle is referred to as "edge code").
That. ) And a histogram in which the number of edge pixels in the measurement area is counted for each edge code value (hereinafter, this histogram is referred to as an “edge code histogram”).
That. ) To create.

FIG. 4 shows how to express an edge code by enlarging a part of the outline of the object. In the figure, E
Are edge pixels extracted by the above equations (1) to (3). In this embodiment, a vector F from a higher density to a lower density is defined as a direction of a density gradient in the edge pixel E, and a vector C orthogonal to the vector F is defined as a direction represented by an edge code. The illustrated example is an example in which the density of the object is lower than the background (that is, the object is darker than the background). When the density relationship between the object and the background is reversed, the vector F , C are reversed.

In this embodiment, an angle Ec when the vector C is viewed in the counterclockwise direction from the vector B with reference to the vector B from the edge pixel E in the positive direction of the x-axis.
(X, y) is the edge code.

The vector F is a composite vector of the density gradients Ex (x, y) and Ey (x, y) obtained by the above equations (1) and (2), and the vector C is a vector orthogonal to the vector F. Therefore, the edge code Ec (x, y) is
It is determined by one of the following equations according to the values of the density gradients Ex (x, y) and Ey (x, y).

Ex (x, y)> 0 and Ey
When (x, y) ≧ 0, Ec (x, y) = atan (Ey (x, y) / Ex
(X, y)) Ex (x, y)> 0 and Ey (x, y) <0
, Ec (x, y) = 360 + atan (Ey (x, y) /
Ex (x, y)) When Ex (x, y) <0, Ec (x, y) = 180 + atan (Ey (x, y) /
Ex (x, y)) Ex (x, y) = 0 and Ey (x, y)> 0
Where Ec (x, y) = 0 Ex (x, y) = 0 and Ey (x, y) <0
, Ec (x, y) = 180

In order to create the edge code histogram, the edge code Ec (x, y) is calculated for each edge pixel in the measurement area by using the above-mentioned to calculate the edge code Ec (x, y). ) Are added one by one, or a frequency weighted by the edge strength Ei (x, y) of the edge pixel of interest is added. In this embodiment, the angle on the edge code histogram is set in units of one degree. Further, when the calculation of the edge code and the frequency addition process for all the edge pixels are completed, the histogram smoothing process is performed to remove noise.

In this embodiment, the edge code histogram is created after the edge pixels in the measurement area are completely extracted. However, the present invention is not limited to this. Each time the calculation is performed, the edge code calculation and the frequency addition processing may be performed continuously.

On the image, the edge code of each edge pixel forming the line segment should all take a value corresponding to the inclination of the line segment. Therefore, when a line segment exists in the measurement area, a peak appears near the angle corresponding to the inclination of the line segment in the edge code histogram. When a plurality of line segments having different inclinations exist in the measurement area, peaks appear at angular positions corresponding to the inclinations of the line segments on the edge code histogram.

FIG. 5A shows a gray-scale image to be processed. In the figure, reference numeral 16 denotes an image of a target object, which has a density lower than that of the background and has a cross-shaped contour shape combining line segments that can be classified in four orthogonal directions. FIG.
(2) shows an edge code histogram created for the object of FIG. 5 (1), in which four peaks P1 to P4 corresponding to four directions of the contour shape appear.

In ST3, an angle corresponding to the peak is extracted on the histogram. In order to extract the peak, for example, the frequencies for each angle in the edge code histogram are compared in ascending order of angle (however, the frequency when the edge code is 359 degrees is compared with the frequency when the edge code is 0 degrees). ), A mountain-like frequency change of a predetermined size or more is extracted, and an angle corresponding to this peak is determined.

When the angles corresponding to the peaks are obtained in this way, in the next ST4, individual labels are set for the angles corresponding to these peaks (hereinafter, referred to as "peak angles"). In the subsequent ST5, labels set at the corresponding peak angles are assigned to edge pixels having edge codes corresponding to the respective peak angles in the measurement area. It should be noted that a label (for example, “null”) indicating that it is not data to be processed is assigned to an edge pixel to which a label corresponding to any one of the peak angles has not been assigned or a pixel that is not an edge pixel.

In ST5, not only the peak angle, but also
Within a range of angles corresponding to the range of the mountain-like frequency distribution around the peak angle (for example, within a range of angles corresponding to a portion where the height of the histogram exceeds a predetermined value, or at a predetermined position before and after the peak angle). It is preferable to assign a label set to the peak angle to an edge pixel having an edge code corresponding to an angle range (for example, within an angle range). FIG.
In (2), for each of the peaks P1 to P4, an angle range around the peaks P1 to P4 at which a relatively high frequency is obtained is extracted, and labels θ1 to θ4 are added to the angles included in these angle ranges. Corresponding.

FIG. 6A shows an example in which the labels θ1 to θ4 are assigned to the edge pixel extraction result of the image of FIG. 5A. According to the illustrated example, since the edge code of the edge pixel forming the line segment corresponds to any one of the peaks P1 to P4 of the edge code histogram, each label corresponds to any label of θ1 to θ4. Will be attached.

In the state shown in FIG. 6A, if there are a plurality of line segments corresponding to the same edge code on the image, the same label is assigned to these line segments. That is, ST5
It is not possible to completely separate each line segment on an image only by the labeling processing of. Therefore, in the next ST6, relabeling is performed so that an individual label is provided for each independent line segment. (Hereinafter, the processing in ST5 is referred to as “temporary labeling processing”, and the processing in ST6 is referred to as “real labeling processing”.)

In the present labeling process, a set of continuous edge pixels having the same label assigned by the provisional labeling process in ST5 is extracted, and individual labels are reassigned for each set of edge pixels thus extracted. .

FIG. 6 (2) shows the result of performing the main labeling process on the result of the temporary labeling process of FIG. 6 (1). The three line segments to which the label θ1 is assigned are represented by θ, respectively. ₁₁ , θ ₁₂ , and θ ₁₃ are separated. Similarly, the three line segments to which the label θ2 is assigned include the labels θ ₂₁ , θ ₂₂ , and θ ₂₃ , and the three line segments to which the label θ3 is assigned include θ ₃₁ ,
The labels of θ ₃₂ and θ ₃₃ are assigned to the three line segments to which the label θ 4 is assigned, and the labels of θ ₄₁ , θ ₄₂ and θ ₄₃ are assigned to the three line segments.
Each line segment is divided and assigned.

By such a two-stage labeling process, each line segment can be recognized for each label. In ST7, in this state, the following operations A to F are executed for each label using a set of edge pixels to which the label is assigned (hereinafter, this is referred to as a “label set”). , The feature amount of each line segment is calculated. Incidentally, A, sigma operations in B is intended to mean that obtaining the sum of the calculation results for each edge pixel which both are included in one set of labels, also (x _n, y _n) is in the label set shows shows one coordinate edge pixels, Ei (x _n, y _n), Ec (x _n, y _n) each coordinates (x _n, y _n) edge intensity of the edge pixels located in the edge code .

A. Gradient center of gravity: GX, GY Calculates the coordinates of the barycentric position of the line segment indicated by the label set, and the edge intensity Ei (x
_n , y _n ) are obtained by the following equations (a) -1 and (a) -2 using weights. _{GX = Σ {Ei (x n} , y n) · x n} / {Ei (x n, y n)} ··· (a) -1 GY = Σ {Ei (x n, y n) · y n _{} / {Ei (x n,} y n)} ··· (a) -2

B. Direction sum: SCX, SCY The edge code of each edge belonging to the label set is x axis, y
This is the sum of the values decomposed into the components in the respective axial directions of the axis, and is obtained by the following equations (b) -1 and (b) -2. _{SCX = Σcos {Ec (x n} , y n)} ··· (b) -1 SCY = Σsin {Ec (x n, y n)} ··· (b) -2

C. Direction average: a feature amount corresponding to the gradient of the line segment indicated by the EC label set, and based on the magnitude relationship between the direction sums SCX and SCY, one of the following equations (c) -1 to (c) -5: Required by When SCX> 0 and SCY ≧ 0, EC = atan (SCY / SCX) (c) -1 When SCX> 0 and SCY <0, EC = 360 + atan (SCY / SCX) (c) -2 When SCX <0 EC = 180 + atan (SCY / SCX) ... (c) -3 When SCX = 0 and SCY> 0 EC = 0 ... (c) -4 When SCX = 0 and SCY <0 EC = 180 (c) -5

D. Straight line equation This shows a straight line P (shown in FIG. 7) passing through the line segment indicated by the label set, and is expressed by the following equation (d) using the direction sums SCX and SCY and the density centroids GX and GY. Is done. SCY.x + SCX.y- (SCY.GX + SCX.GY) = 0 (d)

E. End point coordinates: (x1, y1) (x2,
y2) The coordinates of each end point of the line segment are calculated. In this embodiment, the following calculation is executed in order to obtain the coordinates of the end point located on the straight line P in consideration of the dispersion of the measurement processing. ing. First, the maximum value x _max , y _max , and the minimum value x are respectively selected from the x coordinate and the y coordinate of the edge set to be processed.
_min and y _min are extracted, and the coordinates (x1 ′, y1 ′) (x2 ′, y2 ′) of the temporary end point of the line segment are set based on these coordinates. Incidentally, x-coordinate of each temporary end point, x1' = x _min,
x2' = x _max next, y coordinates, SCY · SCX ≧ 0
The y1' = y _max y2' = y _min when the _{y1' = y min y2' = y max} SCY · SCX <0 is when.

In this way, the provisional end points (x1 ', y1
′) And (x2 ′, y2 ′), the coordinates of the perpendicular leg drawn down from each temporary end point to a straight line are obtained using these coordinates and the equation of the straight line in (d) above, and the coordinates are obtained. The coordinates of the end point are (x1, y1) (x2, y2).

F. The length L of the line segment is equivalent to the distance between the end points of the line segment, and is obtained by applying the coordinates of each end point to the formula for calculating the distance.

FIG. 7 shows, among the characteristic quantities obtained by the above-described processings A to F, the density center of gravity (GX, GY), the straight line P passing through the line segment, the length L of the line segment, and the end point (x1 , Y1)
(X2, y2). In the next ST8 and ST9, arithmetic processing using these characteristic amounts and linear equations is executed.

In ST8, it is determined whether or not there is a set of line segments that can be integrated as one line segment, using the feature amount of each line segment. When a set of line segments that can be integrated is found, a new label set in which edge pixels constituting these line segments are put together is formed, and a new label is set for the set. And for this new set of labels,
The feature amounts and straight line equations shown in the above A to F are obtained.
Each label set before integration and its feature amount are also maintained without being deleted.

FIG. 8 shows a specific procedure for determining whether or not the line segments can be integrated, and FIG. 9 shows the concept of parameters used in this determination processing. Note that FIG.
Among them, A and B are line segments of interest, P1 and P2 are straight lines passing through these line segments A and B, and C is a line segment after line segments A and B are integrated. In this illustrated example, in order to clearly show the parameters, the difference between the inclinations of the line segments A and B and the amount of positional deviation between the line segments are shown large.

The procedure shown in FIG. 8 is executed for all combinations of line segments in the measurement area (here, a combination of two line segments). First, in ST8-1, gradients (the above-described direction average E
The difference δ in C) is obtained, and it is determined whether or not this δ is equal to or less than a predetermined threshold.

In the next ST8-2, the distance d between the end points (points a2 and b1 in FIG. 9) of the line segments A and B is calculated, and it is determined whether or not this distance is equal to or smaller than a predetermined threshold value. .

In ST8-3, the length L of the segment C after integration
3 and this length is the sum of the lengths of the line segments A and B (L1 +
L2) It is determined whether it is not less than or equal to. Note that the length L3 of the line segments after integration is calculated as the distance between the end points of the line segments A and B that do not face each other (end points a1 and b2 in FIG. 9).

In ST8-4, the amount of positional deviation between the line segments is obtained, and it is determined whether or not the amount of positional deviation is equal to or less than a predetermined threshold value. Note that this positional deviation amount is represented as a deviation amount between the facing end points, and in this embodiment,
The length of a perpendicular line drawn from an end point a1 of the other line segment (in the illustrated example, line A) on the line segment B side with respect to a straight line (in the illustrated example, P2) passing through one of the line segments (in the illustrated example, line segment B). H.

The determination processes in ST8-1 to ST8-4 are sequentially performed. If all the determination results are "YES", the process proceeds to S8.
Proceeding to T8-5, it is determined that the line segments A and B can be integrated. On the other hand, if "NO" is determined in any of the determination processes in ST8-1 to 8-4, the process proceeds to ST8-6, and it is determined that the integration of A and B is impossible.

Note that among the threshold values used in each determination process,
The threshold value for the distance d between the end points may be set to a relatively large value, but it is desirable that the threshold value for the difference δ in the direction average and the displacement amount h between the line segments be as small as possible. Difference δ in direction average and displacement amount h between line segments
The threshold value should be set to a small value because, when one straight line portion of the outline of the object is divided and a plurality of line segments are generated, the difference in slope between these line segments and the line segment This is because the amount of positional deviation of should be small. On the other hand, when a plurality of line segments are extracted by dividing one straight line portion of the contour of the target object by a large defect, the value of the distance d between the opposite end points of each line segment may increase. The tolerance for the distance d between the end points can be increased to some extent.

FIG. 10 illustrates the difference between the case where the line segments are integrated and the case where the line segments are not integrated. In (1) and (2) in the figure, two line segments having no difference in inclination are shown. A and B are shown. In FIG. 10A, each line segment A, B
Is large, but the displacement h between the line segments A and B is within the width range of the line segments.
It is determined that the line segments A and B can be integrated. On the other hand, in the example of FIG. 10B, the distance d between the end points a and b is
Although smaller than (1), the displacement amount h exceeds the width range of the line segment, and it is determined that the line segments A and B cannot be integrated.

According to the straight line integration processing described above, FIG.
As shown in (2), when there are a plurality of sets of line segments that can be integrated (two sets of straight lines A and B and straight lines B and D in the illustrated example),
After the integration process is performed for each set to set new line segments C and E, these line segments C and E are further integrated to make each line segment 1
It is also possible to integrate into a line segment F. It is preferable that individual labels are set for the integrated line segments C, E, and F, and the labels of the line segments A, B, and D before the integration are also maintained as they are.

Returning to FIG. 2, each line segment present in the measurement area is individually extracted as described above, and the process of integrating the segments that can be integrated is completed. A line segment satisfying the extraction condition is extracted. The extraction condition may be a condition for extracting one line segment, such as the length and inclination of the line segment. However, in order to measure the position and direction of the object, a plurality of line segments are extracted. It is desirable to set the conditions for doing so.

FIG. 12 shows an extraction method in a case where the straight line extraction condition is set so as to extract a line segment forming two adjacent sides of the outline of the object. In this case as well, similarly to the integration process, a set of line segments in the measurement area is sequentially searched, and the lengths L1, L of the line segments A, B to be processed are determined for each set.
It is determined whether or not 2 is a length corresponding to the size of the object. When the angle formed by each line segment is designated as the extraction condition, the angle α formed by each line is obtained from the equation of the straight lines P1 and P2 passing through each line segment A and B, and this α is compared with the extraction condition. .

FIG. 13 shows a line segment extraction method when the extraction condition is set to extract a set of parallel line segments. In a specific extraction process in this case, a difference in inclination between line segments is obtained for each combination of line segments in the measurement region, and a combination in which the difference takes a value close to zero is extracted. Further, when the distance between the line segments is also included in the extraction condition, the distance between the line segments is obtained from the equation of the straight lines P1 and P2 passing through the line segments A and B recognized as parallel, and is compared with the condition. FIG.
Similarly to the example, the length of each line segment can be used as the extraction condition. In the method of FIG. 13, if the distance between the line segments to be extracted is set to be extremely small, an object having a minute width such as a straight line flaw generated on the package can be extracted.

FIG. 14 shows a line segment extraction method when the extraction condition is set to extract a line segment constituting a rectangle of a predetermined size. In this case, first, FIG.
As shown in (1), two sets of parallel line segments (line segments A and B, line segments C and D in the illustrated example) corresponding to opposing sides of the rectangle are extracted. Note that the extraction of each line segment set is as shown in FIG.
3 is performed by the same method as in FIG.
G is a line segment that did not satisfy the condition as a parallel line segment. Further, after confirming that the difference in the inclination (direction average) of the line segment between the extracted sets is close to 90 degrees, as shown in FIG. g
4 (using the coordinates (GX, GY) of the density center of gravity) and a straight line m connecting the centers of gravity of each set of parallel line segments.
1 and m2 are set. At this time, if the intersection point M of each of the straight lines m1 and m2 is within the region r, it is determined that each of the line segments θ1 to θ4 is a line constituting a rectangle.

When a line segment matching the extraction condition is found in this way, in ST10, a processing result image for identifying the extracted line segment by, for example, a predetermined color or luminance is generated. The generated processing result image is provided to the monitor 9 and displayed. As shown in FIG. 12, when an intersection (cx, cy) of a line segment or a straight line is obtained from an equation of a straight line passing through each extracted line segment, a processing result image in which the position of the intersection is indicated by a predetermined mark is obtained. Can also be generated and displayed.

According to the above-described embodiment, since the measurement is performed using the edge pixels on the grayscale image, even if the brightness of the image changes due to the fluctuation of the illumination or the like, the measurement result is hardly disturbed and the stable measurement is performed. It can be performed. If the size or direction of the target object may change, the extraction conditions for the length and inclination of the line segment may be set gently, or other extraction conditions may be set to change the target line segment. Since extraction can be performed, highly accurate measurement processing can be performed easily and at high speed.

In this embodiment, the line segments constituting the outline of the object are extracted from the measurement area, and the extracted result is displayed. However, the present invention is not limited to this. The position and orientation of the object may be measured. For example, when an object having a rectangular outline as shown in FIG. 14 is to be measured, an image area corresponding to the object is specified using an equation of a straight line passing through the extracted line segments A to D. Later, the position of the center of gravity in the area is obtained, and the value can be specified as the position of the object. The direction of the object can be represented by the inclination of one of the line segments (for example, the longer line segment A, B).

Further, according to the line segment integration judgment processing shown in FIGS. 8 and 9, a contour line divided by a defect can be extracted as an object to be integrated. It is also possible to apply to the purpose of extraction. If you just need to determine the presence or absence of a defect,
The proper number of extracted line segments in the measurement area is obtained in advance by performing measurement using a good product model, and whether the number of extracted line segments obtained from the image to be inspected is the proper number of extracted lines is determined. Check it out.

In the above-described embodiment, the edge code histogram is created on the assumption that the object is rotationally displaced, and the peak corresponding to the line segment is extracted. If the image can be picked up in a state in which the image is captured, the direction of the contour of the object on the grayscale image is input from the I / O port 10 in advance, and the direction corresponding to the direction of the outline of the object is used as the edge code. By labeling the edge pixels, a line segment corresponding to the direction of the contour can be extracted. Further, the configuration may be such that measurement processing after creating an edge code histogram and measurement processing without creating an edge code histogram can be switched as appropriate.

[0093]

According to the present invention, attention is paid to the fact that the direction of the edge pixels can be obtained with a small error because the magnitude of the density gradient is relatively large in the edge pixels constituting the contour line. A line segment is represented by a set of edge pixels extracted on the condition that the labels assigned to the edge pixels corresponding to the direction are common and that the edge pixels are continuous, so the straight line of the contour of the object The part can be accurately extracted. Also,
Irrespective of the state of the grayscale image, the direction of the edge pixels can be obtained stably even when the density difference between the background part and the object part is small or when the illumination is not uniform. The straight line portion of the outline of the object can be accurately extracted.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a procedure of image processing.

FIG. 3 is an explanatory diagram showing a mask used for edge extraction processing.

FIG. 4 is an explanatory diagram showing how to represent an edge code.

FIG. 5 is an explanatory diagram showing a grayscale image of an object and an edge code histogram obtained by processing an edge extraction result of the image;

FIG. 6 is an explanatory diagram showing the results of a temporary labeling process and a main labeling process for the image of FIG. 5;

FIG. 7 is an explanatory diagram illustrating a concept of a feature amount of a line segment.

FIG. 8 is a flowchart illustrating a processing procedure regarding line segment integration determination;

FIG. 9 is a diagram illustrating parameters used for line segment integration determination.

FIG. 10 is an explanatory diagram illustrating an example in which it is determined that integration of line segments is possible and an example in which it is determined that integration is not possible.

FIG. 11 is an explanatory diagram showing an example in which a plurality of line segments are integrated.

FIG. 12 is an explanatory diagram showing a specific example of a line segment extraction process.

FIG. 13 is an explanatory diagram showing a specific example of a line segment extraction process.

FIG. 14 is an explanatory diagram showing a specific example of a line segment extraction process.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Image input part 4 Image output part 7 Control part 9 Monitor 11 CPU

Claims (24)

[Claims] 1. A method for processing a grayscale image, comprising: when an outline of an object image represented in the grayscale image includes a straight line portion, a label setting for setting an individual label for each direction of the straight line. And a pixel label that assigns the label corresponding to the coincident direction to an edge pixel included in the grayscale image and whose edge pixel direction coincides with one of the straight line directions. And a line segment extracting step of sequentially extracting, as a line segment, a set of edge pixels to which the same label is assigned and which can be regarded as being continuous on the grayscale image. . 2. The label setting step includes: a histogram creation step of creating a histogram indicating a degree of the number of edge pixels for each edge pixel direction; and extracting a maximum value on the histogram to correspond to the maximum value. And performing a peak label setting step of setting an individual label for each direction of an edge pixel, thereby setting an individual label for each direction of a straight line included in the contour of the target object image. 2. The image processing according to claim 1, wherein the assigning the label is performed by assigning the label set in the peak label setting step to an edge pixel having a direction of an edge pixel corresponding to each label. Method. 3. The method according to claim 1, wherein a direction shift amount of the line segment in the line segment extracted in the line segment extracting step is within a predetermined range,
3. The method according to claim 2, further comprising, when there is a set of line segments in which the positional deviation amount between the line segments is within a predetermined range, a line segment integrating step of integrating these line segments into one line segment. Image processing method. 4. A line segment selecting step for selecting a line segment satisfying a predetermined condition from the line segments extracted in the line segment extracting step or the line segments integrated in the line segment integrating step. Item 4. The image processing method according to item 2 or 3. 5. A display step of displaying a position of a line segment selected in the line segment selection step or a position of an intersection on an extension line of a plurality of selected line segments so as to be identifiable on the grayscale image. The image processing method according to claim 4. 6. The image processing method according to claim 2, further comprising the step of: determining a presence or absence of a defect in a straight line portion of the outline of the target object by using a line segment extraction state in said line segment extraction step. . 7. The method according to claim 1, wherein, in the defect extracting step, a direction shift amount between the line segments in the line segments extracted in the line segment extracting step is within a predetermined range, and a positional deviation amount between the line segments is within a predetermined range. 7. The image processing method according to claim 6, wherein when there is a set of line segments, it is determined that a defect exists between these line segments. 8. The defect judging step compares the number of line segments extracted in the line segment extracting step with a predetermined reference value, and when the two values are different, a defect is found in a linear portion of the outline of the object. 7. The image processing method according to claim 6, wherein it is determined that the image data exists. 9. The label setting step is to set an individual label for each direction assumed as a direction of a straight line portion in the outline of the object. 2. The image processing method according to claim 1, wherein the label corresponding to the coincident direction is assigned to an edge pixel that coincides with any of the assumed directions. 10. A set of line segments in which the direction shift amount between the line segments is within a predetermined range and the position shift amount between the line segments is within a predetermined range among the line segments extracted in the line segment extraction step. The image processing method according to claim 9, further comprising, when there is a line segment, further executing a line segment integrating step of integrating these line segments into one line segment. 11. A line segment selecting step of selecting a line segment satisfying a predetermined condition from the line segments extracted in the line segment extracting step or the line segments integrated in the line segment integrating step. Item 10. The image processing method according to Item 9 or 10. 12. The image processing method according to claim 9, further comprising the step of: determining a presence or absence of a defect in a straight line portion of the outline of the object by using a line segment extraction state in the line segment extraction step. . 13. A means for inputting a grayscale image, a means for extracting edge pixels included in the grayscale image, a means for determining a direction of an edge pixel in each of the edge pixels, and an object represented in the grayscale image Direction setting means for setting the direction of a straight line portion of the contour of the image; label setting means for setting an individual label for each direction of the straight line; such that the direction of the edge pixel matches one of the directions of the straight line Pixel labeling means for assigning the label corresponding to the coincident direction to the edge pixel; and the same label is assigned to the edge pixel, and the edge pixel which can be regarded as being continuous on the grayscale image. An image processing apparatus comprising: a line segment extracting unit that extracts a set as a line segment. 14. The direction setting means, comprising: a histogram creating means for creating a histogram indicating the degree of the number of edge pixels for each edge pixel direction; and extracting a maximum value on the histogram to correspond to the maximum value. Peak direction setting means for setting the direction of an edge pixel as a straight line direction, wherein the label setting means sets an individual label for each direction set by the peak direction setting means, The labeling means allocates the label set by the label setting means to an edge pixel having a direction of an edge pixel corresponding to each label.
3. The image processing device according to 3. 15. A set of line segments extracted by the line segment extraction means, wherein the direction shift amount between the line segments is within a predetermined range, and the position shift amount between the line segments is within a predetermined range. 15. The image processing apparatus according to claim 14, further comprising a line segment integrating unit that integrates these line segments into one line segment when there is. 16. A line selection means for selecting a line segment satisfying a predetermined condition from the line segments extracted by the line segment extraction unit or the line segments integrated by the line segment integration unit. The image processing device according to claim 14, wherein the image processing device comprises: 17. A display control unit for displaying a position of a line segment selected by the line segment selection unit or a position of an intersection on an extended line of a plurality of selected line segments in an identifiable manner on the grayscale image. 17. The image processing apparatus according to claim 16, further comprising: 18. The apparatus according to claim 13, further comprising a defect discriminating means for discriminating the presence or absence of a defect in a straight line portion of the contour of the object using the extracted state of the line segment extracted by said line segment extracting means. Image processing device. 19. The defect discriminating means, wherein, among the line segments extracted by the line segment extracting means, a direction deviation amount between the line segments is within a predetermined range, and a positional deviation amount between the line segments is within a predetermined range. 19. The image processing apparatus according to claim 18, wherein when there is a set of line segments, it is determined that a defect exists between these line segments. 20. The defect discriminating means compares the number of line segments extracted by the line segment extracting means with a predetermined reference value, and when the two values are different, a defect is found in a straight line portion of the outline of the object. 19. The image processing apparatus according to claim 18, wherein the image processing apparatus determines that the image is present. 21. The direction setting means is for inputting an assumed direction of a straight line portion in the contour of the object, and the label setting means is an individual label for each direction input by the direction setting means. The pixel labeling means, for an edge pixel whose edge pixel direction matches one of the directions input by the direction setting means, the pixel labeling means corresponding to the matching direction 2. A method for assigning a label.
3. The image processing device according to 3. 22. A set of line segments in which the direction shift amount between the line segments is within a predetermined range and the position shift amount between the line segments is within a predetermined range among the line segments extracted by the line segment extracting means. 22. The image processing apparatus according to claim 21, further comprising a line segment integrating means for integrating these line segments into one line segment when there is a line segment. 23. The apparatus according to claim 23, further comprising a line segment selecting unit that selects a line segment satisfying a predetermined condition from the line segments extracted by the line segment extracting unit or the line segments integrated by the line segment integrating unit. 23. The image processing apparatus according to claim 21, wherein 24. The apparatus according to claim 21, further comprising a defect discriminating means for discriminating the presence or absence of a defect in a straight line portion of the contour of the object using the extracted state of the line segment extracted by said line segment extracting means. Image processing device.