JP4993128B2 - Image processing method and image processing apparatus using the method - Google Patents

Description

  The present invention relates to an image processing method for extracting a contour line based on a gray gradation image acquired as digital image data, and in particular, an image processing method suitable for use in image processing of an SEM image and the method thereof The present invention relates to an image processing apparatus using the method.

  Conventionally, image processing for extracting an outline (edge) in an acquired gray-scale image has been performed. As an image processing method for extracting such an outline, a zero crossing between Gaussian and Laplacian is performed. The method by is known.

  For example, Patent Document 1 (Japanese Patent Publication No. 5-44062) describes a source and problems of a method using a Gaussian-Laplacian zero crossing in image processing. In Patent Document 1, “Machiyu Setsutetsu Institute of Technology, Artificial Intelligence Laboratory Memorandum, 518” (“MIT A. I. Lab Lab. Memo 518, 1979”) In the document “Theory of Edge Detection” by D. Marr et al., A function obtained by applying a Laplacian to a two-dimensional Gaussian function as a mask function, It describes a method of performing edge detection by applying a convolution to the input image and detecting the zero point of the output of this convolution, and in this method, noise is increased if the mask function is sufficiently expanded. The effect of the corner can be reduced, but then the corner r It is described that there were problems such as severe deformation of the minute and reduced resolution.

  Patent Document 2 (Japanese Patent Application Laid-Open No. 2007-116402) discloses a data holding circuit that holds image data relating to a plurality of discrete points surrounding a point of interest in order to detect an edge at the point of interest within a two-dimensional area. And a data rearrangement circuit that arranges and outputs image data related to the plurality of discrete points according to a predetermined path, and among the outputs from the data rearrangement circuit, between the discrete points adjacent on the predetermined path A technique is described in which an edge is detected by an edge determination circuit that compares the image data and outputs a signal related to the presence or absence of an edge at the target point based on the comparison result.

Patent Document 3 (Japanese Patent Laid-Open No. 5-167927) describes a method of detecting a peak of a differential value on a cross-sectional profile along a gradient.
Japanese Patent Publication No. 5-44062 JP 2007-116402 A JP-A-5-167927

  Strictly speaking, the widely used edge detection method described in Patent Document 1 and Patent Document 2, that is, a method for extracting a set of points at which a value obtained by applying a Laplacian operator to an image crosses zero is theoretically theorized. In fact, when the edge sharpness is low and the edge is bent, the steepest position of the change of the pixel value is not correctly detected, and it shifts to the outside of the bend. There was a problem.

  The “correct edge position” referred to here is the position of the point where the value of the second derivative zero crosses in the cross-sectional profile perpendicular to the running direction of the contour line. The collection of points at which the value of the bivariate function obtained by applying the Laplacian operator to the image zero-crosses does not necessarily match the “correct” edge position. This is not a problem of rounding error and noise caused by the image being discrete, but a strict theoretical and essential problem. This can be shown in the following example.

As an example of a continuous image z = f (x, y), the following is taken up. Let r ₀ , w ₀ be a constant satisfying 0 <w _0/2 <r ₀ , and let f (x, y) be

Define in. As shown in the upper part of FIG. 1, this image has a brightness peak on the circumference of the radius r ₀ centered on the origin, and is directed to both sides in a belt-like region of width w ₀ centered on the circumference. And the brightness will drop to zero.

  FIG. 1 is a diagram showing an image (upper stage) defined by the equation (1), its X-axis cross-sectional profile (middle stage), and a graph of its second derivative (lower stage). FIG. 2 also shows the graph of the second derivative (lower dotted line), the X-axis cross section (lower dotted line) of the image obtained by applying Laplacian to the image, and the zero crossing position of the two graphs superimposed on the original image. It is a figure which shows the displayed graph (upper stage).

  This function is rotationally symmetric around the origin. Therefore, the cross-sectional profile shapes on an arbitrary straight line passing through the origin all match. Hereinafter, the cross-sectional profile is represented by the one on the X axis. The middle part of FIG. 1 shows a cross-sectional profile. Expression (1) is made so that the value of z draws one cycle of a sinusoidal curve along the cross section. Therefore, the position of the inflection point is clearly

It is. This can be expressed mathematically as follows. The profile on the X-axis of Equation (1) substitutes 0 for y in f (x, y),

As obtained. However, only within a belt-like region having a positive pixel value z is considered. Differentiating this twice with respect to x,

Is obtained. From this it is clear that equation (4) takes the value 0 for the value of x shown in equation (2). The lower part of Fig. 1 shows the graph of equation (4).

  On the other hand, when Laplacian operator is applied to Equation (1)

Is obtained. This cross-sectional profile along the X axis is obtained by substituting 0 for y in Equation (4).

As obtained. Comparing equation (4) and equation (6), it can be seen that the two do not match and equation (4) is obtained by removing the second term of equation (6). The lower part of FIG. 2 is a graph in which the graph of Expression (6) is further superimposed on the lower part of FIG. The upper part of FIG. 2 is a plot of the zero crossing points of the cross-sectional profile and the zero crossing points of the Laplacian.

  From this, it can be read that the zero crossing point of the Laplacian is detected outside the bend rather than the original edge position.

  In addition to the above problem, the problem of the method of detecting zero crossing by the Laplacian operator is apparent from the following example. In other words, when z = f (x, y) is a harmonic function, there is a problem that if a Laplacian operator is applied, it becomes zero everywhere.

  An example of a harmonic function can be made as follows. When w = φ (z) is an arbitrary regular complex function,

The function f (x, y) created by is a harmonic function. However, Re (a) represents the real part of the complex number a. For example,

When,

It is. When a Laplacian operator is applied to this, it can be confirmed that all (x, y) are actually zero.

  Further, Patent Document 3 describes a method for detecting a peak of a differential value on a cross-sectional profile along a gradient. In this method, on a straight line drawn in the gradient direction from one point on an image. At each point, since the direction of the gradient at that point does not always match the direction of the straight line, there has been a problem that a zero crossing in the gradient direction has not been obtained in a strict sense. In addition, there is a problem that calculation for actually creating a profile in an oblique direction of an image is necessary, which is not very efficient.

The present invention is to solve the above-described problems, and the invention according to claim 1 is such that the gray gradation value z at the position (x, y) in the XY plane is z = f (x, y). In the image processing method for detecting the contour line from the image data given by
a set of points at which the value of the second derivative in the gradient direction of z = f (x, y) crosses zero, and the set is detected as an outline of the image ;
The second derivative in the gradient direction of z = f (x, y) is
And obtaining the operator delta _K defined by.

And obtaining the operator delta _K defined by.

The invention according to claim 2 is the image processing method according to claim 1 , wherein each value of z _x , z _y , z _xx , z _xy , z _{yy in} the operator Δ _K is discretely _calculated. The image data defined by z = f (x, y) is obtained by performing a convolution calculation with a filter represented by a matrix having constants as elements.

The invention according to claim 3 is the image processing method according to claim 1 , wherein each value of z _x , z _y , z _xx , z _xy , z _{yy in} the operator Δ _K is an The image is approximated by least squares with a polynomial within a certain range around the position (x, y), and the resulting polynomial is partially differentiated once in the x direction, partially differentiated once in the y direction, and partially differentiated twice in the x direction. It is obtained by substituting the values of (x, y) into polynomials obtained by partial differentiation once in the x and y directions, and partial differentiation twice in the y direction.

The invention according to claim 4 is the image processing method according to claim 1 , wherein the contour line is detected for each pixel constituting the image data in which z = f (x, y) is discretely defined. to calculates the z _x, z _y, the value of at the pixel position (x, y), which calculates the value of z _x ² + z _y ² based on, when it is below a predetermined value, the pixel is determined to belong to the area a, when not, further z _xx, z _xy, calculates the value of z _yy, calculates the value of ^{_{z xx z x 2 + 2z xy}} z x z y + z yy z y 2, When this sign is positive, it is determined that the pixel belongs to the region B. Otherwise, it is determined that the pixel belongs to the region C, and the boundary line between the pixel belonging to the region B and the pixel belonging to the region C is defined as the contour line of the image. It detects as.

The invention according to claim 5 is an image processing apparatus using the image processing method according to any one of claims 1 to 4 .

According to the first aspect of the present invention, by using a method of extracting a set of points at which the value of the second derivative in the gradient direction of the image crosses zero, a value obtained by applying a Laplacian operator to the image at the point at which zero crossing occurs. It can solve the theoretical problem of the method of extracting the gathering, the edge sharpness is low, and even when the edge is bent, the detected edge does not shift to the outside of the bend There is an effect that the correct position of the edge is detected.
In addition, even in the image based on the harmonic function, the problem that the Laplacian becomes zero everywhere can be avoided, and the edge is detected.

Further, the invention according to claim 1, by extracting a collection of points with operator delta _K defined by formula (11), a value obtained by subjecting this to the image crosses zero, actually image The value of the second derivative in the gradient direction of the image can be calculated by a relatively simple calculation without requiring processing for obtaining a cross-sectional profile.

According to the second and third aspects of the invention, when a discrete image is given, the calculation for obtaining the approximate value of Equation (11) can be performed by using a differential value or a secondary differential value of an image that has been conventionally used. A technique for obtaining a value can be suitably used.

Further, according to the invention of claim 4 , there is an effect that division by zero can be avoided in the calculation of Expression (11) and weak edges can be excluded.

Moreover, according to the invention which concerns on Claim 5 , the image processing apparatus which has the above effects can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of the continuous function z = f (x, y) in the XY plane.

  First, a gray gradation image is defined. For m × n discrete points arranged in a grid on the XY plane, each pixel value takes a discrete value within a certain range (for example, an integer value in the range of 0 to 255). Here, the digital data to which is assigned is referred to as a “gray gradation image”. The meaning of the pixel value includes any meaning such as luminance information, black density information, information of one plane of a color image composed of a plurality of planes such as RGB, CMYK, Lab, and Luv. To do. The pixel value at the position (i, j) (i = 0, 1, 2,..., M−1; j = 0, 1, 2,..., N−1) on the XY plane is z Indicated as (i, j).

  In addition, a continuous image is defined. An image takes discrete values for both position coordinate values and pixel values, and this is regarded as a result of sampling what was originally continuous. This continuous function is expressed as z = f (x, y). That is, the gray gradation value z of (x, y) on the XY plane is given by z = f (x, y). It is assumed that f (x, y) is sufficiently smooth and can be differentiated any number of times.

  In the present invention, the contour line is detected from the image data in which the gray gradation value z at the position (x, y) in the XY plane defined as described above is given by z = f (x, y). Then, a set of points at which the value of the second derivative in the gradient direction of z = f (x, y) crosses zero is extracted, and the set is detected as an image outline.

  That is, in the present invention, instead of the conventional method of detecting, as an edge, a set of points at which a value obtained by applying a Laplacian operator to image data z = f (x, y) is zero-crossed, two gradients in the gradient direction of the image are used. A correct edge position is detected by using a method of extracting a set of points at which the value of the second derivative crosses zero.

  According to the present invention as described above, by using a method of extracting a set of points where the values of the second derivative in the gradient direction of the image cross zero, a set of points where the value obtained by applying a Laplacian operator to the image crosses zero. Can solve the theoretical problem of the method of extracting the edge, the edge sharpness is low, and even when the edge is bent, the detected edge does not shift to the outside of the curve, There is an effect that the correct position of the edge is detected.

  The above will be described mathematically. In the present invention,

(11) instead of the Laplacian operator Δ defined by

The correct edge position is detected by extracting a set of points at which the values obtained by applying the operator Δ _K defined in (1) to zero crossing.

  In addition, it can confirm as follows that Formula (11) represents the secondary differential value of a gradient direction.

The continuous image is represented as z = f (x, y). Consider a straight line l passing through an arbitrary point (x ₀ , y ₀ ) on the XY plane and having a direction vector of (cos (θ ₀ ), sin (θ ₀ )). FIG. 4 is a diagram showing a straight line l extending in the θ ₀ direction through the point (x ₀ , y ₀ ) in the XY plane.

  The straight line l uses a parameter t that takes a real value,

It can be expressed as. Consider a cross-sectional profile along a straight line l for a given image z = f (x, y). This can be viewed as a function of t. It

Is written. Specifically, g (t) is obtained by substituting Equation (12) into x and y of z = f (x, y),

It is.

  From here, g (t) is differentiated twice by t.

Is obtained.

By substituting t = 0 to, at the point (x _0, y _0), the secondary differential value g of the cross-section profile along the straight line l '' (0) is,

It is obtained.

Up to this point, the direction of the straight line 1 has been set to an arbitrary angle θ ₀ direction. Here, the direction is defined as the gradient direction at the point (x ₀ , y ₀ ). That is,

And

By the way, this direction vector is not defined when the denominator is zero. The denominator becomes 0 when the gradient vector becomes a zero vector, but at this time, since the gradient is zero at that point, it cannot be an edge. Therefore, at this time, the point (x ₀ , y ₀ ) is excluded from the edge candidates.
The following discussion will proceed only when the denominator is not zero. By substituting equation (17) into equation (16), the point (x _0, y ₀₎ second derivative g '' (0) in the gradient direction in the

It is.

Where (x ₀ , y ₀ ) is replaced with any (x, y)

If it replaces like, Formula (11) will be obtained.

As described above, it was confirmed that Expression (11) represents the second derivative value on the cross-sectional profile in the gradient direction. Here, the operator (operator) delta _K of the formula (11) will be referred to as "Kebayashian".

In the present invention, using the operator Kebayashian delta _K defined by formula (11), by a value subjected to this image to extract a set of points of intersection zero, necessary processing for actually determining the image section profile of Instead, the value of the second derivative in the gradient direction of the image can be calculated by a relatively simple calculation.

  Next, an embodiment suitable for an image processing apparatus that causes a computer to execute the image processing method of the present invention will be described. FIG. 4 is a diagram showing a schematic configuration of the image processing apparatus according to the embodiment of the present invention. In FIG. 4, 9 denotes an SEM device, 10 denotes an image processing device, 11 denotes an image reading unit, 12 denotes an image processing unit, and 13 denotes an image display unit.

  In the present embodiment, the image processing apparatus 10 analyzes an image acquired by an SEM (Scanning Electron Microscope) apparatus 9 and detects an outline (edge). It is not limited to such a thing. The image processing apparatus 10 includes an image reading unit 11 that reads an image acquired by the SEM device 9, an image processing unit 12 that analyzes an image read by the image reading unit 11 and detects an outline (edge), The image display unit 13 displays an outline (edge) detected by the image processing unit 12. Such an image processing apparatus 10 can be configured by a general-purpose information processing apparatus.

  A flow of image processing in the image processing apparatus 10 configured as described above will be described. FIG. 6 is a diagram showing a processing flowchart of the image processing apparatus according to the embodiment of the present invention.

  In FIG. 6, when image processing in the image processing apparatus 10 is started in step S <b> 100, the process proceeds to step S <b> 101, and the image reading unit 11 reads the image acquired by the SEM apparatus 9.

  In step S102, a loop for each pixel in the captured image data is started. This loop is executed for all pixels of the image data.

In step S103, the denominator a in delta _K Z is calculated. That is, the calculation of a ← z _x ² + z _y ² is executed. method will be described later calculation of z _x and z _y.

In step S104, it is determined whether or not the obtained a satisfies a ≦ a _o . a _o is a value very close to 0, and the determination in step S104 is to prevent the numerator of Δ _K Z from being divided by 0 and to detect unnecessary weak edges caused by factors such as noise. This is a step provided to prevent this.

  When the result of determination in step S104 is YES, the process proceeds to step S105, and when the result of determination in step S104 is NO, the process proceeds to step S106.

  In step S105, the pixel is labeled [region A].

In step S106, the numerator b of Δ _K Z is calculated. That is, the calculation of b ← z _xx z _x ² + 2z _xy z _x z _y + z _yy z _y ² is executed. _{z xx, z x, z xy} , z y, will be described later method of calculating the z _yy.

In step S107, it is determined whether b> 0. In other words, it is determined which of the positive and negative values the Kebayayan Δ _K Z takes.

  If the decision result in the step S107 is YES, the process proceeds to a step S108 so as to label the pixel as [Area B].

  If the determination result in step S107 is NO, the process proceeds to step S109, and the pixel is labeled [region C].

  In step S110, it is determined whether or not the loop has been completed for all pixels. If not, the process returns to step S102, and if completed, the process proceeds to step S111.

  In step S111, the boundary between [area B] and [area C] is extracted as an outline (edge).

  In step S112, the image display unit 13 displays the result of the extracted contour line (edge).

  In step S113, the image processing in the image processing apparatus 10 ends.

  According to the image processing in the image processing apparatus 10 described above, there is an effect that division by zero can be avoided and weak edges can be excluded in the calculation of Expression (11).

In the flowchart shown in FIG. 6, in order to calculate the values of a and b, first and second order partial differential values z _x , z _y , z _xx , z _xy , z _yy of the image at each pixel of interest. Need to be calculated.

  The first method for obtaining this is to apply a linear filter (weight coefficient matrix) to the image. By performing a convolution calculation with a specific matrix on image data in which z = f (x, y) is discretely defined, a value corresponding to the partial differentiation in each direction (actually a difference instead of a differentiation) ). That is, if f is the original image data, g is the image data after convolution conversion, and h is a weighting coefficient matrix (convolution kernel),

Thus, the first and second order partial differential values z _x , z _y , z _xx , z _xy , z _yy at the target pixel can be calculated. FIG. 7 is a diagram showing an outline of the convolution calculation in the pixel of interest. 8 and 9 are diagrams showing examples of linear filters (weight coefficient matrix, convolution kernel).

  Since the partial differential value obtained by applying a linear filter (weighting coefficient matrix) to an image is easily affected by the noise of the image, a smoothing filter such as a Gaussian filter is applied to the image in advance. It is common to leave. Also, a DOG method is generally used in which a Gaussian filter and a Laplacian filter are integrated to reduce the amount of calculation.

The second method for calculating the first and second partial differential values z _x , z _y , z _xx , z _xy , and z _yy at the pixel of interest is a continuous method by locally approximating the image with a polynomial. The partial differential value for the polynomial is obtained.

  A neighborhood region of a certain range is provided around each pixel of interest, and within the neighborhood range, the image is approximated by least squares using a two-variable polynomial of x and y. For example, the fourth order polynomial is

It is expressed.

  When using a local coordinate system in which the coordinate origin is restored to the position of the pixel of interest when performing polynomial approximation, the value of the first and second derivatives in each direction at the pixel of interest is f (x, y) as a partial derivative. Thus, the value obtained by substituting (0, 0) is obtained. That means

It is.
By polynomial approximation, in addition to the effect of continuation, there is also an effect of noise removal.

  As described above, when a discrete image is given, the conventionally known first method and second method are suitable for obtaining the values of the first and second partial differentials in the pixel of interest. The first-order and second-order partial differential values can be easily obtained.

FIG. 10 is a diagram comparing an image processing method according to an embodiment of the present invention and a conventional image processing method.
FIG. 10A shows original image data photographed by the SEM apparatus. Figure 10 (b), (c) is obtained by extracting the zero crossing using Kebayashian delta _K of the formula (11) in the original image, FIG. 10 (d), (e) in the original image (10) The zero crossing is extracted by applying Laplacian.

FIG. 10B is a diagram in which areas A to C are labeled according to the processing flow of the image processing apparatus according to the embodiment of the present invention, and FIG. 10C is the image processing according to the embodiment of the present invention. It is the figure which extracted the boundary of the area | region B and the area | region C as an outline (edge) by the processing flow of an apparatus. FIG. 10D is a diagram in which areas A to C are labeled by a conventional image processing apparatus, and FIG. 10E is a contour line that defines the boundary between areas B and C by a conventional image processing apparatus. It is the figure extracted as (edge). In addition, in the calculation for obtaining FIGS. 10B, 10C, _10D , and _10E , primary and secondary partial differential values z _x , z _y , z _xx , z _xy , z _yy are calculated. A polynomial approximation method was used to do this.

As shown in FIG. 10, the corner portion of the edge, in the method of extracting the zero crossing of the conventional Laplacian, is extracted contour line that jumping out toward the zero crossing of the operator by Kebayashian delta _K of the present invention It can be seen that it is solved by the method of extracting and is smooth overall.

  As described above, according to the image processing method and the image processing apparatus using the method according to the embodiment of the present invention, since an edge (contour line) is based on a strictly correct method as a method for detecting an edge of an image. Even when it is bent, it is possible to detect the edge position where the pixel value changes most steeply.

It is a graph relevant to z = f (x, y) shown by Formula (1), and it. It is a graph relevant to z = f (x, y) shown by Formula (1), and it. It is a figure which shows an example of the continuous function z = f (x, y) in an XY plane. The X-Y plane shape at a point (x _0, y ₀₎ is a diagram showing a straight line l extending the street theta ₀ direction. 1 is a diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the process flowchart of the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows the outline | summary of the convolution calculation in a focused pixel. It is a figure which shows the example of the linear filter (a weighting coefficient matrix, a convolution kernel) for obtaining the value of the primary and secondary partial differentiation of each direction. It is a figure which shows the example of the linear filter (a weighting coefficient matrix, a convolution kernel) for obtaining the value of the primary and secondary partial differentiation of each direction. It is a figure which compares the image processing method which concerns on embodiment of this invention, and the conventional image processing method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 9 ... SEM apparatus, 10 ... Image processing apparatus, 11 ... Image reading part, 12 ... Image processing part, 13 ... Image display part

Claims (5)

In an image processing method for detecting a contour line from image data in which a gray gradation value z at a position (x, y) in an XY plane is given by z = f (x, y),
a set of points at which the value of the second derivative in the gradient direction of z = f (x, y) crosses zero, and the set is detected as an outline of the image ;
The second derivative in the gradient direction of z = f (x, y) is
Image processing method characterized by determining by the operator delta _K defined by. Each value of z _x , z _y , z _xx , z _xy , and z _{yy in} the operator Δ _K is a constant element for image data in which z = f (x, y) is discretely defined. The image processing method according to claim 1 , wherein the image processing method is obtained by performing a convolution calculation with a filter represented by a matrix. Each value of z _x , z _y , z _xx , z _xy , z _{yy in} the operator Δ _K is
The image is approximated with a least squares polynomial within a certain range around the position (x, y) on the image, and the resulting polynomial is
partial differential once in the x direction,
partial differential once in the y direction,
partial differentiation twice in the x direction,
partial differentiation once in the x and y directions,
partial differential twice in the y direction,
The image processing method according to claim 1 , wherein the image processing method is obtained by substituting a value of (x, y) for each of the polynomials obtained as described above. When detecting a contour line for each pixel constituting image data in which z = f (x, y) is defined discretely, the values of z _x , z _y , at the pixel position (x, y) are determined. And the value of z _x ² + z _y ² is calculated based on this, and when it is less than or equal to a certain value, it is determined that the pixel belongs to the region A. Otherwise, z _xx , z _xy , z calculates the value of _yy, calculates the value of ^{_{z xx z x 2 + 2z xy}} z x z y + z yy z y 2, when the sign is positive, it is determined that the pixel belongs to the area B, otherwise 2. The image processing method according to claim 1 , wherein the image processing method is characterized in that it is sometimes determined that the pixel belongs to the region C and a boundary line between a pixel belonging to the region B and a pixel belonging to the region C is detected as an image outline. An image processing apparatus using the image processing method according to claim 1 .