JPH032983A - Picture processing method - Google Patents

Abstract

PURPOSE:To attain erasion and automatic restoration with reflection of technical knowledge by calculating the area of a part to be erased and/or restored and a ratio between the area and its peripheral length and deciding whether the relevant part is unnecessary or not based on a function decided previously in response to those calculated area and ratio. CONSTITUTION:When a picture obtained by erasing and/or missing the unnecessary parts out of a produced binary picture is restored, the area of an unnecessary part and the ratio between the area of the part and its peripheral length are calculated. Then a procedure is set to decide whether the relevant part is unnecessary or not based on a function decided previously in response to those calculated area and ratio. Thus it is possible to erase and restore the missing parts and the unnecessary parts in terms of noises out of a binary picture based on the technical knowledge with no intervention of experts.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image processing method, and particularly to an image processing method suitable for cases where the boundaries of an image are unclear.

[Conventional technology]

In recent years, as computers have become more sophisticated, image processing devices have been increasingly used for shape measurement. For example, image processing devices are used to measure the crystal grain size of metal materials and the grain size of powder. Also, JP-A No. 63-228062
The publication discloses a technology that measures the crystal grain shape of metal material grains and predicts creep life from the amount of change in shape.
In this case as well, an image processing device is used to measure the shapes of many crystal grains and perform statistical processing.

Generally, in image processing, an analog signal of an object is input through a camera, and the analog signal is converted into a digital signal by an A/D converter. That is, an image is divided into hundreds by hundreds of pixels, and the coordinates of each pixel and color information for analog signals are captured as digital signals. This digital signal is processed by a computer to perform shape measurement, morphological classification, etc. In shape measurement, an object to be measured is expressed by a digital signal having a value corresponding to the density of a color, and this digital signal is classified into two types using the density of a certain color as a threshold, and an image of the object is specified. This is called binarization, and when this binarization is performed, various geometric quantities such as length and area are calculated by computer from the coordinate data of the image.

As long as it can be binarized in this way, it is easy to measure the shape, but in reality, in camera images of crystal grains and powder, the boundary lines are often unclear or broken (missing). . Furthermore, unnecessary images may be generated due to the inclusion of noise-like data. Therefore, when simply binarizing, adjacent objects are connected, and two objects may be determined to be one. For this reason, it is necessary to correct the image, but there are two ways to do this: write in the missing boundaries with a light pen or digitizer, or the ``automatic boundary valve method using image processing'' invented by the inventor. In the latter method, candidate points are selected based on the curvature of the contour coordinates of objects connected when they are binarized, and whether or not the connection is appropriate is determined based on the distance and positional relationship between the two candidate points.

[Problem to be solved by the invention]

Among the above-mentioned conventional techniques, the method of writing with a light pen, digitizer, etc. requires specialized knowledge on the part of the operator, which places a heavy burden on the operator.

Since the automatic boundary separation method automates the processing, most of the burden on the operator is eliminated, but it is mechanically automated using numerical calculations, does not fully reflect professional knowledge, and is not accurate enough. The problem of the present invention is to automatically detect missing parts in a binarized image and unnecessary parts of a noisy image using the knowledge of an expert, without requiring manual labor with specialized knowledge. repair based on,
It's in Erase.

[Means to solve the problem]

The above-mentioned problem arises in an image processing method for erasing unnecessary parts and/or restoring missing images from an image created by binarization. , and determines whether the portion is an unnecessary portion based on a predetermined function corresponding to the area and the ratio.

The above problem also requires an image processing method for erasing unnecessary parts and/or restoring missing images from an image created by binarizing. The angle formed by the line and the traveling direction of one end of the missing portion, the area within the closed curve formed by connecting both ends, and the missing portion within the closed curve formed when both ends are not connected. The ratio of the major axis to the minor axis of the area including the missing part is calculated, and the distance between the two ends of the missing part, the angle formed by the line connecting the two ends and the direction of movement of one end of the missing part, and the connection between the two ends. and the ratio of the major axis to the minor axis of the area including the missing part in the closed curve formed with the two ends not connected, respectively. Based on the function
This can also be achieved by providing a procedure for determining whether or not the missing portion needs to be repaired.

The above problem is further solved by performing a binarization operation by changing the binarization threshold between both ends of the missing part, and calculating the ratio of the number of pixels between the ends and the number of pixels forming the image part. This is also achieved by the image processing method according to claim 2, characterized in that a predetermined function corresponding to the ratio is taken into consideration in determining whether or not the missing portion needs to be repaired.

Furthermore, at least two of the predetermined functions are
Claims 1 and 2 characterized in that it is a membership function.
Alternatively, the above problem can also be achieved by the image processing method described in 3.

[Effect]

There are commonalities in the shapes of images of objects depending on the object. Based on the area of the image part and the perimeter/area of the image part, the presence or absence of the commonality is determined by a function expressing expert knowledge, and it is automatically determined whether to delete it as an unnecessary part. It is processed.

Whether the middle part of the binarized image is missing is determined by the relationship between the length of the part, the area of the closed curve formed when the parts are connected, and the commonality of the shape. Experts on matters such as whether the shape of the connected portion is natural when both ends of the intermediate portion are connected, and whether the shape is unnatural when the intermediate portion is not connected. Since the knowledge is determined by expressed predetermined functions, the image processing implementation is not performed by each individual knowledge, but is processed automatically.

〔Example〕

FIG. 8 shows the flow of image processing for measuring the shape of a fully curved crystal grain having boundary discrimination ability, to which the present invention is applied.
The figures each show the hardware configuration of an image processing device for implementing the flow. The image processing apparatus shown in the figure includes a microscope 101 that generates an optical image of an object, a camera 102 that is attached to the microscope 101 and converts the optical image into an analog signal, and is connected to the camera 102.

An A/D converter 103 that converts the analog signal into a digital signal, a common line 116 connected to the A/D converter 103, and a G P
-I B (General Purpose I n
t.

-erface Bus) 112, and the host computer 1
Fuzzy controllers 11 connected to 13 and 13 respectively.
4 and a digitizer 115, a control processor 104, a CPU memory 105, an image controller 106, a measurement ROM 107 connected to the common line 116, an image processor 108 connected to the image controller 106, and an image memory A10.
9A, image memory 8109B, image memory Cl
09C and image memory D109D, D/A converters 110A to 110D connected to the image memories A to D via the common line 116, and the D/A
512X4 connected to converter 110A~11oD
It has an 80-pixel color display 111. The common line 116 connects the image processor 10
8 is also connected. Each pixel of the color display 111 has 256 colors each of RGB (red, green, blue).
It is possible to display gradations divided into stages, and color information is determined by the combination of these RGB,
The data is stored as a digital value in the image memory A-
It is stored in D.

In the image processing apparatus having the above configuration, the object of image processing is performed by the camera 102 as an input image (optical image) as shown schematically in FIG.
The optical image is converted into an electrical analog signal by the camera 102. The image signal output as an analog signal from the camera 102 is sent to the A/D converter 10.
3, each of the three RGB colors is converted into a digital signal of 256 gradations, stored in the image memories A to D, and then processed by the host computer 113.

First, the binarization process will be described. One of the RGBa colors is selected, and a certain range of 256 gradations is defined as 11111, and the other range is defined as It OIt. Here, we will explain the case of measuring the shape of ferrite grains (area surrounded by lines) in the metal structure shown in Fig. 10 as an example. As mentioned above, one color of RGB, R in this case, is selected, and the range of gradations O to 130, which is the dark part of the 256 gradations, is set as Pa1'''. In this operation, R information is retrieved from the image memory, a command is sent from the host computer 113 to the CPU to identify pixels corresponding to 130 or more of R, and data of R pixels of 130 or less is :128, G=O1B=O,
The data of other pixels are replaced with R=0, G=O1B=O. 6R=0, G=O1B=O indicates that the data is off, and when these data are binarized, '
``It becomes OII.

The replaced data is then transferred to the D/A converter.
It is displayed on the color display 111 via 0A to 110D. After that, only the image memory of R is used, and only the information of the R:128 part and the R=0 part is processed. Figure 11 shows that the image in Figure 10 is 2
This is an example where the grain boundary (black line part) is interrupted, or an independent black part (black line part) within the crystal grain.
Contains intragranular carbides and dirt).

Connecting these broken portions (missing portions, hereinafter the same) of grain boundaries and erasing the independent black portions, which are unnecessary portions, are performed by converting specialized knowledge into membership functions and using fuzzy inference. Figure 12 shows that by fuzzy inference,
This is a diagram in which unnecessary portions have been deleted from the image shown in FIG. 11 and missing grain boundaries have been connected. 1st
Once the grain boundaries are clarified as shown in Figure 2, then II
I The II part (black part in the figure) is inverted to '0'', the "0" part (white part in the figure) is inverted to 11117, and the 13th
Crystal grains are displayed as shown in the figure. The crystal grain is the first
When displayed as shown in Figure 3, the coordinate data of that pixel is processed by the host computer, and the area, horizontal and vertical axis projected diameter, maximum diameter, etc. of the object (crystal grain) are calculated. It is possible to measure the shape of large diameters, such as direction angle and circumference.

Next, an embodiment of a grain boundary connection (missing portion) determination method to which the present invention is applied will be described. This method involves a host computer's calculations based on information on the image memory, and
FIG. 1 is a flowchart illustrating the steps of the method. First, isolated points due to intragranular carbides and dirt are erased from the binary image as unnecessary parts.

This erasure is performed based on the area and perimeter of the portion of the binary image whose value is 111 II (the portion expressed in black in FIG. 11). In the case of an isolated point, the area is relatively small, and the ratio of perimeter/area is smaller than in the case of a part of a grain boundary.

Therefore, as shown in Figure 2, the vertical axis shows the confidence level (the degree to which it is believed that it is an isolated point), and the horizontal axis shows the area and perimeter/area (actually, to make it dimensionless (perimeter )2
/(area・4π), and in the case of 1 yen, the minimum value is 1
), the membership function is set. This membership function is based on the expert's knowledge that an isolated point has a somewhat small area and (perimeter) 2/(area·4?c) is small (the area is not elongated).

This membership function is called the antecedent part (left half of FIG. 2). In addition to the antecedent part, a membership function in which the probability of being an isolated point is taken on the horizontal axis and the certainty is taken on the vertical axis is set as the consequent part (right half of FIG. 2). Measured value of area (black area in Figure 11)
(value calculated by computer) is Sl, (perimeter)2/(
If the calculated value of area・4π) is R1, then in the antecedent part, S,
, R, respectively. In addition to the certainty obtained for the antecedent part, m is a line a indicating the certainty of the consequent part.
, b are respectively multiplied, and lines a and b are changed to line ab'. In other words, the confidence value of the consequent is reduced to the value multiplied by m as a whole, and the shape (trapezoid) of the hatched part shown in the upper right part of FIG. 3 is determined. Then, the two trapezoids are
As shown in the lower right diagram of FIG. 3, a side with a confidence level of 0 and a side with a probability of being an isolated point of -1 are matched and overlapped, and the center of gravity G is obtained by combining the centers of gravity of both shapes. Based on the determined position of the center of gravity G, the confidence and probability of being an isolated point corresponding to the position of the center of gravity G are determined on FIG. 3c. In this embodiment, it is appropriate to determine that a region is an isolated point when the probability of the region being an isolated point is 0.5 or more and the confidence is 0.2 or more.

Next, locations where grain boundaries are interrupted, which are expressed in black on the binary image, are searched for and connected.

As mentioned above, it is assumed that the image is composed of 512×480 pixels, and a portion of the image has the shape shown in FIG. Each square frame in the figure represents one pixel, and the pixel surrounded by a black frame and with a number written in it is the 11th pixel.
This is the area expressed in black in the figure. Starting from an arbitrary pixel 1 in this area (here, the pixel on which the number 1 is written), surrounding pixels in the area are searched.

There are a maximum of eight pixels adjacent to pixel 1, but in Figure 4, the lower left of pixel 1 is not included in the area, so there are seven pixels 2 adjacent to pixel 1. Pixels adjacent to pixel 1 and pixel 2 that are not pixel 1 and pixel 2 are searched, and these become a group of pixels 3. When this search operation is repeated and the next pixel adjacent to a certain pixel is not found, the certain pixel (pixel 12 in FIG. 4) becomes a broken point (one end of the missing part). Note that with this method, only one broken point can be searched at one time, so the search is repeated several times, excluding the first pixel.

After the broken points are searched, a determination of connectivity is made using fuzzy inference. Used to determine connection.

The information obtained is the distance expressed in number of pixels between two arbitrary points that form both ends of the missing part among the broken points searched earlier, and the angle formed by the direction in which each of the two arbitrary points is moving. , the area of the part surrounded by the closed curve formed when the two points are connected, and the aspect ratio of the part surrounded by the closed curve (the closed curve already formed in the binarized state) when they are not connected. be. The direction in which the broken point is moving is
As a result of various studies, we found that the pixel closest to the broken point (in Fig. 4) is selected from among the pixel group with the number 5 before the final number (for example, 12 in Fig. 4) when the relevant point was searched. ■)
It is appropriate to define it as the direction of the line connecting the pixel at the interrupted point (12 in FIG. 4), and it was defined as such.

5A, with the four pieces of information on the horizontal axis and the confidence level (the degree of confidence that the two points are correctly connected) on the vertical axis.
Four membership functions shown in the left half of FIGS. 5D to 5D are set, and these functions form the antecedent part. Furthermore, the above 4
A total of four membership functions are set, with the vertical axis representing the confidence and the horizontal axis representing the probability of connection between the two points, and these form the consequent. .

The four membership functions of the antecedent part will be explained below.

In order to recognize a connection between two broken points, the distance between the two points must be small to a certain extent, and this length condition makes one variable the length between the two points expressed in the number of pixels. is set as the first membership function. In the graph in the left half of FIG. 5A, lengths of 100 or more are considered to have a confidence level of 0. In addition, the directions in which the two points at the broken points are moving must be almost the same, and the angle formed by this direction and the straight line connecting the two points (angle θ in Figure 6) is required.
) is small is set as the second membership function. In addition, in FIG. 5B, the angle θ is 45
If the value is greater than or equal to 100 degrees, the confidence level is 0. Next, if the area of the part surrounded by the closed curve formed when two points are connected is extremely small, it cannot be considered a crystal grain, so it is surrounded by the closed curve formed when the two points are connected. A third membership function is set that indicates the condition that the area is not extremely small.

The area referred to here is the area of the white part in Figure 11, which is 1" (R: 128.G: 0, B: O
) “0” surrounded by pixels (R:O.

It is expressed by the number of pixels (G:O, B:O). Finally, regarding the validity of the shape of the closed curve when the two points of interest are not connected, based on the knowledge of experts that crystal grains are not extremely elongated, the shape of the closed curve when the two points are not connected is determined. If it is extremely long and narrow, there is a high probability that the two points will be connected. Therefore, the maximum length of the part surrounded by the closed curve when not connected (hereinafter referred to as 'major axis') and the maximum length of the part surrounded by the closed curve in the direction perpendicular to the direction of the maximum length (hereinafter referred to as 'major axis') , short axis) is calculated, and a fourth membership function is set that indicates whether it is appropriate to connect based on the ratio.

The graphs in the left half of each of FIGS. 5A to 5D represent the four antecedent membership functions. First, based on the binary image shown in FIG. is calculated as 0, then the confidence of the antecedent according to the respective membership function.

The shape of the graph showing the certainty of the corresponding consequent is changed based on each calculated certainty.0 For example, in the case of Figure 5A, the certainty calculated for the antecedent is changed. If p, the value of the graph e indicating the certainty of the consequent part is multiplied by the P, resulting in a graph e'. Similar operations are performed for other membership functions, and the resulting graphs e'f/, gl, and hl are superimposed as shown in FIG. 5E. The centroids of the respective superimposed figures are synthesized to calculate the centroid G of the entire superimposed figure, and from the position of the centroid G, the synthesized certainty factor Qa and probability of connection Pa are calculated. Based on the calculations of individuals and the knowledge of experts, if the probability of a synthesized connection is 0.6 or more and the confidence is 0.2 or more, it is correct that the two points of interest are connected. FIG. 12 schematically shows the results of eliminating isolated points within crystal grains and connecting broken portions of grain boundaries using the above-described procedure for the binary image shown in FIG. 11. ing.

Shape measurement Jll is performed on the image shown in FIG. The value f# (OII part) is the target, and due to data handling, the R:O part is 111 II, and the R128 part is I
IOII and the image is inverted in the image state shown in FIG. 13.

In the above embodiment, the binarization of the image is performed using the gradation data R
: Pixels of 130 or less were set as "1", but in the binarized image (Figure 11), the discontinuous part of the image before binarization shows that the image data value is R:130. There may be cases where the pixels are connected by more than one pixel. Taking this into consideration, a membership function may be added that provides a condition that the line between the two points to be determined is thin in the image before binarization when determining the connection. For the pixels between two broken points in the subsequent image, when the threshold value for binarization is shifted by 20 to the side where the gradation becomes lighter, the percentage of pixels that becomes 11111 is It is a membership function in which the confidence level approaches 1 as the ratio increases. Figure 7 shows the antecedent and consequent parts of this membership function, which are calculated in the same way in addition to the membership functions shown in Figures 5A to 5D, and added to the center of gravity calculation in Figure 5E. be done. By adding this fifth membership function to the connection determination, the accuracy of the connection determination is further improved.

〔Effect of the invention〕

According to the present invention, it is possible to erase unnecessary parts from a binarized image based on expert knowledge and to automatically repair missing parts. Accuracy is improved and the required man-hours are also reduced.

[Brief explanation of drawings]

FIG. 1 is a flowchart showing an embodiment of the image processing method according to the present invention, and FIGS. 2A and 2B are examples of membership functions used for erasing unnecessary parts of a binarized image according to the present invention. The graphs shown in Figures 3A to 3C are
2A and 2B are graphs showing usage examples of the membership functions, FIG. 4 is an explanatory diagram of the method for searching for missing parts of a binarized image according to the present invention, and FIGS. 5A to 5E are A graph showing the membership function used for repairing the missing part according to the present invention and an example of its use, FIG. 6 is an explanatory diagram of the angle formed by a line connecting two points (both ends of the missing part) and the direction of movement of one end,
Figure 7 is a graph showing other examples of membership functions;
The figure is a flowchart for measuring the shape of metal crystal grains by image processing, Figure 9 is a block diagram showing an example of a hardware configuration for shape measurement by image processing, and Figure 10 is an example of an input image of a metal structure. Schematic diagram, Figure 11 is a schematic diagram of a simple binarized image of the input image in Figure 10, Figure 12 is a schematic diagram of the input image of Figure 1.
Figure 13 is a schematic diagram showing the result of the binarized image shown in Figure 1 being subjected to the process of erasing unnecessary parts and restoring missing parts according to the present invention. FIG.

Claims (1)

[Claims] An image processing method for erasing an unnecessary portion and/or restoring a missing image from an image created by 1. binarization, the area of the portion, and the perimeter and area of the portion. , and determining whether the portion is an unnecessary portion based on a predetermined function corresponding to the area and the ratio. 2. In an image processing method for erasing unnecessary parts and/or restoring missing images from an image created by binarizing, the distance between both ends of the missing part, a line connecting the both ends, and the missing part The angle formed by the traveling direction of one end, the area within the closed curve formed by connecting the two ends, and the major axis of the area including the missing part within the closed curve formed when the ends are not connected. and the ratio of the short axis are calculated, and the distance between both ends of the missing portion is formed by connecting the two ends to the angle formed by the line connecting the two ends and the traveling direction of one end of the missing portion. Based on a predetermined function that corresponds to the area within the closed curve and the ratio of the major axis to the minor axis of the area including the missing part in the closed curve that is formed with both ends not connected. An image processing method characterized in that it is determined whether or not the missing portion needs to be repaired. 3. A binarization operation is performed by changing the binarization threshold between both ends of the missing part, and the ratio of the number of pixels between the two ends and the number of pixels forming the image part is calculated, and the number of pixels corresponding to the ratio is calculated. 3. The image processing method according to claim 2, wherein the predetermined function is taken into consideration in determining whether or not a missing portion needs to be repaired. 4. The image processing method according to claim 1, 2 or 3, wherein at least two of the predetermined functions are membership functions.