JPH0744708A - Shape extraction system - Google Patents

Abstract

PURPOSE:To provide a shape extraction system which sharply and highly precisely extracts an object shape from an unsharp picture by mean of setting plural pieces of binary picture information to be input values for a neural network. CONSTITUTION:The neural network system is provided with the neural network 101 which sets featured values obtained by binarizing an input picture and the processing picture to be input and learns the featured values so as to obtain extracted picture data of the shape and with an output means 102 outputting the various pictures. At least one processing picture is obtained by picture- processing the picture inputted from an image pickup means 103 in a pre- processing means 104. The plural pieces of the processing pictures among the input picture and the processing pictures are binarized in a binarization processing means 105 and the feature values the calculated from the plural obtained binary pictures in a feature values calculation means 106. The calculated feature values are inputted to the neural network 101 and plural pieces of extracted picture data are obtained. Plural pieces of obtained extracted picture data are inputted to a shape extraction means 107 and one sharp extracted picture is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shape extraction system used in an apparatus for inspecting, measuring, recognizing an object using image processing.

[0002]

2. Description of the Related Art Conventionally, inspection of an object using image processing
Various methods have been proposed for extracting the shape of a clear object from an unclear image when performing measurement and recognition. For example, as described in various image analysis methods in Image Processing Handbook, Shokoido Co., Ltd., issued June 8, 1987, 263p to 286p, density conversion (p263), emphasis (p263), Sharpening (p26
6), smoothing (p269), noise removal (p269), difference type operator (p280), and binarization (p277)
The shape of the object was extracted by image analysis such as.

Further, an image inspection apparatus has been proposed which determines the quality of the appearance of a product or the like to which a neural network is applied, which enables the inspection to be executed by the same method even if the inspection objects are different (Japanese Patent Laid-Open No. 3-2980). -242773 gazette).

[0004]

However, in the conventional shape extraction method using the image processing method described in the above-mentioned image processing handbook, the image of density conversion, enhancement, sharpening, noise removal, difference operator, etc. Since the image was converted into a binary image by the processing, noise and omission occurred in an image including uneven illumination or an image with poor contrast, and it was not possible to extract the object shape clearly and with high accuracy. Further, in the image inspection apparatus described in JP-A-3-242777, the input value to the neural network is 1
Since it is the image information of one sheet, and the appearance is determined from this image information, it is impossible to perform clear and highly accurate processing on an unclear image.

The present invention has been made in consideration of the above circumstances. For example, by using a plurality of image information as input values to a neural network, an object can be sharply and accurately extracted from an unclear image. A shape extraction processing system is provided.

[0006]

The technical means taken by the present invention to achieve the above object are as follows.
FIG. 1 is a block diagram showing the basic configuration of the present invention. Figure 1
In the present invention, the present invention uses the input image and its processed image to
A shape extraction system comprising a neural network 101 for inputting a characteristic amount obtained by binarization and learning the characteristic amount to obtain extracted image data of a shape, and an output means 102 for outputting various images. An image capturing unit 103 that captures a shape as an input image, a pre-processing unit 104 having various image processing functions for creating at least one processed image by performing image processing on the captured input image, and an image capturing unit 103. A binarization processing unit 105 that binarizes a plurality of images out of the input image obtained in step S1 and the plurality of processed images obtained by the preprocessing unit 104 and converts the images into a plurality of binary images; A plurality of binary images obtained by the processing unit 105 are further divided into a plurality of regions, respectively, and a feature amount calculation unit 106 for calculating a feature amount of the regions and a feature amount calculation unit 106 are provided. Shape extraction processing comprising: a shape extraction unit 107 that creates one extracted image from a plurality of extracted image data obtained by inputting the calculated feature amount into a neural network that has been learned in advance. System.

The preprocessing means 104 is preferably configured to perform preprocessing by at least one image processing method from various image processing functions such as density conversion, enhancement, sharpening, noise removal, and difference operator. .

It is preferable that the characteristic amount calculating means 106 is configured to obtain the characteristic amount by dividing the binary image into n areas and calculating the image area of the object contained in each area.

In the present invention, the neural network 101 is a network formed by connecting neurons (nerve cells forming the brain) which are the basic constituent elements of the brain, and the neurons are connected in a neural network form and are connected to each other. It is composed of a neuro computer which performs parallel information processing by moving processing units in parallel while exchanging signals with each other, a personal computer which stores a program capable of executing these functions, and the like. As the output unit 102, a display device such as a CRT display device, an LCD (liquid crystal display) device, an EL display device, or a printing device such as a thermal transfer printer, an inkjet printer, a laser printer, or the like is used.

As the image pickup means 103, a CCD (charge coupled device) camera or an ITV camera is used as an image input device. As the preprocessing unit 104 and the binarization processing unit 105, an image processing device having an image processing function such as density conversion, enhancement, sharpening, noise removal, difference operator, and binarization process is used. Further, the image processing device is a CPU, R
It is composed of a microcomputer including an OM, a RAM, an I / O port, and an external storage device. A personal computer is used as the feature amount calculating means 106 and the shape extracting means 107, and the neural network 1
01 is connected.

[0011]

According to the present invention, the image pickup means 10 shown in FIG.
The image processing from the image input from 3 is performed by the preprocessing unit 104.
Image processing is performed to obtain at least one processed image, a plurality of images of the input image and the processed image are binarized by the binarization processing unit 105, and the obtained plurality of binary images are feature amounts. The calculating unit 106 further divides the region into a plurality of regions to calculate the feature amount of the region, inputs the calculated feature amount to the neural network 101 learned in advance, obtains a plurality of extracted image data, and obtains the plurality of obtained image data. The extracted image data of is input to the shape extraction means 107 to obtain one extracted image. That is, it is possible to selectively take out the correct portion from a plurality of binary images and extract the object shape clearly and with high accuracy.

According to the above arrangement, the preprocessing means 104
Has various image processing functions such as density conversion, enhancement, sharpening, noise removal, and difference operator.
The pre-processing can be performed by more than one image processing method.

Further, according to the above configuration, the feature amount calculating means 106 obtains the feature amount by dividing the binary image into n regions and calculating the image area of the object included in each region. You can

[0014]

The present invention will be described in detail below based on the embodiments shown in the drawings. The present invention is not limited to this. Further, the present invention is suitable for, for example, shape extraction processing of fine particles of inorganic powder, and each constituent element achieves the shape extraction processing of the present invention, as well as inspection / measurement / recognition of objects using image processing. It can be applied to the shape extracting device.

FIG. 2 is a block diagram showing an embodiment of the shape extraction system of the present invention. In the figure, 1 is a CCD
A camera (camera using a charge-coupled device), which is an imaging device that captures a sample as an image. 2 is an optical microscope, which magnifies the image of the sample. Reference numeral 3 is a sample, for example, inorganic powder (barium sulfate) is used. Reference numeral 4 denotes an auto stage, which is an apparatus that moves a stage (a mounting table) on which the sample 3 is placed to a designated imaging position. Reference numeral 5 denotes an image processing device, which emphasizes an image input by the image pickup device,
Image preprocessing function 5 that obtains a processed image by performing image processing such as sharpening, smoothing, noise removal, and difference operation.
A (see, for example, Reference: Showa Kyoudou (1987.6), “Image Processing Handbook”, p263-286), and a binarization processing function 5B for binarizing an input image or a processed image, 5A, Processing is performed in the order of 5B.

Reference numeral 6 denotes a monitor TV, which is a CCD camera 1
It is a display device for displaying the input image imaged in (1) or the processed image processed by the image processing device 5. An autofocus control device 7 is a control device for automatically focusing the optical microscope 2 on the sample 3. Reference numeral 8 denotes a personal computer, which has a feature amount calculation processing function 8A for obtaining a feature amount from a binary image obtained by the image processing device 5, an image extraction processing function 8B for obtaining an extracted image from the feature amount using a neural network, A computer system capable of performing the shape calculation processing function 8C for obtaining the size and shape is provided. Reference numeral 9 denotes a display, which is a display device for displaying the measurement result of the personal computer. Reference numeral 10 denotes an auto stage control device, which is a control device for controlling the movement amount of the auto stage.

FIG. 3 is an explanatory view showing the image processing procedure of the shape extraction system of the present invention. This figure shows a processing procedure for extracting a clear binary image of an object from a blurred image obtained by capturing fine particles of inorganic powder (barium sulfate) with the optical microscope 2. The processing procedure will be described with reference to the block diagram shown in FIG. 2 and the explanatory diagrams of FIGS. 4 to 8. FIG. 3A is an input image of the sample 3 whose shape is to be extracted. Based on the movement amount set by the auto stage control device 10, the auto stage 4 moves to a position where the stage on which the sample 3 whose shape is to be extracted is mounted is imaged. Then, the focus of the optical microscope 2 is automatically focused on the sample 3 by the autofocus control device 7, and the CCD camera 1 captures an enlarged image of the sample 3 by the optical microscope 2. The input image is displayed on the monitor TV 6.

Here, the image processing device 5 applies enhancement, sharpening, smoothing, noise removal, difference operator, etc. to the input image picked up by the CCD camera 1, FIG. 3-1, by the image preprocessing function 5A. And obtain at least one processed image. FIG. 3-2 is a difference operator processing image, and FIG.
1 is a processed image obtained by performing Kirsch (see "Kirsch" in "Image Processing Handbook", p281) which is one of the difference operators. Next, the image processing device 5
The binarization processing function 5B obtains a plurality of binary images from a plurality of images of the difference operator processed image and the input image as shown in FIG. 3-2. For example, FIG.
3A is a binary image obtained by the binarization processing function 5B of FIG. 3A, and FIG. 3A shows the binary image of FIG.
3b shows the binary image of FIG. Further, the processed image by the image processing device 5 is displayed on the display screen of the monitor TV 6.

Next, a description will be given of the feature amount calculation processing function 8A in which the personal computer 8 obtains a feature amount from a plurality of binary images obtained by the image processing device 5. Figure 4
FIG. 4 is an explanatory diagram showing area division of a binary image and a feature amount of the area. In FIG. 4, FIGS. 4-1a and 4-1b are enlarged views of the two binary images shown in FIGS. 3-3a and 3-3b, respectively. As shown in FIGS. 4-1a and 4-1b, the number of regions is divided into n in total. A1, A2, ..., Ai, ..., Aj ,.
And Each of these areas is composed of a set of pixels, which is the smallest unit that constitutes an image, and the pixels are represented by white when an object exists and black when it does not exist.

Next, the number of pixels in which an object exists in a certain area Ai, that is, the number of white pixels is set as a feature amount of that area,
It is represented by S (Ai). In this way, a total of n feature quantities, S (A1), S (A2), ...
S (Ai), ..., S (Aj), ..., S (An) are obtained. For example, FIG. 4-2 is an enlarged view of the area Ai. The area Ai in the figure is composed of 16 (4 × 4) pixels, and the characteristic amount S (Ai) of this area indicates 7. Also here
Although the areas of a plurality of images are equally divided, they may be divided into arbitrary sizes and shapes. 3-4a and FIG.
-4b is similar to FIG. 4A and FIG.
FIG. 3A is a diagram of a stage in which two binary images shown in FIG.

FIG. 5 is an explanatory diagram showing a hierarchical neural network. Further, FIG. 3-5 shows a hierarchical neural network as in FIG. Further, FIG. 6 is an explanatory diagram showing the neurons constituting each layer of the neural network, and FIG. 7 is an explanatory diagram showing the sigmoid function that determines the output of the neural network. Here, regarding this hierarchical neural network, FIGS.
Will be explained. As shown in FIG. 5, the hierarchical neural network includes an input layer, an intermediate layer (one or two layers).
3), it is composed of 3 or 4 layers consisting of output layers,
It is coupled only in one direction from the input layer to the output layer.

Each layer is composed of elements called neurons shown in FIG. This neuron has each input Xj and the corresponding weight W as shown by the equation in the figure.
The sum of products with ij is converted by the response function f2 to obtain the output Yi. This response function f2 is a non-linear function called a sigmoid function shown in FIG. Also the load W
As ij, a value determined by a learning method called a back propagation method (error back propagation method) is used (reference:
Kaibundou, (1990.12), "Basic Theory of Neurocomputing", p. 48-50).

FIG. 8 is an explanatory diagram showing an example of the extracted image. Next, the image extraction processing function 8B for the personal computer 8 to obtain a sharp extracted image as shown in FIG. 8 by using the neural network will be described.
Specifically, an input layer consisting of n neurons equal in number to the feature quantity, an intermediate layer whose number is not specified, but half of the number of neurons in the input layer is empirically considered to be good, extraction A neural network composed of three layers, that is, an output layer composed of m neurons having the same number of pixels as the image, is used.

Characteristic quantities S (A1), S (A2), ..., S
(Ai), ..., S (Aj), ..., S (An) are input values IN1, IN2, ..., INi, ..., IN, respectively.
j, ..., INn, process, output value OU
When each of T1, OUT2, ..., OUTj, ..., OUTm is 0.9 or more, it is assumed that the pixel corresponding to the output value is “1”, that is, an object exists.
If "0", that is, no object exists, a clear extracted image of the object as shown in FIG. 8 is obtained. Also, FIGS.
An extracted image is shown similarly to. Further, the size / shape can be obtained from the extracted image by the shape calculation processing function 8C of the personal computer 8. Display 9
Displays the size / shape of the extracted image obtained by the personal computer 8.

FIG. 9 is a flow chart showing the processing contents of the shape extraction system of the present invention. In the figure, step S1: it is judged whether shape extraction is to be continued or to end. If not, the process proceeds to the next step S2. Step S2: The sample on the stage 4 is moved to the next measurement position. Step S3: Focus the optical microscope 2 on the sample. Step S4: An image is taken by the CCD camera 1. Step S5: A measurement process is performed on the input image captured in step S4.

Step S5-1: A difference operator (Kilsch) is performed on the image captured in Step S4. Step S5-2: The image processed in step S5-1 is binarized. Step S5-3: The image captured in step S4 is set to 2
Quantify. Step S5-4: A feature amount is calculated from the binary image processed in steps S5-2 and S5-3. Step S5-5: The neural network is processed according to the feature amount calculated in step S5-4.

Step S5-6: The figures processed in step S5-5 are numbered for each figure. Step S5-7: The size / shape is obtained for each figure numbered in Step S5-6. Step S6: The measurement result is displayed on the monitor TV6. Thus, by performing image processing using the neural network, it is possible to automatically and highly accurately extract an object from an unclear image.

[0028]

According to the present invention, it is possible to automatically and highly accurately extract an object from an unclear image by processing a plurality of image information using a neural network. It can be used for devices that perform measurement and recognition with high accuracy.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of the present invention.

FIG. 2 is a block diagram showing an embodiment of a shape extraction system of the present invention.

FIG. 3 is an explanatory diagram showing an image processing procedure of the shape extraction system of the present invention.

FIG. 4 is an explanatory diagram showing area division of a binary image and feature amounts of the area.

FIG. 5 is an explanatory diagram showing a hierarchical neural network.

FIG. 6 is an explanatory diagram showing neurons forming each layer of a neural network.

FIG. 7 is an explanatory diagram showing a sigmoid function that determines the output of a neural network.

FIG. 8 is an explanatory diagram showing an example of an extracted image.

FIG. 9 is a flowchart showing the processing contents of the shape extraction system of the present invention.

[Explanation of symbols]

 1 CCD Camera 2 Optical Microscope 3 Sample 4 Auto Stage 5 Image Processing Device 6 Monitor TV 7 Auto Focus Control Device 8 Personal Computer 9 Display 10 Auto Stage Control Device

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 7/60 8837-5L G06F 15/70 355

Claims (3)

[Claims] 1. A neural network for inputting a feature amount obtained by binarizing an input image and its processed image and learning the feature amount to obtain extracted image data of a shape, and an output means for outputting various images. A shape extraction system comprising: an image pickup means for picking up the shape of an object as an input image; and various image processing functions for creating at least one or more processed images by processing the picked-up input image. Binary processing means for binarizing a plurality of images among the input image obtained by the preprocessing means and the image pickup means and the plurality of processed images obtained by the preprocessing means, and converting the plurality of images into a plurality of binary images. A plurality of binary images obtained by the binarization processing unit, each of which is further divided into a plurality of regions, and a feature amount calculation unit that calculates a feature amount of the region; and a feature amount calculated by the feature amount calculation unit A shape extraction system comprising: a shape extraction unit that creates one extracted image from a plurality of extracted image data obtained by inputting the amount into a neural network that has been learned in advance. 2. The preprocessing means performs preprocessing by at least one image processing method from various image processing functions such as density conversion, enhancement, sharpening, noise removal, and difference operator. The shape extraction system according to Item 1. 3. The feature amount calculating means divides the binary image into n regions and calculates the image area of an object included in each region to obtain the feature amount. The described shape extraction system.