JPS58222383A - Picture processing system - Google Patents

Abstract

PURPOSE:To extract coarse profile in high speed with a simple constitution, by processing a contrast picture in the unit of picture elements, and using a smoothing operation section and a secondary differentiation operating section comprising a space filter. CONSTITUTION:An analog picture signal in the unit of picture elements inputted from a TV camera 1 is converted into a picture signal of a multiple level at an A/D converter 2 and inputted to the smoothing operation section 3, which uses the space filter comprising a data shift circuit 3-1 storing a prescribed length of data, a multiplication circuit 3-2 multiplying the stored data with a set weight number, an adder 3-3 summing the result all together, and a divider dividing the level increased by the addition so as to be brought into a suitable range, and reduces noise superimposed on the picture data and inputs the result to the secondary differentiation circuit 4. The circuit 4 uses a similar space filter, emphasizes a sudden change in the lightness and inputs the result to a pole extracting circuit 5. The circuit 5 extracts the pole of the picture from the input signal and set it to a picture memory 6. Thus, the profile of the picture is obtained in high speed in synchronizing with the camera 1.

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Technical Field of the Invention The present invention relates to an image processing method, and in particular, image processing such as smoothing operations, quadratic differential operations, and coarse contour line extraction for multilevel gray image signals. This invention relates to a system that performs processing at high speed to quickly extract parts where brightness changes rapidly, and enables real-time processing using a synchronization signal from a TV camera.

(2) Background of the technology Recently, industrial robots have been used in various fields.9 For example, in the case of mechanical assembly robots, the gastric pot can perform nine operations in a short period of time until one operation, such as tightening a screw, is completed. If you don't recognize what a certain object is for the process, you won't be able to continue the action. For this purpose, it is necessary for the robot's eyes to quickly process multilevel image signals input from a TV camera to quickly recognize what the object is. To do this, we extract parts where the brightness changes suddenly at high speed.

It is necessary to capture the outline of an object.

(8) Prior art and problems In general, image data obtained using a TV camera etc. has a size of 256 x 256 (or 128 x 128)
) Since it is a two-dimensional 8-bit gray level of pixels,
The amount of information is extremely large. For this reason, conventionally when trying to handle such image data,
Either the image signal from the TV camera is stored in the image memory for one frame and the data is transferred to a large computer for processing, or the video signal from the TV camera is converted into binary data by determining the slice level in some way. Techniques have been used to compress data, store it in memory, and perform logical operations on the data.

Therefore, while the former can perform advanced image processing, it has the disadvantage of requiring large memory and high-speed large-scale computers to achieve high-speed processing, resulting in high costs. Determining the appropriate slice level for binarization is difficult, and since multi-level information is omitted to binary level information, the amount of information is missing, so even if this is processed, It has the disadvantage that it cannot perform sophisticated processing, for example, it cannot process images with complex changes in shading.

Recently, some general-purpose image processing devices have been designed to speed up image processing by arranging one basic calculation module for each pixel or multiple pixels and performing parallel calculations. In order to do this, it is necessary to arrange a large number of basic arithmetic modules, which has the drawbacks of increasing the size of the processing device itself, increasing costs, and making address control complicated.

(4) Purpose of the invention The purpose of the present invention is to improve the above-mentioned problems.
For multilevel grayscale image data.

By using a spatial filter to perform image processing such as smoothing calculations, quadratic differential calculations 9 image enhancement, and coarse contour extraction in real time in synchronization with the TV camera: 1
An object of the present invention is to provide an image processing method that can process multilevel grayscale image data at high speed with a simple hardware configuration.

(5) Structure of the Invention In order to achieve the above object, the image processing method of the present invention processes a grayscale image pixel by pixel and extracts its rough outline, and the image processing method uses data that holds a constant length of data. A shift circuit, a multiplication circuit that multiplies the held data by a preset weighting coefficient, an addition circuit that adds all the results, and a system that keeps the increased level due to squaring and addition within an appropriate range. A smoothing operation means that reduces random noise superimposed on image data using a spatial filter configured by a division circuit that performs division, and a smoothing calculation means that reduces the brightness of image data using a spatial filter configured similarly. a second-order differential calculation means for emphasizing sudden changes;
The present invention is characterized in that it includes a pole point extracting means for extracting pole points from these processed data, and extracts the rough outline of a multi-level grayscale image as a binary image.

(6) Embodiment of the Invention Before describing in detail one embodiment of the present invention, the general structure of the present invention will be explained based on Fig. 1 and Fig. 2. Here, FIG. 1 shows a simple configuration diagram of the image processing method of the present invention, and FIG. 2 shows that the image data of (b) is obtained for the original image #ζ of (cL). And in Figure 1,
1 is a TV camera, 2 is an A/D converter, 3 is a smoothing calculation section, 4 is a second-order differential calculation section, 5 is a pole extraction section, 6 is an image memory, 7 is a scanning signal generation section, and a second In the figure, 8 is an object to be recognized.

In FIG. 1, an analog image signal input from a TV camera 1 is sent to an A/D converter 2.

For example, it is converted into an 8-bit multilevel image signal.

This image signal is then processed by a smoothing calculation unit 3 to reduce noise such as flickering lighting, background noise, and changes in the external environment, and then by a second-order differential calculation unit 4 to calculate the second-order differential of the grayscale image signal to create an image. The parts where the brightness suddenly changes are extracted.

In this way, a second-order differential output as shown in FIG. 2(b-1) can be obtained from the image signal for the object 8 in FIG. 2(g-1), for example. Therefore, by extracting the extrema of this quadratic differential output using the extrema extracting section 5, it becomes possible to extract the rough contour of the object without particularly providing a slice level. Therefore.

By setting the output of the pole point extracting section 5 in the image memory 6, the outline of the object can be extracted without setting a special slice level. If such image signal processing control is performed in synchronization with the scanning signal from the scanning signal generating section 7, this image processing can be performed in real time and at high speed in synchronization with the TV left camera.

Hereinafter, one embodiment of the present invention will be described based on FIGS. 3 to 12 and with reference to maps.

Figure 3 is an explanatory diagram of the 3x3 spatial filter that constitutes the smoothing calculation unit 3, second-order differential calculation unit 4, etc.; ':)v4 diagram is an explanatory diagram of the purpose of the spatial filter and its weighting coefficient. Figure 5 6 is an explanatory diagram of an example of a data shift circuit, FIG. 7 is an explanatory diagram of a multiplication circuit, and FIG. 8 is an explanatory diagram of an adder circuit.

Figure 11 is an explanatory diagram of the division circuit; O'iθ diagram is an explanatory diagram of an example of the data shift circuit and the pole extractor that constitute the pole extractor; Figure 11 is an explanatory diagram of the basic operation of the pole extractor; 12
The figure is a binarized image diagram showing that different coarse contours can be obtained depending on the slice level.

The smoothing calculation unit 3 and the second-order differential calculation unit 4 are composed of 3×3 spatial filters, and differ only in their weighting coefficients. Therefore, first, for the sake of simplicity, a 3×3 spatial filter will be explained with reference to FIG.

This spatial filter is a linear expression that determines the gray level values of pixels in nine regular rows.

・・・・・・■ Here, Xm, % and Y m, n are respectively screen M
The values before and after filtering of the gray level value of the pixel in row m and column % above are shown, and fi and j are weighting coefficients. Therefore, when determining the load coefficient '*rj as shown in Figure 3 (b), *Yrn, n is Ym
, n = −[Ytn−s , n−1x 1+Xm,
n-I X2 +Xm+ 1. n-1x 16 +Xm-1. n x 2+Xtn, n x 4+Xm+
t, nX 2+Xm-1. n+x X 1+X, yn,
n+t×2+Xm+t, n+*].

This spatial filter * converts the values of Ym, n to Xm, n
and the surrounding 8 pixels Xm+i, n+j (s = -L
O+ 1 p j =-LOl but i=0. j=0
is determined by multiplying by "rj" set in advance. Therefore, "+
The characteristics of this filter can be changed in various ways depending on how the value of 3 is selected, and the value setting according to the processing content to be performed is shown in .As described in the division circuit described later, the denominator is as follows. This is to ensure that the size of the calculation data (the bit length of the Y4 field) is within the dynamic range of the data processing device, and may be replaced with an appropriate coefficient to allow straddling.

FIG. 4 shows examples of uses of spatial filters and corresponding weighting coefficients. If it is used as a smoothing filter to reduce random noise on the screen, it is effective to use smoothing filters (L-2) to (L-4), and it is effective to reduce the brightness on one screen. If you want to emphasize parts with sudden changes, use 2.
(LP-1), (LP-2) for order differential filters and (H-3) for emphasis.

(H-4) etc. are effective. If you want to detect a boundary line, you can use (B-1), (B-2), etc. (L-2) or (L-4) is used as the smoothing calculation unit 3 in Fang 1 diagram, and (L-2) or (L-4) is used as the second-order differential calculation unit 4.
LP-1) or (LP-2) is used.

Next, we will explain the hardware configuration that realizes the calculation of the above formula (1). As mentioned above, the smoothing calculation and quadratic differential calculation are performed using the formula (2).As shown in Fig. 5, the smoothing calculation section 3 is a data shift circuit 3-11, a multiplication circuit 3-2. Addition circuit 3
-3, a division circuit 3-4, and the like, and the second-order differential operation section 4 also includes a data shift circuit 4-11 and a multiplication circuit 4.
-2. It is composed of an adder circuit 4-3, a divider circuit 4-4, and the like.

Therefore, a spatial filter for realizing the operation of the formula (2) is composed of a data shift circuit, a calculation circuit, an addition circuit, a division circuit, and the like.

The data shift circuit is 3× for one screen worth of image data.
In order to execute the spatial filtering process in step 3, as shown in Figure 6, the data of 2 rows + 3 pixels is stored using a shift register, and the data of Xm+i, n+j is stored in the first-stage multiplication circuit.
(t= 1+ 0+ 1: j= 1+ (Ll
) has the function of transmitting 9 pixels. In Figure 6, 1
To explain the case of screen data with row p pixels and 2g' gray level, 1 pixel? Data expressed in bits as 1
The shift registers are used to hold three pixels in the first stage, and two pixels in the second and third stages. For the last three pixels of each stage, latches or flip-flops R1~
By cascading R9 or using a serial-in-parallel-out shift register, transmission of 9 pixels can be realized.The second and third stages are (p-3
) The shift registers 11 and 12 for pixels are used to hold data of pixels other than those for transmission.

This data shift circuit shifts one pixel at a time in response to the TV left camera scanning signal.

As shown in the off-line diagram, the multiplication circuit applies a preset weighting coefficient ft,j(ze=-1,01,1;/=-1,
0, 1) to get Wm+t, n+1. =Xm+i, n+jxf<, j−-
−■(I”−1, (L 1 * j= L O+ 1
) has a multiplication function. For this purpose, nine multipliers, ie, the first multiplier 13 to the O9 multiplier 21, are used as shown in the OFF diagram, and the first multiplier 13 has loading coefficients /-s, -
1 is set, and multiplication with pixels Xm-1 and s-1 sent from the data shift circuit is performed. and the second
The multiplier 14 is set with weighting coefficients /, -1, and is multiplied by the pixel Xm-1,n sent out from the data shift circuit. In this way, the multiplication in the equation 0 can be performed using the multiplication circuit shown in FIG.

As shown in Fig. 8, the adder circuit calculates the nine values W+i, n+3 (I=-1・0・II) calculated by the multiplier circuit.
p j”'-L O・1) to the sum of these (s=1,0
+1; j=-1+0+1). For this purpose, first adder 22 to O8 adder 29 are used as shown in FIG.

Further, the division circuit divides the gray level Zm, n increased by the multiplication operation and addition operation in the multiplication circuit and the addition circuit by a normalization coefficient H so that it falls within the range of the gradation level of the image memory and the monitor TV. The purpose is to seek
As shown in Figure 9, it is composed of a first data selector 30 to a third data selector 37, and by performing the calculation of the 0 formula, the final calculation results Ym and n of the 0 formula can be obtained. . Figure 9 shows an example of reducing the 16-bit gray level data Zm,n obtained by multiplying and adding 8-bit gray pixels to 8-bit data Y, n. Therefore, the first data selector 30 contains 16-bit data Z? FL,? 2 to 9 bits of l are input, and 1 is input to the second data selector 31.
Of the 6-bit data Zm, n, 3 to 10 bits are input, and similarly, 4 to 11 bits are input to the third data selector 32, 5 to 12 bits are input to the off data selector 33, and the fifth data Selector 34 has 6 to 13 bits,
7 to 14 bits are input to the off data selector 35, 8 to 15 bits are input to the off data selector 36, and 9 to 16 bits are input to the off data selector 37. Each data selector 30-37 selects and outputs any one of the first to eighth bits of the 8-bit input in response to a 3-bit selection signal transmitted from the selector line SL. For example, when quoting Zm- by 27, each data selector 30 to 37 is set so that the first bit is output from each data selector 30 to 37.
is controlled, the first data selector 30 outputs 2 to
The first bit of the 9 bits, ie the 2nd bit of the 16 bits, is output.

The second data selector 31 outputs the first bit of 3 to 10 bits, that is, the 3rd bit of 16 bits, and the third data selector 32 outputs 4 to 11 bits.
The first bit of the bits, that is, the fourth bit of the 16 bits, is output, and in the same way, the first bit of the 9 to 16 bits, that is, the 4th bit of the 16 bits, is output from the O8 data selector 37. The 9th bit of
2~ which is the value obtained by dividing 16 bits Zm- by 27 from 7
It will output 9 bits. Similarly, when quoting Zm and n by 26, it is sufficient to output the second bit from each data selector 30 to 37, that is, 3 to 10 bits from 16-bit data; is the third pick from each data selector 30 to 37.
It is better to output 4 to 11 bits of 16-bit data by outputting i. In this way, the Nth data selector receives 2 to 2 of the 16 bit data.
By inputting the hit number and setting the value of k in the selector line corresponding to the number 2k divided by 1, division can be realized at high speed. Here, the value of the normalization coefficient H is selected. In this way, by using the data selector, it is possible to speed up division.

The pole extractor 5 includes a data shift circuit 5-1 shown in FIG. 10 (P) and a data comparison circuit 5-2 shown in FIG. 10 (B).
The operating principle will be explained with reference to Fig. 11. For simplification of explanation.

An example of finding the extreme point R when the original image P of the tooth 11 (a) has the one-dimensional density distribution as shown will be described. A left-shifted image obtained by shifting the original image P by one pixel △ to the left is called PL, and a right-shifted image obtained by shifting the original image P by one pixel △ to the right is called PR.

Then, by comparing the original image P and the shifted image PL, the density of the original image is greater or equal (hereinafter this is referred to as P≧P
The original image P and the right-shifted image PR are compared to extract a portion QR in which the density of the original image is greater or equal (hereinafter referred to as P≧PR). The extreme point (maximum point) R found at this time is considered to be at least greater than or equal to the neighboring pixels on both sides, so
The extreme point R can be found by extracting the portion QL and QR as shown in FIG. 11(e). however.

Exclude when P=pL=pR.

By the way, when a pole exists in a pattern as shown in FIG. Although As can be extracted, the pole sides It, ,
Q4 cannot be extracted, so by scanning this also in the Y direction, the pole side Q! , λ4 can be extracted. Therefore, for example, an extraction window W′ having pixels B, D, E, F, and H as shown in FIG.
By using , you can extract the extreme points in the X or Y direction of a two-dimensional grayscale image.

Now, one frame shown in the example in FIG.
The scanning line hO+hl
+ hM... Register the second-order differential calculation data obtained by sequentially scanning and passing through the smoothing calculation section 3 and the second-order differential calculation section 4! , r!・・・・・・r7 and (p
-3) It is transmitted to the data shift circuit 5-1 constituted by bit-length shift registers 11 and 12. The registers r1 to r7 and the shift registers 11 and 12 have a width of 8 bits, and thereby output second-order differential calculation data for the pixel E and each of the pixels above and below it, F, B, and H. These data are stored in the comparators C-1 to C-4 as shown in FIG. 10(b). and day LA
N-1, AN-2, OR gate 0R-1 to 0R-5,
NAND-1 to NAND-2, a peak register PM, and the like are transmitted to a data comparison circuit 5-2.

Comparator C-1 compares the second-order differential calculation data of pixels E and D, comparator C-2 compares the second-order differential calculation data of pixels E and F, and comparator C-3 compares the second-order differential calculation data of pixels E and F.
The second-order differential calculation data of pixels E and B are compared, and the second-order differential calculation data of pixels E and H are compared by the comparator C-4. Therefore, when pixel E is the pole in the horizontal direction, comparators C-1 and C-2 output rlJ, so AN-1 outputs rlJ, and pixel E is the pole in the vertical direction. In the case of , comparator C-3,
Since "1" is output from each C-4.

"1" is output from AND gate AN-2.

Therefore, OR-1 outputs "1" when pixel E is a pole in at least one of the horizontal or vertical directions, and outputs "0" when it is not a pole in either direction. do. Therefore, the peak flag can be determined using the output X of the OR gate OR-1 as an index. Then, the peak register PM is set to 8 of pixel E at that time.
If bit original image data is applied, it becomes possible to output the actual peak value of the pixel E at the extreme point.

In this way, by using two stages of spatial filters and appropriately selecting the weighting coefficients, it is possible to reduce random noise on one image and emphasize sudden changes in brightness.

For the original image in Figure 2 (a), the (L-2) me'□i1F multiplex coefficient in Figure 4 is used as a smoothing filter, and the load (LP-2) is used as a filter for second-order differentiation. FIG. 2(b) shows how the coefficients are used. In Fig. 2 C→, (, -1) is a case in which light is shining on a cylinder, which is an object $, from diagonally upward to the right, and (α-2) is a view when it is viewed from above. Further, (α-3) shows the change in brightness in the X-Y one-dimensional direction of (a-2).

This is a three-dimensional display of the gray level after smoothing calculation and second-order differential calculation using each of the above-mentioned filters (
b-1), and (6-z) shows its shading level with isobright lines.

And (b-3) shows a cross section of (b-2> in the x-yi dimension direction.

Points where the brightness changes suddenly in the gray scale image are important points corresponding to line points on the surface of the object. now.

As shown in Fig. 2<b-s>, the case where a sudden change in brightness emphasized by a two-stage spatial filter of smoothing calculation and second-order differential calculation is extracted by binarizing it at an appropriate slice level is shown. think. Figure 2 (6-
3) When three levels A, B, and C are selected and binarized, different coarse contours are obtained as shown in Figures 12 (a) and (b) (6). Slice level A
In this case, as shown in FIG. 12(a), the level is too high and breaks occur in the contour, and at slice level C,
As shown in FIG. 12(c), the contour line 9 becomes thick. Therefore, it is necessary to perform another process of repairing the discontinuity or thinning the contour line. Furthermore, if an appropriate slice level can be set as in the case of slice level B, a contour line as shown in FIG.
Although it is convenient because line thinning processing is not necessary, the shading level obtained by the two-stage spatial filtering processing described above depends on the surface state of the object at that time, such as color, shape, and color.
Lighting conditions, lens brightness.

It is very difficult to always maintain an appropriate slicing level because the slicing level differs each time due to the aperture, etc.

Therefore, if only the ridge portion shown in FIG. 2(b-t) can be extracted, the feature points of the original image can be extracted as a series of line drawings without the troublesome task of setting the slice level. In order to execute this ridge portion extraction process, the above-mentioned extreme point extraction section 5 is used. As described above, this pole point extracting unit 5 extracts the pole points in the horizontal direction and the vertical direction and sets them to high level "1" and sets the other points to low level rOJ, so slice level 2 is required.
For example, Fig. 2 (6-1)
Only the ridge portion of can be extracted as high level rlJ. For this reason, this image processing method is capable of extracting extreme points without having to set the slice level during binarization, which is subjective and strongly influenced by the external environment.

In addition, in this image processing method, the data shift circuit of the data shift circuit, Xm+ i, n+ j to the multiplication circuit
(I”-1+ 0,1; j= 1.Or 1)
input and calculation results Wm+t, n+j (i=L
By using a control signal that synchronizes the output of (L1;
0 can be completed in 6.7fILs.

Furthermore, by providing an address generating section after the extreme point extracting section and transferring the address information to the computer only when the pixel is an extreme point, it becomes possible to dramatically save memory.

In addition, in the above explanation, the case was described using a 3X3 spatial filter, but the size of the filter can be changed to 5X5, 7X7.
By increasing the size, the hardware device becomes larger, but it becomes possible to perform more detailed image processing. The polar point extracting section can also be executed by software means instead of -do.

By configuring this image processing method using one chip of VLSI, it is possible to classify and sort objects.

In fields that require visual recognition, such as product inspection, it becomes possible to recognize two-dimensional shapes using microcomputer-level computers.

(7) Effects of the Invention According to the present invention, it is possible to perform image processing at high speed because the smoothing calculation unit and the second-order differential calculation unit are used, each of which is composed of a spatial filter. Then, the pole extraction part is
If the camera is configured in a single mode, higher speed processing can be achieved, and image processing can be performed online in synchronization with the TV left camera scanning signal. Furthermore, by extracting extreme points, it is now possible to accurately extract the rough outline of an object without setting a special slice level.

[Brief explanation of drawings]

FIG. 1 is a simple configuration diagram of the image processing method of the present invention. Fig. 2 is an explanatory diagram of the operation of the present invention, Fig. 3 is an explanatory diagram of the spatial filter, Fig. 4 is an explanatory diagram of the use of the spatial filter and its loading coefficient, and Fig. 5 is a concrete configuration of an embodiment of the present invention. Figure, 6th
The figure is an explanatory diagram of an example of a data shift circuit, the OFF diagram is an explanatory diagram of an example of a multiplication circuit, and FIG. 8 is an explanatory diagram of an example of an adder circuit.
FIG. 9 is an explanatory diagram of an example of a division circuit, FIG. 10 is an explanatory diagram of an example of a data shift circuit and an extremum extractor constituting the extrema extractor, and FIG. 11 is an explanatory diagram of the basic operation of the extrema extractor. FIG. 12 is a binarized image diagram showing that different coarse contours can be obtained depending on the slice level. In the figure, 1 is a TV controller, 2 is an A/D inverter, 3 is a smoothing calculation unit, 4 is a second-order differential calculation unit, 5 is a pole extraction unit, 6 is an image memory, 7 is a scanning signal generator. 8 is an object. Figure 4 tsrw

Claims (1)

[Claims] (1) In an image processing method that processes a grayscale image pixel by pixel and extracts its rough outline, a data shift circuit that holds a certain length of data and multiplication of the held data by a preset weighting coefficient are used. A spatial filter is used to process image data using a spatial filter consisting of a multiplier circuit that performs multiplication, an adder circuit that adds all the results, and a divider circuit that divides the level increased by 1 multiplication and addition so that it falls within an appropriate range. A smoothing calculation means for reducing random noise superimposed on the image data, a quadratic differential calculation means for emphasizing sudden changes in brightness of the image data using a spatial filter configured in the same way as in 1, and a second-order differential calculation means for emphasizing sudden changes in brightness of the image data. 1. An image processing method, comprising: extrema extracting means for extracting a rough contour of a multi-level grayscale image as a binary image.