JPH0222421B2 - - Google Patents

Description

[Detailed Description of the Invention] (1) Technical Field of the Invention The present invention relates to an image processing method, and particularly to image processing methods such as smoothing operations, quadratic differential operations, and coarse contour line extraction for multilevel gray image signals. This technology performs processing at high speed to extract areas with sudden changes in brightness at high speed, and enables real-time processing using synchronization signals from TV cameras.

(2) Background of the technology Recently, industrial robots have been used in various fields. For example, in the case of mechanical assembly robots, the robot can quickly move on to the next process in a short period of time until it finishes one operation, such as tightening a screw. If you don't recognize what the object is, you won't be able to continue the action. To achieve this, the robot's eyes must process multilevel image signals input from a TV camera at high speed to quickly recognize what the object is. To do this, we extract parts with rapid changes in brightness,
It is necessary to capture the outline of an object.

(3) Conventional technology and problems In general, image data obtained using a TV camera etc. has a screen size of, for example, 256 x 256 (or 128
128) Since it is a two-dimensional, 8-bit gray level of pixels, the amount of information is extremely large. Conventionally, when attempting to handle such image data, the image signal from the TV camera is stored in an image memory for one frame, and the data is transferred to a large computer for processing, or Techniques have been used such as determining the slice level of a video signal from a camera using some method, converting it into a binary value, compressing the data, storing it in a memory, and performing logical operations on the data.

Therefore, while the former can perform advanced image processing, it has the disadvantage of requiring large memory and high-speed large-scale computers to achieve high-speed processing, resulting in high costs. It is difficult to determine the appropriate slice level for digitization, and since multi-level information is omitted to binary level information, the amount of information is missing, so even if this is processed, advanced processing is not required. For example, it cannot process images with complex changes in shading.

Recently, some general-purpose image processing devices have been designed to speed up image processing by arranging one basic calculation module for each pixel or multiple pixels and performing parallel calculations. In order to do this, it is necessary to arrange a large number of basic arithmetic modules, which has the drawbacks that the processing device itself becomes large and costly, and address control becomes complicated.

(4) Purpose of the Invention The purpose of the present invention is to improve the above-mentioned problems by providing a
Perform image processing such as smoothing operations, quadratic differential operations, image enhancement, and coarse contour extraction using spatial filters.
The object of the present invention is to provide an image processing method capable of high-speed processing of multi-level grayscale image data with a simple hardware configuration by synchronizing with a TV camera and executing the processing in real time at high speed.

(5) Structure of the Invention In order to achieve the above object, the image processing method of the present invention processes a grayscale image pixel by pixel and extracts its rough outline, and in this image processing method, a certain length of data is shifted by the data shift method. A data shift circuit that is held in the circuit, a multiplication circuit that multiplies the held data by a preset weighting coefficient, an addition circuit that adds all the results, and a level increased by multiplication and addition. A smoothing calculation means that reduces random noise superimposed on image data by using a spatial filter configured with a division circuit that performs division so that the division falls within an appropriate range, and a spatial filter configured in the same manner. The system is equipped with a second-order differential calculation means for emphasizing sudden changes in brightness in image data, and an extreme point extraction means for extracting extreme points by comparing the gray levels of adjacent pixels from the data obtained through these processes. The present invention is characterized in that the coarse outline of a value-level grayscale image is extracted as a binary image.

(6) Embodiment of the Invention Before describing an embodiment of the present invention in detail, a schematic configuration of the present invention will be explained based on FIGS. 1 and 2. Here, FIG. 1 shows a simple configuration diagram of the image processing method of the present invention, and FIG. 2 shows that image data b is obtained from an original image a. In Figure 1, 1 is a TV camera, 2 is an A/D
Converter, 3 is a smoothing calculation unit, 4 is a second-order differential calculation unit, 5 is a pole extraction unit, 6 is an image memory, 7 is a scanning signal generation unit, and in FIG. 2, 8 is an object to be recognized. It is an object.

In FIG. 1, an analog image signal input from a TV camera 1 is sent to an A/D converter 2.
For example, it is converted into an 8-bit multilevel image signal. Then, noise such as flickering of lighting, background noise, and changes in the external environment is reduced from this image signal in a smoothing calculation section 3, and then a second derivative of the grayscale image signal is calculated in a second-order differential calculation section 4 to form an image. The parts where the brightness suddenly changes are extracted.
In this way, for example, a second-order differential output as shown in FIG. 2 b-1 can be obtained from the image signal for the object 8 in FIG. 2 a-1. Therefore, by extracting the extrema of this second-order differential output using the extrema extracting section 5, it becomes possible to extract the rough outline of the object without particularly providing a slice level. Therefore, by setting the output of the extreme point extraction section 5 in the image memory 6, the outline of the object can be extracted without setting a special slice level. If such image signal processing control is performed in synchronization with the scanning signal of the scanning signal generating section 7,
This image processing can be performed at high speed in real time by synchronizing with the TV camera.

Hereinafter, one embodiment of the present invention will be described based on FIGS. 3 to 12 and with reference to other figures.

FIG. 3 is an explanatory diagram of a 3×3 spatial filter that constitutes the smoothing calculation unit 3, the second-order differential calculation unit 4, etc.
The figure is an explanatory diagram of the use of the spatial filter and its load coefficient, and Figure 5 is a specific configuration diagram of an embodiment of the present invention.
FIG. 6 is an explanatory diagram of an example of a data shift circuit, and FIG.
FIG. 8 is an explanatory diagram of the multiplication circuit, FIG. 8 is an explanatory diagram of the addition circuit, FIG. 9 is an explanatory diagram of the division circuit, and FIG. 10 is an explanation of an example of the data shift circuit and the extremum extraction section that constitute the extremity extraction section. 11 is an explanatory diagram of the basic operation of the extreme point extracting section, and FIG. 12 is a binarized image diagram showing that different rough contours can be obtained depending on the slice level.

The smoothing calculation unit 3 and the second-order differential calculation unit 4 are composed of 3×3 spatial filters, and differ only in their weighting coefficients. Therefore, first, for the sake of simplicity, a 3.times.3 spatial filter will be explained with reference to FIG.

The spatial filter calculates the gray level value of a pixel in m columns and n rows using the following equation.

Y _n,o = ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 X _n+i,o+j・_i,j ／ ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 _i,j …… Here X _n,o and Yn _,o respectively represent the gray level value of the pixel in m rows and n columns on the screen M before and after filtering, and _i,j represent the weighting coefficients. Therefore, when the load coefficients _i,j are determined as shown in Figure 3B, Y _{n, o} = 1/16 [Y _n-1,o-1 ×1 + X _n,o-1 ×2+ X _n+1,o-1 ×1+X _n-1,o ×2+X _n,o ×4+
X _n+1,o ×2+X _n-1,o×1 +1+X _n,o+1 ×2+
X _n+1,o+1 ].

This spatial filter converts the value of Y _n,o to X _n,o and the surrounding 8 pixels X _n+i,o+j (i=-1,0,1:j=-
1, 0, 1 (excluding i=0, j=0), and is determined by multiplying by _{i, j} set in advance. Therefore, the characteristics of this filter can be changed in various ways by selecting the values of _{i and j} , and the values can be set according to the processing content to be performed.

In addition, in the above formula, the denominator is ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 _i,j
The denominator is the size of the operation data (Y _n,o
(bit length) is within the dynamic range of the data processing device, and may be replaced with an appropriate coefficient.
₁ 〓 ^i=-1 ₁ 〓 ^j=-1 If _i,j are zero, this can be set to 1.

FIG. 4 shows examples of uses of spatial filters and corresponding load coefficients. If it is used as a smoothing filter to reduce random noise on the screen, it is effective to use smoothing filters (L-2) to (L-4), and it also reduces the brightness on the screen. If you want to emphasize parts with sudden changes, use (LP-1), (LP-2) for second-order differential filters or (H-) for emphasis.
3), (H-4), etc. are valid. Furthermore, when a boundary line is detected, (B-1), (B-2), etc. may be used. In FIG. 1, (L-2) or (L-4) is used as the smoothing calculation unit 3, and (LP-1) or (LP-4) is used as the second-order differential calculation unit 4.
2) is used.

Next, the hardware configuration for realizing the calculation of the above equation will be explained.

As mentioned above, smoothing calculations and second-order differential calculations are performed using formulas, and as shown in FIG. -3, a division circuit 3-4, etc., and the second-order differential operation section 4 also includes a data shift circuit 4-1, a multiplication circuit 4-2, an addition circuit 4-3, a division circuit 4-4, etc. It consists of Therefore, a spatial filter for realizing the calculation of this equation is composed of a data shift circuit, a multiplication circuit, an addition circuit, a division circuit, and the like.

In order to perform 3×3 spatial filtering processing on image data for one screen, the data shift circuit stores and stores data of 2 rows + 3 pixels using a shift register, as shown in FIG. X _n+i,o+j (i=-1,0,1;j=-
It has the function of transmitting nine pixels (1, 0, 1). In Fig. 6, the case of screen data with p pixels in 1 row and ^2q gray level will be explained.The data displayed in 1 pixel and q bits is divided into three pixels in the first row, and p in the second and third rows. It is held using a shift register for pixels. For the last three pixels of each stage, a latch or flip-flop R is installed so that the data of each pixel can be simultaneously sent to the multiplier circuit of the next stage.
By cascade-connecting pixels 1 to R9 or using a serial-in-parallel-out shift register, nine pixels can be transmitted. The second and third stages are shift registers 1 for (p-3) pixels.
1 and 12 are used to hold data of pixels other than those sent out. This data shift circuit shifts one pixel at a time according to the scanning signal of the TV camera.

As shown in FIG. 7, the multiplication circuit applies weight coefficients _i,j (i=-1, 0,
1; j=-1,0,1) W _n+i,o+j =X _n+i,o+j × _i,j ... (i=-1,0,1; j=- 1, 0, 1). For this purpose, as shown in FIG. 7, the first multiplier 13 to the ninth multiplier 2
1, nine multipliers are used, and the first multiplier 13 is set with a weighting coefficient of _-1,-1 , and is multiplied by the pixel _Xn-1,o-1 sent from the data shift circuit. It will be done. A weighting coefficient of _-1,0 is set in the second multiplier 14, and multiplication with the pixel X _n-1,o sent from the data shift circuit is performed. In this way, the multiplication in the above equation can be performed by the multiplication circuit shown in FIG.

As shown in FIG. 8, the adder circuit calculates the nine values W _n+i,o+j (i=-1, 0,
1,;=-1,0,1), the sum of these Z _n,o = ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 W _n+i,o+j …… (i=1,0,1 ; j=-1, 0, 1). For this purpose, as shown in FIG. 8, first to eighth adders 22 to 29 are used.

Furthermore, the division circuit uses a normalization coefficient H to adjust the gray level Z _n,o increased by the multiplication and addition operations in the multiplication circuit and the addition circuit so that it falls within the range of the gray level of the image memory and the monitor TV. The purpose is to divide Y _n,o = Z _n,o /H... as shown in Figure 9.
It is composed of a data selector 30 to an eighth data selector 37, and by calculating the above equation, Y _n,o, which is the final calculation result of the above equation, can be obtained. FIG. 9 is an example of reducing the 16-bit gray level data Z _n,o to 8-bit data Y _n,o by multiplying and adding 8-bit gray pixels as described above. First, first
2 to 9 bits of the 16-bit data Z _n,o are input to the data selector 30, 3 to 10 bits of the 16-bit data Z _n,o are input to the second data selector 31, and so on. 4 to 11 bits are input to the third data selector 32, and the fourth data selector 3
3, 5 to 12 bits, fifth data selector 34
6 to 13 bits to the 6th data selector 35, 7 to 14 bits to the 7th data selector 36, 9 to 15 bits to the 8th data selector 37.
~16 bits each. Each data selector 30-37 selects and outputs any one of the first to eighth bits of the 8-bit input in response to a 3-bit selection signal transmitted from the selector line SL. For example, when quotient Z _n,o by 2 ⁷ , data selectors 30 to 3 are used.
Since each data selector 30 to 37 is controlled to output the first bit from bit 7, the first data selector 30 outputs the first bit from bits 2 to 9, that is, 2 out of 16 bits. The second data selector 31 outputs the first bit of the 3rd to 10th bits, i.e.
The third bit out of the 16 bits is output, and the third data selector 32 outputs the first bit out of the 4th to 11th bits, that is, the 4th bit out of the 16 bits. The data selector 37 outputs the first bit of 9 to 16 bits, that is, the 9th bit of 16 bits, and as a result, each data selector 30 to 37 outputs 16 bits Z _n,o It will output 2 to 9 bits, which is the value obtained by multiplying by ²⁷ . Similarly, when Zn _,o is multiplied by ²⁶ , it is sufficient to output the second bit from each data selector 30 to 37, that is, 3 to 10 bits from the 16-bit data, and the quotient is multiplied by ²⁵ . If so, each data selector 30
It is sufficient to output the third bit from .about.37 and output the 4th to 11th bits from the 16-bit data. In this way, ^{the 215-N} to ^{28-N bits of the 16-bit data are input to the Nth data} selector, and k is input to the selector line corresponding to the dividing number ^2k .
By setting the value of , division can be realized at high speed. Here, a value of 2 ^k (k=0, 1, 2...) close to ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 _i,j is selected as the normalization coefficient H. In this way, by using the data selector, it is possible to speed up division.

The pole extractor 5 is composed of a data shift circuit 5-1 shown in FIG. 10A and a data comparison circuit 5-2 shown in FIG. 10B, and the principle of operation thereof will be explained with reference to FIG. To simplify the explanation, an example will be described in which the extreme point R is determined when the original image P shown in FIG. 11A has the one-dimensional density distribution as shown. Let P _L be a left-shifted image obtained by shifting the original image P by one pixel Δ to the left, and P _R be a right-shifted image obtained by shifting the original image P by one pixel Δ to the right. Then, the original image P and the left-shifted image P _L are compared, and the original image has a higher or equal density (hereinafter referred to as P≧ _PL ), a portion Q _L is extracted, and the original image P and the right-shifted image P _R is compared and a portion Q _R in which the density of the original image is greater or equal (hereinafter referred to as P≧P _R ) is extracted. The extreme point (maximum point) R found at this time is considered to be at least at a level greater than or equal to that of the pixels on both sides, so the first
The extreme point R can be found by extracting the part Q _L and Q _R as shown in Figure 1 E. However, the case where P=P _L =P _R is excluded.

By the way, when there is a pole in a pattern as shown in L in Fig. 11, the pole sides l ₁ and l ₃ of the pattern L can be extracted by scanning only in the X direction, but the pole sides l ₂ and l ₄ cannot be extracted. Since this is not possible, the pole sides l ₂ and l ₄ can be extracted by scanning this in the Y direction as well. Therefore, for example, the 11th
By using an extraction window W' having pixels B, D, E, F, and H as shown in the figure, it is possible to extract the extreme points in the X direction or Y direction of a two-dimensional grayscale image.

Now, one frame shown in FIG.
The second-order differential operation data obtained by sequentially scanning h ₀ , h ₁ , h ₂ ... and passing through the smoothing operation section 3 and the second-order differential operation section 4 are stored in registers r ₁ , r ₂ ... r ₇ and (p-
3) It is transmitted to the data shift circuit 5-1 constituted by the data length shift registers 11 and 12.
Registers r ₁ to r ₇ and shift registers 11 and 12 are configured with an 8-bit width, so that the second-order differential calculation data for pixel E and each of the pixels D, F, B, and H on the left, right, upper, and lower sides thereof are output. Become. These data are input to the comparator C-- as shown in FIG. 10B.
1~C-4, AND gate AN-1, AN-2,
OR Gate OR-1 ~ OR-5, Nand Gate
NAND-1 ~ NAND-2, peak register
The data is transmitted to the data comparison circuit 5-2, which includes a PM and the like. Then, by comparator C-1, two of pixels E and D are detected.
The second-order differential calculation data are compared, and the second-order differential calculation data of pixels E and F are compared by comparator C-2.
Comparator C-3 compares the second-order differential calculation data of pixels E and B, and comparator C-4 compares the data of pixels E and H.
The second-order differential calculation data of are compared. Therefore, when pixel E is at the pole in the horizontal direction, comparators C-1 and C-2 each output "1", so AND gate AN-1 outputs "1", and pixel E When is the pole in the vertical direction, comparators C-3 and C-4 each output "1", and therefore AND gate AN-2 outputs "1". Therefore, OR GATE OR
-1 outputs "1" when the pixel E is at the extreme point in at least one of the horizontal direction or the vertical direction, and outputs "0" when it is not at the extreme point in either direction. Therefore, OR Gate OR-
The peak flag can be determined using the output X of 1 as an index. By applying the 8-bit original image data of pixel E at that time to the peak register PM, it becomes possible to output the actual peak value of pixel E at the extreme point.

In this way, by using two stages of spatial filters and appropriately selecting the weighting coefficients, it is possible to reduce random noise on the image and emphasize sudden changes in brightness.

Figure 2 shows the situation when using the weighting coefficient L-2 in Figure 4 as a smoothing filter and the loading coefficient LP-2 as a second-order differentiation filter for the original image in Figure 2a. It is shown in Figure 2b. In FIG. 2, a-1 shows a case in which light is shining on the cylinder, which is the object 8, from diagonally upward to the right, and a-2 shows the case when it is viewed from above. Further, a-3 shows the change in brightness in the XY one-dimensional direction of a-2. b-1 is a three-dimensional display of the shading level after smoothing calculation and second-order differential calculation using each of the above-mentioned filters, and b-2 is a three-dimensional display of the shading level using isobright lines. This is what is shown. And b-3 shows a cross section of b-2 in the X-Y one-dimensional direction.

The points where the brightness changes suddenly in the grayscale image are important points corresponding to the edge points of the surface of the object. now,
As shown in Fig. 2b-3, the case where a sudden change in brightness emphasized by a two-stage spatial filter of smoothing calculation and second-order differential calculation is extracted by binarizing it at an appropriate slice level. think. When three levels A, B, and C in FIG. 2 b-3 are selected as slice levels and binarized, different coarse contours are obtained as shown in FIG. 12 a, b, and c. At slice level A, as shown in FIG. 12a, the level is too high and breaks occur in the contour, and at slice level C, the contour line becomes thick, as shown in FIG. 12c. Therefore, it is necessary to perform another process of repairing the discontinuity or thinning the contour line. Furthermore, if an appropriate slice level can be set as in the case of slice level B, a contour line as shown in FIG. , However, the gray level obtained by the above two-stage spatial filtering process depends on the surface condition of the object at that time, color, shape, lighting conditions, lens brightness,
It is very difficult to always determine an appropriate slicing level because the slicing level differs each time due to the aperture, etc.

Therefore, if only the ridge portion shown in FIG. 2b-1 can be extracted, the feature points of the original image can be extracted as a series of line drawings without the troublesome task of setting the slice level. In order to execute this ridge portion extraction process, the above-mentioned extreme point extraction unit 5 is used. As described above, this pole point extraction unit 5 extracts the pole points in the horizontal and vertical directions and sets them to high level "1" and sets the other points to low level "0", so two slice levels are required. For example, without providing a value processing section,
Only the ridge portion of -1 can be extracted as high level "1". Therefore, this image processing method can extract the extreme point portions without performing operations such as setting the slice level during binarization, which are subjective and strongly influenced by the external environment.

In addition, in this image processing method, data shift of the data shift circuit and X _n+i,o+j (i
=-1,0,1;j=-1,0,1) input and operation result W _n+i,o+j (i=-1,0,1;j=-
By using a control signal that synchronizes the output of 1, 0, 1) with the horizontal scanning signal of the TV camera, one frame of grayscale image processing can be processed in 1/60 seconds, or approximately
It can be completed in 16.7ms.

Furthermore, by providing an address generating section after the extreme point extracting section and transferring the address information to the computer only when the pixel is at the extreme point, it is possible to dramatically save memory.

In addition, in the above explanation, the case was described using a 3×3 spatial filter, but the size of the filter was changed to 5×3.
By increasing the size to x5, 7x7, etc., the hardware device becomes larger, but finer image processing becomes possible. The extreme point extracting means can also be executed by software instead of hardware.

By configuring this image processing method using a single-chip VLSI, it becomes possible to recognize two-dimensional shapes using a microcomputer-level computer in fields that require visual recognition, such as object classification, sorting, and product inspection. .

(7) Effects of the Invention According to the present invention, it has become possible to perform image processing at high speed because the smoothing calculation section and the second-order differential calculation section are used, each of which is composed of a spatial filter. If the pole point extraction section is configured as a hardware, even higher processing speed can be achieved, and image processing can be performed online in synchronization with the scanning signal of the TV camera. Furthermore, by extracting extreme points, it is now possible to accurately extract the rough outline of an object without setting a special slice level.

[Brief explanation of drawings]

Fig. 1 is a simple configuration diagram of the image processing method of the present invention, Fig. 2 is an explanatory diagram of the operation of the present invention, Fig. 3 is an explanatory diagram of the spatial filter, and Fig. 4 is an explanation of the use of the spatial filter and its loading coefficient. 5 is a specific configuration diagram of an embodiment of the present invention, FIG. 6 is an explanatory diagram of an example of a data shift circuit, FIG. 7 is an explanatory diagram of an example of a multiplication circuit, and FIG. 8 is an explanatory diagram of an example of a multiplication circuit. An explanatory diagram of an example, FIG. 9 is an explanatory diagram of an example of a division circuit, FIG. 10 is an explanatory diagram of an example of the data shift circuit and the extrema extractor that constitute the extrema extractor, and FIG. 11 is the basics of the extrema extractor. The operation explanatory diagram, FIG. 12, is a binarized image diagram showing that different coarse contours can be obtained depending on the slice level. In the figure, 1 is a TV camera, 2 is an A/D converter, 3 is a smoothing calculation unit, 4 is a second-order differential calculation unit, and 5
6 is an image memory, 7 is a scanning signal generator, and 8 is an object.

Claims (1)

[Claims] 1 In an image processing method that processes a grayscale image pixel by pixel and extracts its rough contour, a data shift circuit that holds data of a certain length, and a preset weighting coefficient and the data held in the data shift circuit are used. A spatial filter consists of a multiplication circuit that performs multiplication with , an addition circuit that adds all the results, and a division circuit that divides the level increased by multiplication and addition so that it falls within an appropriate range. a smoothing calculation means for reducing random noise superimposed on the image data; a second-order differential calculation means for emphasizing sudden changes in brightness of the image data using a similarly configured spatial filter; The present invention is characterized in that it is equipped with an extreme point extracting means for extracting an extreme point by comparing the gray levels of adjacent pixels from the data obtained by the processing, and extracts a rough outline of a multi-level gray image as a binary image. An image processing method that