JPH05342350A - Picture processor - Google Patents

Abstract

PURPOSE:To perform the labeling processing which does not put restrictions on the number of temporary labels with a small amount of hardware by using a CAM as the temporary label picture memory. CONSTITUTION:The difference processing between an inputted picture signal and a picture signal stored in a frame memory part 21 is performed. The separated picture signal is binarized by a binarization processing part 23, and the background area and the object area are separated. A processing mask acts on the seapared binary pictures raster-like by a temporary labeling processing part 7 to determine a temporary label number of a picture element X. A temporary label rewrite instructing part 25 discriminates size relations of temporary label numbers stored in a connection relation storage part 24 and inputs the instruction, which rewrites larger label numbers with smaller label numbers, to a CAM part 26. The CAM part 26 rewrites all temporary label numbers stored on the CAM based on the rewrite instruction.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is a process for binarizing an image picked up by a television camera or the like, separating a background region and an object region, and giving a label number to the closed region of the separated object in real time. The present invention relates to an image processing apparatus that performs (labeling processing).

[0002]

2. Description of the Related Art Image processing apparatuses that perform labeling processing can be classified into three types: raster scan type, run code type, and boundary tracking type. In realizing an image processing apparatus, a raster scanning type is generally used because of the possibility of real-time processing, the circuit scale, the ease of realization, and the like. Therefore, after the raster scanning type labeling processing algorithm is described, the conventional technique of the image processing apparatus for raster scanning type labeling processing using the algorithm will be described.

Increasingly numbering by labeling process
The closed region to be described will be described. First, the labeling process
Before, the image is binarized and separated into background area and object area
Then, the background area is set to “0” and the object area is set to “1”. Closure
Area is one thing connected at the connection point described below.
The body area. Figure 8 shows 4 neighborhoods, 4 neighbors or 8
It is a figure explaining a neighborhood and 8 adjacent points. Where
Pixel x₀Consider a 3 × 3 block centered on (i, j)
It At this time, x₀A set of pixels {x₁, X ₃,
x_Five, X₇} X₀4 neighbors or 4 adjacent points of
A set of pixels {x ₁, X₂, X₃, X_Four, X_Five,
x₆, X₇, X₈} X₀8 neighbors or 8 adjacent points
U For example, if the object region after binarization is “1”,
There is a region of "1" on the 4 or 8 adjacent points.
For example, connect pixel x to a point on the 8th adjacent point or 4th adjacent point.
The set of connected points is a closed region.

Next, a labeling algorithm of the raster scanning type labeling process will be described. In general, the raster scanning type labeling process scans in a raster manner starting from the upper left corner of the image. A 3 × 3 mask as shown in FIG. 8 is required to detect the above-mentioned 4 adjacent points or 8 adjacent points, but in the ordinary labeling processing, it is only necessary to search for the points for which labeling processing has been completed.

FIG. 9 is a view showing a processing mask used for the provisional labeling processing. Where X is the point of interest, a, b,
c and d are search points at 8 adjacent points. In the case of four adjacent points, only a and c need to be searched. Below, 8
The search for adjacent points will be described.

FIG. 10 is a diagram for explaining a raster scanning type labeling processing algorithm. 1 is an input binary image, 2 is a temporary label image, 3 is a connection relation table Ta,
Reference numeral 4 is a lookup table LUT, and 5 is a final label image.

Scanning is started in a raster form, and it is determined whether the pixel of interest is the background area or the object area. When the target point is the object region, that is, “1”, the pixels a, b, c, and d points before and after the target point in FIG. 9 are searched, and if the pixel of “1” is detected among them, the target point is determined. It is determined that the pixels are in the same region, and the same number as the label number of the detected pixel is provisionally given. If the pixel of "1" is not detected, a new provisional label number obtained by incrementing the conventional label number is given as a new area and recorded in the provisional label image. Here, when different label numbers are given to the points a, b, c, and d, these labels belong to the same closed region, and therefore this relationship is concatenated with, for example, “2” and “4”. However, "4" is connected to "5", "3" is connected to "2", and so on, and is recorded in the label connection relation table Ta3. Perform this process one after another,
Finally, the provisional label image 2 in which the provisional labels are given to all the pixels is completed.

Next, using the label concatenation relationship table Ta3, the label numbers in the same area are set as, for example, "1" is independent of "1" and "2" to "5" are "2". As a grouping, "6" is independent as "3" ... Clustering process for grouping labels is performed, provisional labels having a connection relationship are grouped into one label, and the grouped label number is looked up. Record in table LUT4. Finally, the look-up table LUT4 is used to update the provisional label number to the final label number. In this way, the final label image 5 having a closed region having the same label number formed by a set of 8 adjacent points is obtained.

Usually, the clustering process using the label connection relation table requires software processing, and its processing time is proportional to the square of the provisional label number L (*), which is a factor that hinders real-time processing. Was there. Therefore, the conventional image processing apparatus using the raster scanning type labeling processing reduces the provisional label number L (*), devises a clustering processing algorithm,
There are two points. A conventional image processing apparatus that has performed these two points will be described below.

As a conventional image processing apparatus for performing raster scanning type labeling processing, an image processing apparatus using a pipeline type high speed labeling algorithm (“pipeline type high speed labeling algorithm”, Computer Processing Research Report of Information Processing Society of Japan 43). -1, 1986; "Video Rate Labeling Processor", Information Processing Society of Japan, Computer Vision Research Report 54-1'98). In order to perform real-time processing, the apparatus performs dilation processing on the image as pre-processing for labeling to reduce the number of provisional labels, and then uses graph theory to reduce the calculation amount of clustering processing.

The dilation processing and the clustering processing using graph theory will be described below. FIG. 11 is a diagram showing a configuration of a labeling processing unit of a conventional device, 6 is a preprocessing unit, 7 is a temporary labeling processing unit, 8 is an image restoration unit, 9 is an image data delay unit, 10 is a label updating unit, Reference numeral 11 is a connection relation organizing unit, and 12 is a clustering processing unit.

First, the binarized image data is input to the preprocessing unit 6, and the recessed portion of the input image is expanded in order to reduce the number of provisional labels. Specifically, a 3 × 3 mask is operated to set the value of the central pixel to “1” when the value of the central pixel is “0” and when adjacent regions are not connected due to the combination of peripheral pixels. FIG. 12 shows the arithmetic logic of the expansion process. Here, x ₀ ... x ₈ correspond to each pixel of the 3 × 3 mask shown in FIG. 8, and * has a value of “0” or “1”. If the input is one of the patterns shown in the figure, the output of x ₀ is set to “1”, and the output of x ₀ is set to “0” otherwise.
And so on. After the provisional label is given by the provisional labeling processing unit 7, the expanded regions are restored in shape by taking a logical product with the binary image before the expansion processing in the image restoration unit 8.

In the provisional labeling processing unit 7, the binary image expanded by the preprocessing unit 6 described above is input, and provisional labeling is performed by the labeling process described above. At the time of labeling, the connection relation of each pixel is detected and recorded in the label connection relation table Ta. After the provisional labeling process is performed on all the pixels on one screen, the label connection relation table is subjected to processes such as creation of a pointer table for clustering process in the connection relation organizing unit 11. As described above, in the normal clustering process, the calculation amount increases in proportion to the square of the provisional label number L (*). Therefore, the conventional device reduces the calculation amount of the clustering process by using the graph theory. ing. In the graph theory, when the provisional label L (*) is regarded as a node and the connection relation Cm is regarded as an arc, there are the following relations.

Cm ≦ 3L (*)-6 From the above relationship, the number of arcs is proportional to the number of nodes, that is, the provisional label number L (*).

Next, the created pointer table and label concatenation relationship table are input to the clustering processing unit 12, the clustering processing is performed with an amount of calculation proportional to the provisional label number L (*), and a lookup table for update is used. LUT4 is created. Finally, the look-up table LUT 4 is input to the label update unit 10, and the image data delay unit 9 updates the temporary label number of the temporary label image delayed according to the time of the clustering process.

[0016]

In the conventional image processing apparatus described above, in order to reduce the provisional label number L (*), the preprocessing unit 6 performs expansion processing, and the expansion image obtained is subjected to expansion processing. Since the provisional labeling process is performed by performing the logical product of the provisional label image and the binary image before the expansion process to restore the provisional label image, a large amount of processing is applied to the preprocessing unit 6. I need. Since this processing is basically pipelined, real-time processing is possible, but there is a drawback in that the amount of hardware for that processing becomes large. Even if the graph theory is used, the amount of processing is basically the provisional label number L
Since it is proportional to (*), L (*) cannot be increased. For this reason, L (*) is reduced in the preprocessing unit, but the provisional label number L
(*) Has a drawback that it can only process up to about 2000. According to the conventional method, the provisional label number L (*) is 20
When the number exceeds 00, the input image is output as it is without performing the labeling process as an overflow, so that a desired label image cannot be obtained.

The present invention uses a graph theory in the clustering process, which is disadvantageous in that the hardware amount of the preprocessing unit for reducing the provisional label number L (*) in the conventional image processing apparatus for labeling process becomes large. However, the processing amount is proportional to the provisional label number L (*), and the provisional label number L * is 20
The purpose of the present invention is to solve the drawback of having to limit the number to around 00 and to enable labeling processing that does not limit the number of provisional labels with a small hardware.

[0018]

FIG. 1 is a block diagram showing the principle of the present invention. Reference numeral 7 in the figure is a provisional labeling unit, 24
Is a connection storage unit, 25 is a temporary label rewriting instruction unit,
Reference numeral 26 denotes a CAM unit, and 27 denotes a label ascending order reassignment instruction unit.

The present invention can perform only the provisional label number L (*) = 2000 or so even if the preprocessor which has been a factor of increasing the hardware of the conventional apparatus and the graph theory for the real time processing are used. Since the clustering processing unit that does not exist is omitted, the CAM (Content Addressab
Le Memory) is the biggest feature.

[0020]

In the case of the present invention, the rewriting of the label is performed at a high speed by utilizing the characteristic of the CAM. The conventional technique is that there is no pre-processing unit for reducing the number of provisional labels L (*), no clustering unit for organizing connection relations and connecting provisional labels, and using CAM instead of normal RAM for provisional label image memory. Use, instead of performing all the provisional labels after the provisional label is rewritten to the actual label in the clustering processing unit, while provisioning the provisional label, the provisional label number on the CAM at the time when the connection relation is extracted All are different at the same time, rewriting to a label number having a connection relationship.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 2 is a diagram for explaining the structure of an image processing apparatus for carrying out the labeling processing of the present invention. In the figure, reference numeral 7 is a provisional labeling processing unit, 20 is an A / D conversion unit, 21 is a frame memory unit, 22 is a differential signal processing unit, 23 is a binarization processing unit, 24 is a connection relation storage unit, and 25 is A tentative label rewriting instruction unit, a CAM unit, and a label ascending order reassignment instruction unit 27 are provided.

First, the features of CAM will be described. In addition to the normal memory (RAM) function, the CAM is a memory capable of collating and searching the contents of the stored data of each word with respect to the input search data and accessing the word having the corresponding contents. One of its characteristics is the parallel writing function to the selected word. When CAM is used as a label image memory for labeling processing,
When the connection relation is detected, all the labels to be connected can be updated at the same time by using the function. That is, the update process can be performed by assigning the temporary label numbers and replacing all the large temporary label numbers among the numbers which are found to be linked on the CAM with the small label numbers when the connection relation is found. The labeling process using the CAM will be described below with reference to FIG.

First, the input image is, for example, 512 × 5.
Consider a labeling process at 8 adjacent points with an image of 12 pixels. At this time, the provisional label number L determined by the 8 adjacent points
The maximum number of (*) is 512 × 512/4 = 66k. C
When AM is used as a label image memory for labeling processing, its capacity is 512 × 512 words and 16 bits are required.
It is 4.2 Mbits. The image signal is A / D converted by the A / D conversion unit 20, and stored in the frame memory unit 21 for one frame. Next, difference processing is performed between the input image signal and the image signal stored in the frame memory unit 21. The separated image signal is binarized by the binarization processing unit,
The background area and the object area are separated. The separated binary image is processed by the provisional labeling unit 7 in a raster manner using the processing mask shown in FIG. 9, and the provisional label number of the pixel X is determined. FIG. 3 is a logical table for obtaining a provisional label to be given to the pixel X at the eight adjacent points shown in claim 3,
A, B, C, and D are provisional label numbers on the CAM assigned to the pixels {a, b, c, d}, respectively.

The provisional label number of the pixel X is obtained according to this table, and the provisional label number is assigned to the CAM section 26. In the case of items 3 and 4 in FIG. 3, pixels are connected, and the label numbers B and D, and B and C belong to the same closed region. FIG. 4 shows the input binary image “1” in FIG.
It is a figure which shows the temporary label image (contents of CAM) in the middle of being labeled with respect to, and the connection relation is detected by the point of X. In FIG. 4, B and C, that is, the label numbers “2” and “4” shown in the figure show that they have a connection relationship and belong to the same closed region, and this relationship is input to the connection relationship storage unit 24. .. In the case of the present embodiment, the label number “2” which is the value of B is given to the attention point X.

Next, the fact that the provisional label numbers "2" and "4" are in the connection relation is stored in the connection relation storage unit 24,
At the same time, the provisional label rewriting instruction unit 25 is instructed to rewrite the provisional label based on the connection relation stored in the connection relation storage unit 24. Specifically, the tentative label rewriting instruction unit 25 determines the size relation of the tentative label numbers stored in the connection relation storage unit 24, and issues an instruction to replace the larger label number with the smaller label number. 26. That is, an instruction to replace all the provisional label numbers “4” accumulated on the CAM with “2” is issued. Based on the rewriting instruction, the CAM unit 26 rewrites all the temporary label numbers “4” stored in the CAM to “2”.

Specifically, first, the address of label number "4" is detected from the data stored on the CAM.
Next, the label number "2" which is the rewriting destination data is set in the CAM. Finally, all the detected data of the address part (that is, the label number "4") is rewritten to "2". As described above, the provisional label number “4” is all rewritten to “2”. The rewritten label image is shown in FIG. If these processes are carried out for all the pixels, the finally obtained label image becomes the updated label image as it is.

FIG. 6 is a logical table for obtaining a provisional label to be given to the pixel X at the time of labeling processing at four adjacent points centering on the point of interest X described in claim 4. Similar to the logic table shown in FIG. 3, the provisional label is given to the attention point X. Further, the processing when the connection relation is detected is the same as the labeling processing at eight adjacent points.

As for the label number of the obtained label image, one label number is given to each closed area, but the order is not continuous. Usually, the labeling process only needs to be given a unique label number that can distinguish each closed region, and its size and continuity do not matter. However, depending on the purpose of the image processing, the size and continuity thereof may be important. In the case of claim 1, an appropriate label number as the actual label number is used as the representative value. In this case, unique real label numbers are provided discontinuously to each closed area. Further, in the case of claim 2, the assigned real label number starts from “1” and continues discontinuously in order to set the smallest label number among the detected label numbers having the connection relation as the representative value. It will be.
This is effective when it is necessary to specifically recognize the label number “1”, such as when tracking an object with the actual label number “1”. Further, in the case where each closed area is classified based on the assigned label number, if the actual label numbers are arranged in ascending order and continuously arranged, the processing such as the recognition processing after the labeling processing becomes easy. An embodiment of claim 5 in which the assigned label numbers are continuously re-assigned in ascending order will be described below.

The above-mentioned temporary label number rewritten by the label rewriting instruction unit 25 is inputted to the label ascending order reassignment instruction unit 27, and based on the temporary label number to be rewritten, the label number in the rewriting label number flag table FTa is increased. Flag. At the time when all the labeling processes are completed, the unlabeled label number in the rewriting label number flag table FTa is recorded as the final label number on the CAM. Therefore, based on the rewriting label number flag table FTa, the unflagged numbers are called from the CAM in ascending order, and C
A process of rewriting the label number of AM to an ascending number is performed.

FIG. 7 is a diagram showing a flow of processing for rewriting label numbers in ascending order. Here, L (*) is the number of provisional labels, FTa (i) is the content of the first flag of the provisional label, and m is the label number arranged in ascending order. First, m = 1 and i = 1 are set as initial values (S1), and i
Starting from = 1, the contents of the flag of FTa (i) are searched (S2, S3). If the flag is "1", the provisional label number has been rewritten, and since the label number does not exist on the label image, i is incremented by 1 and the same process is performed (S4). If the flag of FTa (i) is 0, the label number exists on the label image, so the number is rewritten in ascending order. First, CA
All the label numbers are searched from M (S5), and the label numbers of the corresponding addresses are all rewritten to m (S5).
6). Then, m and i are incremented by 1 and the process is repeated (S7). When this process is repeated L (*) times, CA
All label numbers stored on M are rewritten in ascending order.

Since the configuration is as described above, the CAM
Using the search function of and the parallel writing function to the selected word, the temporary label number is given, and the label number which is found to be connected on the CAM when the connection relation is found is replaced by the label number Update processing can be performed, and as a result, a clustering process that sorts and concatenates the connection relations detected when assigning a provisional label number, a preprocessing unit for reducing provisional label numbers, as in the conventional method. No need for parts, labeling processing is possible without limiting the number of provisional labels,
Produces the effect of. As is clear from this result, since the preprocessing unit and the clustering processing unit of the conventional method are not required, the hardware can be downsized. Further, unlike the conventional method, it is possible to perform the labeling process up to the theoretically maximum number of provisional labels without limiting the number of provisional labels for real-time processing.

[0032]

As described above, according to the present invention,
A label number that has a linking relationship on the CAM when the linking relationship is found while using the CAM as the label image memory and using the CAM search function and the parallel writing function to the selected word while assigning the temporary label number The label number update process can be performed by substituting for, and as a result, the preprocessing unit for reducing the provisional label number as in the conventional method, the connection relation detected when the provisional label number is assigned Since there is no need for a clustering processing unit for organizing and clustering, there is an advantage that the hardware can be downsized. Moreover, there is an advantage that the labeling process can be performed up to the theoretically maximum number of provisional labels without limiting the number of provisional labels for real-time processing.

[Brief description of drawings]

FIG. 1 shows a principle configuration diagram of the present invention.

FIG. 2 is a diagram illustrating a configuration of an image processing apparatus that performs labeling processing according to the present invention.

FIG. 3 shows a logical table for obtaining a provisional label to be given to a pixel X at eight adjacent points.

FIG. 4 shows a provisional label image (contents of CAM) in the process of being labeled with respect to the input binary image “1”.

FIG. 5 shows a rewritten label image.

FIG. 6 shows a logical table for obtaining a provisional label to be given to a pixel X at the time of labeling processing at four adjacent points around the point of interest X.

FIG. 7 is a diagram showing a flow of processing for rewriting label numbers in ascending order.

FIG. 8 is a diagram illustrating 4 neighbors, 4 neighbors or 8 neighbors, 8 neighbors.

FIG. 9 shows a processing mask used for provisional labeling.

FIG. 10 is a diagram illustrating a raster scanning type labeling processing algorithm.

FIG. 11 is a diagram showing a configuration of a labeling processing unit of a conventional device.

FIG. 12 is a diagram showing the arithmetic logic of expansion processing.

[Explanation of symbols]

 1 Input Binary Image 2 Temporary Label Image 3 Concatenation Relation Table Ta 4 Lookup Table LUT 5 Final Label Image 6 Pre-Processing Section 7 Temporary Labeling Processing Section 8 Image Restoring Section 9 Image Data Delay Section 10 Label Updating Section 11 Connection Relationship organizing unit 12 Clustering processing unit 20 A / D conversion unit 21 Frame memory unit 22 Difference signal processing unit 23 Binarization processing unit 24 Concatenation relation storage unit 25 Temporary label rewriting instruction unit 26 CAM unit 27 Label ascending order reassignment instruction unit

Claims (5)

[Claims] 1. In an image processing apparatus for performing an image labeling process, if an input point is a pixel for which labeling is not to be performed on an input image, processing is not performed and the pixel is moved to the next pixel to perform labeling. If it is a pixel, a means for provisionally providing a label number, a means for storing a connection relation of the label when a connection relation is detected when the label number is provisionally provided, And a means for storing the label number in the CAM, and when storing the provisionally assigned label number in the CAM, if a connection relationship is detected, the storage processing is continued based on the stored connection relationship. , One of the stored temporary label numbers in the linked relation is used as a representative value as a real label number, and the other temporary label numbers linked to the representative label number are used as CAM. And et retrieval, the image processing apparatus characterized by having a processing means for performing an instruction to rewrite the tentative label number that is the search in all simultaneously surrogate table label number. 2. The provisional label number is C
When the connection relation is detected when storing in the AM, following the storage process, based on the stored connection relation,
Of the stored temporary label numbers in the linked relationship, the smallest temporary label number is used as a real label number as a representative value, and other label numbers linked to the representative label number are used as CAMs.
2. The image processing apparatus according to claim 1, further comprising processing means for instructing to rewrite all of the retrieved temporary label numbers with the minimum representative label number at the same time. 3. A label number is tentatively assigned to the attention point, and when the connection relation of the labels is stored when the connection relation is detected, among the eight adjacent points around the attention point, The point of interest is X (i, j), and the four points already scanned are a (i, j-1), b (i + 1, j-1), and c (i-, respectively.
1, j), d (i-1, j-1), already scanned,
Labeled label numbers are A, B, C,
D, and the values (a, b, c, d) of the four points already scanned are (1, *, *, *) (where * represents 1 or 0)
In the case of, the provisional label number of the attention point X is given A, and in the case of (0, 1, 0, 0), the provisional label number of the attention point X is given B, (0, In the case of (1, 0, 1), B is given as the provisional label number of the point of interest X, and at the same time, the connection relationship between B and D is stored, and in the case of (0, 1, 1, *), B is given as the provisional label number of the attention point X, and at the same time, the connection relationship between B and C is stored. In the case of (0, 0, 1, *), C is given as the provisional label number of the attention point X. In the case of (0, 0, 0, 1), D is given as the provisional label number of the attention point X, and in the case of (0, 0, 0, 0), the attention point X As a provisional label number, the provisional label number is provided with a processing means for assigning a number obtained by adding 1 to the maximum number of provisional label numbers. The image processing apparatus Motomeko 1 or 2 wherein. 4. When a label number is tentatively assigned to the point of interest and the linking relation of the labels is stored when the linking relation is detected, among the four adjacent points with the focus of the attention point, Let X (i, j) be the point of interest, and let a (i, j-1) and c (i-1, j) be the two points that have already been scanned, and label numbers that have already been scanned and labeled, respectively. A and C. If the values (a, c) of the four points already scanned are (1, 0), A is given as the provisional label number of the point of interest X, and (0, 1) Is assigned C as the provisional label number of the attention point X, and in the case of (1, 1), A is assigned as the provisional label number of the attention point X, and at the same time, the connection relationship between A and C is stored. However, in the case of (0, 0), the provisional label number of the attention point X is the highest among the provisional label numbers assigned so far. The image processing apparatus according to claim 1, wherein further comprising a processing means for applying the added number 1 to number. 5. The image according to claim 1, 2, 3 or 4, further comprising means for re-assigning the label numbers stored in the CAM in ascending order continuously after the search of all pixels is completed. Processing equipment.