JPH04148283A - Separation processing method for mobile object by clustering - Google Patents

Abstract

PURPOSE:To discriminate an individual object even when the plural objects are displayed to be overlapped by extracting an optical flow at every picture element on a picture, clustering on a velocity space, applying a label to the picture element, and extracting a mobile object corresponding to the picture element. CONSTITUTION:Based on the difference between picture elements 1A and 1B, the optical flows of all the picture elements on the picture are extracted. Next, concerning the obtained clustering, the clustering is executed on the velocity space, the label is applied to the picture element corresponding to grouped optical flow groups, and the mobile object corresponding to the picture element to which the same label is applied is separated from other mobile objects and extracted. Therefore, rectangular bodies 3 and 4 are extracted to be separated as the rectangular body 3 moves upward and the rectangular body 4 moves in the right direction. Thus, concerning the objects displayed to be overlapped on the picture, it is possible to separate, to discriminate and to know the moving state.

Description

[Detailed Description of the Invention] [Summary] The present invention extracts an optical flow corresponding to the temporal movement of an object shown in one image, and separates and determines the movement of each moving object. Regarding the separation processing method of moving objects using clustering, we aim to identify two individual objects even when multiple objects are displayed overlapping each other on an image. Optical flow extraction processing process.

a clustering process for clustering the obtained optical flows in velocity space;

The present invention is configured to have a pixel grouping process that gives labels to pixels corresponding to clustered optical and flow groups, and a pixel group output process that extracts moving objects corresponding to pixels that have been given the same label.

[Industrial application field]

The present invention relates to a method for separating moving objects using clustering, which extracts optical flows corresponding to temporal movements of objects depicted in one image, and separates and determines the movements of each moving object.

Separating objects by their differences in motion from an image is
This technology is essential for robot eyes in factory automation (FA) and object identification for unmanned vehicles.

[Conventional technology]

Separating a moving object from an image is to separate the objects in the image from a given moving image based on the difference in motion.For example, when the image shown in Figure 1O is given as input, r The circle has moved to the upper right. One rectangle is moving up. Another rectangle is to get the output j moving to the right.

Here, the arrows in FIG. 1O represent the movement (optical flow) between images for each pixel. The optical flow calculation method is a known technique.

In reality, all pixels are densely calculated, but they are displayed after being thinned out by one. Furthermore, calculation of optical flow involves a considerable amount of error.

In the conventional technology, in order to avoid calculation errors in optical flow, a method called labeling is used to roughly separate each object, and processing is performed by averaging the optical flow for each object. FIG. 11 shows the processing results.

[Problem to be solved by the invention]

Objects that appear to overlap on one image as shown in Figure 11 cannot be separated by the above labeling.For this reason, the circle in Figure 11 is calculated to have moved to the upper right, but 1 The two overlapping rectangles are treated as one object, and are moved as a whole to the upper right. is calculated incorrectly.

In this way, in the conventional technology using labeling, 1
It was not possible to separate objects by movement regarding overlapping objects.

An object of the present invention is to enable individual objects to be identified even when a plurality of objects are displayed overlappingly on three images.

[Means to solve the problem]

FIG. 1 shows the principle configuration diagram of the present invention, and the reference numeral IA in FIG.
Each IB is an image and represents an image in a temporally shifted state. 2 represents a circular object, and 3 and 4 each represent an image of a rectangular object.

Processing ■Niobtical flow extraction processing step 101, in which the optical flow for all pixels on the image is extracted based on the difference between the iI images IA and IB (although it does not have to be for all pixels). (the pixels to be processed are averaged and densely selected).

Processing ■: Clustering processing step 102.

Regarding the clustering obtained above, clustering is performed on the velocity space. In other words, in this case, considering the movement on a 52-dimensional plane, the tips of the arrows representing each optical flow are plotted on the XY-axis plane. By grouping.

Processing ■: 8-pixel grouping processing step 103.

Labels are given to pixels corresponding to the grouped optical flows.

Process (2): yti prime group output processing step 104, in which moving objects corresponding to pixels given the same label are extracted while being distinguished from other moving objects. The illustrated rectangular bodies 3 and 4 are:
Rectangular body 3 moves upward.

The rectangular body 4 is distinguished and extracted as moving to the right.

[For production]

An optical flow is obtained by associating one image IA and IB for each pixel corresponding to the circular body 2 and the rectangular bodies 3 and 4 (process ①).

For the pixel corresponding to the circular body 2, in the example shown in Figure 1, an optical flow is obtained in the diagonally upper right direction (represented by an arrow, it is in the diagonally upper right direction), and for the pixel corresponding to the rectangular body 3, an optical flow is obtained. , an upward optical flow is obtained. For the pixels corresponding to the rectangular body 4, a rightward optical flow is obtained (processing ■).

Each optical flow is represented by an arrow.

When displayed by plotting the tip of the arrow on the XY axis plane, the plot corresponding to the pixel of 1 circular body 2 is
It is distributed in an area corresponding to a line approximately 45@ with respect to the axis. Further, the plots corresponding to the pixels of the rectangular body 3 are distributed in corresponding regions on the Y axis.

Similarly, the plots corresponding to the pixels of the rectangular body 4 are distributed in corresponding regions on the X axis. by this.

Each grouped pixel is obtained, and the same label is given to the pixels in the same group (processing ■).

Determine the object corresponding to the pixel given the same label,
For example, the optical flow for each pixel constituting the object is averaged and determined as the motion corresponding to the object (ij element group output processing process).

〔Example〕

A method of calculating optical flow from a given moving image is a known technique. For more information.

Hildreth, E.C., The 5eas
uresent of VisualMotion
, MIT Press, Cambridge+M
ass, + 1984° (Hildreth, E, C
, Author, Measuring Visual Movement Kemprich, Massachusetts Institute of Technology Press.

However, to be precise, the optical flow should be given in the form of a vector such as (Vx, Vy) for each pixel.

Next, clustering will be explained. This is a statistical method, and is a general term for methods for classifying observed data containing errors into several classes. The simplest example is dividing toy blocks into two boxes based on their shape, which is one type of clustering.

For example, in Figure 1O, optical flow is represented by arrows in correspondence with pixels on each object, but each arrow is plotted on the XY-axis plane as shown in Figure 2. . Since the optical flow corresponds to the moving speed of the object, the process of plotting this will be referred to as clustering in the velocity space.

In the plotted state shown in FIG. 2, plot points that are considered to exist in approximately the same area are grouped together. FIG. 3 represents this situation, and labels are given to plot points corresponding to each optical flow.

This is the same as giving labels to two pixels.

FIG. 4 shows a state in which labels are given to element iij. As shown in FIG. 4, objects 2.3.4 corresponding to pixels with the same label are identified as separate objects.

FIGS. 5 and 6 are diagrams for explaining the process of outputting an image representing the movement of an object.

As shown in FIG. 4, the center of gravity of pixels with the same label is determined, and a label corresponding to the center of gravity is written as shown in FIG.

On the other hand, as described above, the average value of the optical flow for the pixels with the same label is calculated.

Then, as shown in FIG. 6, arrows corresponding to the average values are displayed from the center of gravity of each object 23.4 separated by nine.

Of course, instead of calculating the average value, an arrow giving the maximum optical flow may be displayed at the center of gravity position.

The extraction of optical flows and the clustering method will be explained in more detail below. For the extraction of optical flows, for example, see "New Technology" Communications Publishing / Special Edition Q plus E Image Processing Algorithm If a pixel on an image a is represented by data that gives its brightness, an image is provided with two-dimensionally arranged data as shown in FIG. 7.

Assuming that the image on the left side of FIG. 7 is the earlier image in time, and the image on the right side is the later image in time, the optical flow is calculated from these two images.

Then, the optical flow can be calculated using the following procedure.

(1) Initialization: Set the origin at the top left as the pixel of interest (2)
Operation: Search for a pixel that has the same value as the value of the image 5 hours before the current pixel of interest from the image that is 5 hours later.

(3) Write: Write the number of moves from the current pixel of interest in the table separately for the X and Y components. (4) Move: Move the current pixel of interest one step to the right, move to the right. If not, move to the left end of 1-crotch. ii
! It ends when you reach the bottom right of the J statue.

(5) Loop: Calculate again. FIG. 8 shows the result of performing the above procedure on the one shown in FIG. In the case shown in FIG. 8, it is shown that the pixel with the value "20" shown in the left side of FIG. It is shown that there is no movement in the axial direction. The same applies to the two pixels having the value "10" shown in the diagram shown in FIG.

In this way, optical flow is extracted corresponding to each pixel.

When performing clustering, the optical fucoid (v,,v,>) of each input pixel is used.

(V, (1), Vy(1)), (Vll fil,
y, (!l)... (V, (-1, Va(
″)), the processing is performed as follows.

(6) Set the variable i that stores the initialization number to 1.

minutes Empty the area that stores categorized classes.

(7) Operation = i-th input (■, full, V,
ill ), find a class whose distance from it is smaller than the specified value, the distance between the class and the input is ■, (1) - X component of current class average] 2 + V, 01
- Let the Y component of the current class average be 1χ.

(8) Writing: If a class is found by the above calculation, classify the i-th input into that class. If not found, a new class is created and the 51st input is classified into the new class.

(9) Repeat: If i is n, end. If not, the value of i is increased by I and the calculation is performed again.

FIG. 9 shows the configuration of an embodiment of the present invention.

Reference numeral 11 in the figure is an image input unit that reads an image.
Reference numeral 2 denotes an optical flow calculating section which extracts an optical flow as explained with reference to FIGS. 7 and 8 above. Reference numeral 13 denotes a clustering unit, which performs clustering by the processes shown in the above processes (6) to 9). Reference numeral 14 denotes a pixel-by-pixel grouping unit, which groups pixels with the same label. Reference numeral 15 denotes a display section 1, which performs display as described with reference to FIG. 6, for example.

〔Effect of the invention〕

As explained above, according to the present invention, it is possible to separately identify objects that are displayed overlappingly on N images and to know the state of movement.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the principle configuration of the present invention, FIGS. 2 to 6 are explanatory diagrams explaining the processing in the present invention, and FIGS.
9 is an explanatory diagram for explaining processing related to optical flow, FIG. 9 is an explanatory diagram for explaining the configuration of an embodiment of the present invention, and FIGS. 10 and 11 are explanatory diagrams for explaining the prior art, respectively. In the figure, IA and IB are images, 2 is a circular object, 34 is a rectangular object, 101 is an optical flow extraction process, 10
2 represents a clustering process, 103 represents a pixel grouping process, and 104 represents a pixel group output process.

Claims (3)

[Claims] (1) The system is configured to extract optical flows corresponding to the movement of the moving object based on two temporally shifted images in which the moving object is shown, and the optical flows for each object are averaged. In the moving object determination processing method in which the movement of each object is determined by determining the movement of each object, an optical flow extraction processing step of extracting the optical flow corresponding to the pixel corresponding to the object for all objects on the image. (101); a clustering process (102) for clustering each of the optical flows on the velocity space based on the extracted optical flows;
A pixel grouping process (103) in which labels are given to pixels corresponding to a flow group, and a pixel group output process (104) in which moving objects corresponding to pixels given the same label are extracted while being distinguished from other moving objects. A method for separating and processing moving objects by clustering, characterized in that the following steps are performed sequentially. (2) In the pixel group output processing step (104), the average value of the optical flow for each pixel is calculated for the pixel group given the same label, and the display corresponding to the average value is displayed for the pixel group. 2. The method of separating and processing moving objects by clustering according to claim 1, wherein the moving object is output in association with the clustering. (3) In the pixel group output processing step (104), the maximum value in the optical flow for each pixel is extracted for the pixel group given the same label, and the display corresponding to the maximum value is 2. The method for separating moving objects by clustering according to claim 1, wherein the moving object is output in association with a pixel group.