KR20030044437A - An object recognition method via network control - Google Patents

Abstract

PURPOSE: An image object recognizing method based on a control network is provided to extract an object and control a camera in real time using a motion vector. CONSTITUTION: Video information captured by a plurality of cameras constructing a network is received(S1). A stream header from the received video information is analyzed to extract a motion vector(S2). Competition learning and neural network learning with respect to the motion vector are performed(S3). Average and deviation values of pseudo weight vector are computed(S4). An index function value is calculated(S5). It is judged whether or not the calculated index value exceeds a threshold(S6). A separate object is generated or the object of the captured video information is merged with existing objects according as the index value exceeds the threshold or not(S7-1,S7-2). A representative vector of the object and image position are calculated(S8). A camera target focus point is obtained according to the calculated values to output a signal for controlling the postures of the cameras(S9,S10).

Description

An object recognition method via network control

The present invention relates to a method of extracting and recognizing an image object on a control network and a control technique of a plurality of cameras using the method. More specifically, the object extraction technique and the camera control technique applied to the present invention are a multimedia content technology that is rapidly developing recently. It is an original technology based on intelligent information processing technology represented by the network and neural network. Instead of the video signal processing method based on the still image processing technology up to now, it uses the motion vector basically provided in video processing in real time. The present invention relates to a method for realizing object extraction and camera control.

Conventional methods typically extract image objects using image post-processing. Existing techniques use image object extraction by post-processing after image restoration, which requires a lot of computation time, making high-speed processing difficult. In other words, the previous intelligent surveillance system replaced the video information with a still image and used the existing image processing method, so the algorithm was very large. The price of is very high, and due to the knowledge-intensive technical difficulty, it is commercialized only in a few countries in developed countries.

An object of the present invention is to provide a method for realizing object extraction and camera control in real time using a motion vector basically provided in video processing.

Another object of the present invention is to provide a method for recognizing an image object that can perform camera control through high speed processing of a video.

Still another object of the present invention is to provide an object recognition method which is easy to process in real time by greatly reducing the amount of computation of a processor in processing video information.

It is still another object of the present invention to provide an object recognition method capable of directly extracting a complex object from video information by repeating a simple operation using a neural network technique.

Another object of the present invention is to provide a method of controlling a camera by extracting an image object through a competitive learning neural network using a motion vector represented by video encoding without a separate decoding process.

In accordance with an aspect of the present invention, there is provided a method of recognizing a video object, the process of receiving image information acquired by a plurality of cameras in a network, and analyzing a stream header from the received image information to obtain a motion vector. Extraction process, competition learning and neural network learning for motion vectors, calculating mean and deviation values of pseudoweight vector, calculating index function value, threshold of calculated index value The process of determining whether there is an excess, the process of creating a separate object or merging into an existing object according to the excess, calculating a representative vector and image position of the object, and calculating a camera target focus position based on the calculated value And outputting a signal for controlling the posture of the camera.

1 shows an actual image coming in at time t and t + h,

2 shows motion vector information in an MPEG stream shown as an image;

3 is an object extracted through the object extraction algorithm,

4 is a disturbed object larger than the Threshold value,

5 is an object region extracted by variance,

6 is an image when a plurality of objects exist,

FIG. 7 illustrates a case in which a plurality of objects are present, and objects are distinguished by three weight vectors.

8 shows merging of objects using neural networks;

9 shows two objects located on the screen;

10 is an average value position of each object,

11 is a representative vector of the selected object,

12 is a control signal extraction using a representative vector,

13 is a representative vector allocation for individual cameras in video sharing;

14 is a control concept by assigning a representative vector for a plurality of cameras,

15 is a control conceptual diagram of a camera system,

16 is a flowchart illustrating a method of recognizing an image object according to the present invention.

Hereinafter, a control network based image object recognition system according to the present invention will be described with reference to the accompanying drawings.

The object recognition method according to the present invention is made around a system in which a plurality of cameras are controlled based on a network.

For development, it is necessary to extract the object to be monitored through the motion vector in the obtained MPEG stream. In general, it is assumed that an image captured by a camera is the same as that of FIG. 1. In FIG. 1, there is a change in the image of the moving person between the times t (a) and t + h (b), but there is no change in the image at the back background. Therefore, if you configure a surveillance system for a certain area, if an intruder or unauthorized user enters a certain area, the video information of the specific area in the background will not change. However, since the image corresponding to the intruder causes a lot of changes, if the appropriate change is extracted and processed, the image object recognition system based on the control network can be constructed.

In the present invention, such image information is processed as an MPEG-1 / MPEG-2 stream, so the MPEG stream has motion vector information between the times t and t + h for the image shown in Fig. 2 is configured. A portion indicated by a dot in FIG. 2 indicates that a motion vector is a zero vector, and a portion where an arrow appears indicates that there is an image change corresponding to a vector size corresponding to a time t and t + h. So if h is small enough, we can simply extract the object as

, … Equation 1

In equation 1 Denotes a motion vector having information of (x, y) in the (i, j) -th macro block. That is, the upper subscript ij indicates index information of the motion vector, and the lower subscript xy is the content of the motion vector. 3 is an exemplary diagram illustrating an object extracted through an object extraction algorithm.

Ο means an object, which is considered to be a specific set of motion vectors. The threshold value is a value that determines whether the position vectors i and j of the motion vector are included in the object Ο and is set as follows for an image of size n · m.

… Equation 2

In Equation 2, α uses a value smaller than 1 as the proportional constant, and it is empirically estimated by the characteristics of the image. 4 illustrates an example of extracting a disturbed object larger than a threshold hold value.

In the case of object extraction by Equation 1, object extraction may not be correct due to noise or other reasons. In this case, correct object extraction is a filtering problem, but since the object itself is a set of positions of a motion vector, the problem can be solved through simple filtering on the position. If we call the set of motion vectors captured as an object, we obtain the approximate center point by

… Expression 3

I (i, j) ∈Ο is an Index function, and if (i, j) ∈Ο is 1, Is 0. Using m obtained in Equation 3, the variance of the object position is obtained using the following equation.

… Equation 4

σ corresponds to the radius of the circle in FIG. Therefore, the disturbed object can be removed by

, … Equation 5

In Equation 5, β is a positive number β≤1, which is a factor for controlling the size of variance, and is empirically determined by the environment to be monitored.

In the case of more complex images, the objects may not be extracted by the simple object extraction method mentioned above. For example, in the case of an image as shown in FIG. 6, a plurality of motion vectors are mixed to make it difficult to apply a simple object extraction algorithm. In the case of FIG. 6, two kinds of motion vectors appear in the image. In this case, each motion vector must be distinguished by class using a pattern recognition technique. It can be easily implemented as such a pattern recognition tool and extracts complex images by introducing a competitive learning neural network algorithm with excellent pattern classification performance. In this system, the learning equation of competitive learning neural network is as follows.

… Equation 6

In Equation 6, W _r (t) is the r-th weight vector at time t and becomes a representative vector representing a specific set of objects. The training of the competitive learning neural network can generate a weight vector that is sufficiently representative in 10 times of training, and can generate a large number of objects using the weight vector. This is illustrated in FIG. 7. The formula for distinguishing objects using the weight vector of the neural network is as follows. In theory, if there are r weight vectors, r-1 objects can be distinguished.

… Equation 7

In Equation 7, Λ is the macro block's index set, which is in phase with R ² , Ο _r means the r th object, and is equal to the index of the weight vector.

When the Index function is given to the neural network, as shown in FIG. 8, objects can be merged. If there are different motion vectors on the same object, for example, in the human limbs and torso, the movement of the macro block encompassing the limbs is more likely to move than that of the torso. In FIG. 8, it is assumed that all six objects are classified by neural networks, and the merging problem of objects represented by w1, w2, and w3 and objects represented by w4, w5, and w6 among them is the meaning of each object in the monitoring system. Since there is no prior knowledge of, there is no other way to merge objects through similarity of weight vectors. Therefore, the object can be merged by constructing an index function based on the deviation of the similar weight vector using the deviation of the overall weight vector. 9 is an exemplary diagram in which an object of FIG. 8 is located on a screen.

To find the deviation of the weight vector, the pseudo weight vector mean is calculated as follows when the number of weight vectors is r.

… Equation 8

When the average of the pseudo weight vectors is obtained by Equation 8, the deviation of the weight vectors is obtained as follows.

… Equation 9

Put the index function as below,

… Equation 10

In Equation 10, 0 <λ≤1 is a value determined empirically as a parameter for the deviation. FIG. 10 is an exemplary diagram illustrating a position of an average value of each object of FIG. 9. Larger values create fewer objects; smaller values create more objects. If the index function in Eq. 10 is greater than a certain threshold, merge the objects as follows:

… Equation 11

After the object on the video is extracted, camera pan-tilt control for the system is performed using the representative vector representing the object. If the image is simple, a representative vector is generated as shown in FIG. 11 by using average values of motion vectors of the position vectors included in the extracted object.

… Equation 12

When multiple objects appear due to complex images, the representative vectors for each object become the weight vector of the neural network. In the case of the competitive learning neural network, the weight vectors use the property that the weight vectors become an average vector of the receptive field of the weight vector due to the VQ effect.

… Equation 13

When multiple weight vectors represent the same object by merging objects, the arithmetic mean of the weight vectors representing the same object is used as the representative vector of the object.

… Equation 14

When multiple surveillance systems are used for efficient surveillance, certain cameras allow tracking only the object closest to the camera focus position. Therefore, the arithmetic mean of the motion vector position (i, j) of each object is obtained to determine the focus position of a specific camera.

Target focal point of camera = … Equation 15

In Equation 15, C (t) ∈R ² is the current focus position of the camera.

When the object to be tracked is determined through the focal position, the control input is obtained by using the representative vector of the object obtained above. In general, a sampling time of an image is δ _τ and a sampling time of a control input is h. To be down-sampling with respect to the image input. The pan-tilt control of the camera is performed by moving the pan-tilt in the direction of the representative vector of the object by adjusting the focus of the camera to the center point of the object. The Wr _x and Wr _y vectors shown in FIG. 12 become pan and tilt control signals, respectively. Therefore, when the state vector is placed on the screen of the object center point, the first derivative with respect to the time of the state vector becomes the representative vector. Since the output of the control object becomes the object center point, the system can be modeled as follows.

x (t = δτ) = A (t) x (t) + u (t)

y (t) = x (t)

… Equation 16

In equation 16, x represents the center point of the object. Estimation of A (t) is very important in the linear model of Eq. 16, and if δτ is very small, Eq.

When the control input u (t) is obtained using the LQ control method using the estimated model, the object can be tracked by applying a control signal in the opposite direction to the control input to the pan-tilt.

Each camera is controlled to follow the representative vector of the object. On the other hand, if multiple cameras exist in the system, assigning a representative vector to each camera becomes the core of the multiple camera control technology. Assuming that each camera does not share the target image, multiple camera control may follow the representative vector with respect to the image captured by each camera. However, as shown in FIG. 13, when multiple cameras share the same or all of the same image, a representative vector of the image should be appropriately assigned to each camera. The image allocation of the representative vector follows the representative vector at the closest distance through the distance between the starting point of the representative vector and the focal point of the image. This allows each camera to follow an object from another camera even if the object disappears from the image, allowing for efficient control. Since a representative vector exists in the left camera of FIG. 14, a control for following a vector close to the focus is performed. However, when the object leaves the search area, the object appears on the image of another camera at time t + h. At this time, since the representative vector does not exist at time t and the control is stopped, the right vector shows the representative vector at t + h. The left camera stops controlling because the representative vector disappears. 15 illustrates control of the camera according to the movement of the object.

16 is a flowchart illustrating a method of recognizing an image object according to the present invention. Receiving image information acquired by a plurality of cameras forming a network (S1), analyzing a stream header from the received image information (S2), and extracting a motion vector (S2). Competitive learning and neural network learning (S3), calculating the mean and deviation value of the pseudo-weight vector (S4), calculating the index function value (S5), and the threshold of the calculated index value The process of determining whether the excess (S6), the process of creating a separate object (S7-1) or merging into an existing object (S7-2) according to the excess, and the process of calculating the representative vector and image position of the object And a step S10 of calculating a camera target focal position based on the calculated value (S9) and outputting a signal for controlling the attitude of the camera (S10).

As described above, the present invention implements this technical difficulty in a manner completely different from those of developed countries by using newly developed new technologies, thereby securing price competitiveness and securing intellectual property rights to be implemented. Very advantageous. In addition, the present invention implements object extraction and camera posture control by directly using video information without post-processing as a still image, thereby greatly reducing the amount of computation of the processor, thereby making it easier to perform real-time processing, thereby greatly improving the performance of the system. Can be. In addition, using the neural network technique, a representative intelligent information processing method, complex objects can be directly extracted from video information with a few iterations of simple operations, which greatly increases the stability and reliability of the system, and is characterized by high performance. The system is unnecessary and can be implemented as a low-cost commercial system that can be purchased at low cost. Currently, in Korea, as the society develops, there is an urgent need for the automation of security and surveillance systems, but since there are no localized domestic equipment, the import of very high priced intelligent surveillance systems from the United Kingdom, etc., enormous import substitution effect and intelligent It is expected that new markets will be created in industries that require high levels of security, such as buildings and financial institutions.

Claims (1)

Receiving image information acquired by a plurality of cameras in a network; Extracting a motion vector by analyzing a stream header from the received image information; The process of performing competitive learning and neural network learning on motion vectors, Calculating average and deviation values of the pseudo weight vector; Calculating index function values, Determining whether the calculated index value exceeds a threshold; Creating separate objects or merging them into existing objects, Calculating a representative vector and image position of the object; And calculating a camera target focal point position based on the calculated value and outputting a signal for controlling the pose of the camera.