KR20050114817A - Automatic detection method of human facial objects for the digital video surveillance - Google Patents

Abstract

 The present invention relates to a human face object detection system applicable to a digital video security camera for indoor surveillance. The present invention targets a human face and constitutes an interactive vision system capable of automatic extraction and tracking of a human face area. Conventional object tracking systems often use active sensors, in which case the sensor's function is not simple, intelligent and expensive. In order to automatically extract the face region, the present invention is characterized by an object-centered detection method that prepares for deformation of an image according to changes in light and surrounding environment by utilizing information in which various forms such as distance, color, and motion are fused together. . In order to separate the face object from the complex background, it uses displacement map and distance histogram representing distance information and uses matching pixel count (MPC) to improve matching accuracy. The color conversion technique is used to find the face region from the image segmented area by distance information, which is modeled as two factors of light intensity and color of light source and normalized both components. Define a generalized skin color distribution within the feature space. The motion detection technique is defined in the color conversion area where the threshold value is adaptively changed according to the probability of flesh color, and the related operation converts the input color image into the area for expressing the probability of color and motion information. Finally, the component distribution of the detected facial region is defined in the feature space through the convergence of flesh movement information and distance information, and used as model data for tracking. The newly input face image (target) is characterized by performing a mean shift-based tracking through the same process to create a distribution in the feature space and measure similarity with the already set model data.

Description

Automatic detection method of human facial objects for the digital video surveillance}

The present invention relates to a human object tracking system applicable to indoor surveillance cameras or videoconferencing cameras. The present invention makes an intelligent and interactive vision system capable of tracking a human face and automatically extracting and automatically tracking a human face area. Conventional object tracking systems often use active sensors, in which case the sensor's function is not simple, intelligent and expensive.

The present invention has been made in view of the above-described background, and an object of the present invention is to provide a system capable of detecting and tracking a face area of a moving person for constructing a surveillance camera or an image camera.

In order to achieve the above object, the present invention provides an object-background separator for detecting a face object, a face color information detector for detecting a final face area using face color and motion, and a region having a face color area and a weighted motion at the same time. A motion color information extracting unit for extracting an image, a final face object discriminating unit defining an area detected by a plurality of pieces of information as a final face object area, and a large area tracking unit for tracking an object moving the detected data as a target It is composed of parts.

In the present invention, in order to accurately detect a face region in a real environment having various noise elements, a multi-modal method of fusing various types of information such as distance, flesh color, and object motion information is adopted. After the image segmented area and the motion detected area are extracted by distance and face color, the pixel values of the resultant images are combined and converted into a black and white image between 0 and 255. This indicates the probabilities of the face areas, where the areas with high probability values are again grouped and image segmented to define the final face area.

The object-background separator 10 of FIG. 1 is formed by extracting distance information using a stereo image.

Fig. 2 shows in more detail a method of image segmentation based on distance information, which plays an important role in dividing a human object separated from a complex background area. The distance information is processed to obtain a plurality of images from the stereo camera and the edge emphasis process to minimize the noise component and increase the matching probability before obtaining the displacement map (101). In order to calculate the displacement map by stereo matching, MPC (Matching Pixel Count) similarity measurement method was used (102).

3 is a result of obtaining a displacement map of an input image including a plurality of people by using the MPC. The displacement histogram (DH) used here is defined as the number of occurrences according to the distribution of the displacement values (Fig. 4), which is basic information to obtain the distance from the camera and the number of objects. The distance image segmentation process 103 using the DH is specifically performed as follows. First, the DH outline is curved by the average filter. In this case, areas having a frequency of occurrence less than or equal to a predetermined threshold value are considered to have no frequency of occurrence. Successive displacements with a number of occurrences above a certain threshold are recognized as representing one object with the same distance from the camera. Pixels having the same displacement value are extracted from the image, and are given an independent label for each of the areas connected to each other. The area with the smallest displacement is regarded as the background.

The face color information detection unit 20 of FIG. 1 is a technique for predicting an area of a primary human object using color information. Normalized RGB color spaces have been widely used to eliminate the effects of intensity of light sources, but this is limited by assuming that the brightness is fixed and a single color of illumination, a simple background, etc. The present invention proposes a Color Synthetic Normalization (CSN) method which is an improved color normalization method in order to remove the influence of the intensity and color of the light source. When the response of the camera is linear, the image can be expressed by a common factor of S times. Many studies have attempted the RGB normalization process to obtain pure color components using this simple model, which can minimize the effects of changes in light intensity. CSN normalization applies component-specific normalization to remove the dependence of the various colors of the light source in addition to the normalization process for this light intensity. Considering the above two color change factors, the resulting color components can be expressed as follows from the original color components, and can be defined as a general expression model for color components.

Equation 1

The last term on the right side of the above equation is defined as an initial variable due to the camera environment including nonlinear noise and is assumed to be zero in the present invention.

5 illustrates the process of CSN normalization proposed in the present invention and follows the following simple method. That is, first, the input image is normalized 201 with respect to the intensity of the light source, and then normalized by the color component 202. These two consecutive operations are repeated until the normalized color components converge into small regions. When pixel values in a normalized color space are viewed on a color histogram, pixel values having similar colors are concentrated in a small area, and the distribution is similar to that of a Gaussian distribution (203). Therefore, the present invention defines a general skin color distribution by applying a two-dimensional Gaussian distribution, and converts the input image into a region of the already defined face color component distribution using this definition as a parameter.

FIG. 6 is a step for increasing the reliability of the final face object extraction by defining the distribution probability of the face object component as a highly reliable parameter by reflecting the motion component in the image as a weight in the conversion process 301 of FIG. One of the important information in the image data for interactive communication is the movement information of the face because most of the region of interest has movement. An unmatched pixel count (UPC) motion detection measure was used to find motion information of an object (302). UPC has a simple operation in block units. AWUPC (Adaptive Weighted Unmatched Pixel Count) operation proposed in the present invention is defined as Equation 2, Equation 3, Equation 4, and Z (x, y, t) is a result image converted to a general face color distribution and U (i, j, t) is the UPC motion detection result. The AWUPC operation shows the result of emphasizing the moving component in the chrominized region.

Equation 2

Expression 3

Equation 4

On the other hand, the threshold value in Equation 4 is used sigmoid function as shown in Figure 7 so that it can be adaptively determined according to the skin color similarity of the input color image, the detailed calculation method is shown in Equation 4. Where Z (x, y, t) is the input pixel value at time t and Q is the coefficient that determines the slope of the curve of the sigmoid function. The reasons for using adaptive thresholds are as follows. The pixel value of the input image represents the probability of face color. Therefore, a pixel that already has a high probability of face area needs to have a low threshold in order to be detected as a face object even with small movements. On the contrary, an area that has a low probability of being a face through color conversion is a region of interest such as a face. Since this is not the case, use a high threshold so that it can be detected only when there is a large movement.

8 is a diagram illustrating a method for defining a final face object component using a plurality of information. In the first step, the background area containing the most noise and the target area of interest are separated from each other by the image segmentation by the displacement map, and the area where the face color is emphasized and the area where the motion color information is reflected in the second step. To extract. In the third step, we construct the probability distribution in the space including all the above components by converting the pixel unit, and configure the feature space so that the component in the most probable region is in the center of the Gaussian distribution, Define (401).

The area tracer of FIG. 1 defines the pixel unit component distribution in the final face object component defining step 401 as a probability density function of a model image, forms a feature space for a newly input image, and defines a probability density function. The ROI is tracked in the newly input image by measuring the similarity between the distribution of the image and the target image. The component distribution of the model projection improves the center point for tracking the ROI by means of weight shifting based on mean shift.

The present invention proposes an object-centered face region detection technique using information in which various forms such as distance, color, and motion are fused together. In order to separate the face object from the complex background, a stereo displacement histogram representing distance information is used, and the displacement measurement method using MPC improves the accuracy of matching and finds the face area from the image segmented area by distance information. Color conversion technology was used. The normalized distribution of eigencolors is stabilized by the CSN normalization method that removes the color factor of light, and the generalized face object region distribution is defined using the 2D Gaussian function in the color space. The AWUPC motion detection technique defines a color conversion region in which the threshold is adaptively changed according to the probability of flesh color, and the AWUPC operation reconverts the input color image into a probability value having both color and motion information.

As described above, the method for automatically detecting a human face object according to the present invention is a technology for effectively separating a face object from a complicated background by fusing a plurality of pieces of information with each other, and increasing the probability of working as a face region. -It can be applied in technical fields such as digital video surveillance device and video communication that require computer-to-computer interaction.

1 is an overall flow diagram depicting the overall process of the present invention.

2 is a detailed flowchart for the object-background separation unit during the entire process of the present invention.

3 is an example of generating a displacement map in step 102 of the object-background separator of FIG. 2.

FIG. 4 illustrates an example of generating a disparity histogram for explaining distance information image segmentation in step 103 of the object-background separator of FIG. 2.

5 is a detailed flowchart of the face color information detection unit of FIG. 1.

6 is a detailed flowchart of a motion color information detection unit of FIG. 1.

FIG. 7 illustrates a function 303 for applying an adaptive threshold value for detecting inter-frame motion among the motion color information detection units of FIG. 6.

8 is a flowchart illustrating a detailed process for determining a final face object region.

Claims (7)

In the method of detecting a region of interest of an input image for digital image security, an object-background separator 10 for separating an object-background by extracting distance information by a stereo camera and a face through color normalization of the input object A motion color information extractor 30 and a final face object discriminator 40 which focus on the same input image as the face color information detector 20 that defines and extracts the color gamut by weights of the motion information and the face color gamut Image processing / monitoring system comprising a region tracking unit (50). The image of claim 1, wherein the face color information detection unit is defined in a feature space in which the intensity normalization unit 201 of the light source and the color normalization unit 202 of the light source are simultaneously taken in the normalization process and converts the ROI by Gaussian distribution. Processing / Surveillance System. The method of claim 1, wherein the motion color information extracting unit reflects the distribution of the face color component of the ROI and the motion value between the image frames at the same time by performing AWUPC (Aaptive Weighted Unmatched Pixel Count) calculation in the process of estimating the motion of the pixel unit. Image processing / monitoring system using feature space. The image of the method of claim 3, wherein, in a threshold value used for calculating a difference in motion between image frames, a separate function is applied such that the determination of the threshold value is variably changed according to pixel values of the input image to be compared. Processing / Surveillance System. 4. The image processing / monitoring system according to claim 3, wherein the sigmoid function as shown in Fig. 7 is used as a transform function to determine a threshold value used for calculating a difference between motions between image frames. The method of claim 1, wherein the face color information detecting unit 20 and the motion color information extracting unit 30 are processed to the region of interest including only the short-range objects already separated by the object-background separating unit 10. Image processing / monitoring system. The method of claim 1, wherein the processes 10, 20, 30, and 40 for detecting a human face object are performed only on the initial input image, and the ROI is tracked through the region tracker 50 based on the mean shift algorithm. The image processing / monitoring system does not repeat the object detection process (10, 20, 30, 40) of interest in the process.