KR100865531B1 - Method for separating individual pedestrians by clustering foreground pixels - Google Patents

Abstract

A method for separating individual pedestrians by clustering foreground pixels is provided to separate the discrete pedestrian accurately even though pedestrians can be overlapped in the foreground image by grouping each pixel about the foreground image to a plurality of groups and effectively merging and removing groups. A method for separating individual pedestrians by clustering foreground pixels relates to the method for separating the discrete pedestrian from the source image. More particularly, it is about the discrete pedestrian separation method by the grouping of the foreground pixel separating discrete pedestrians from foreground pixels separated from the source image for the behavioral analysis of pedestrian. Each pixel of the source image is classified into the background pixel and foreground pixel. And the foreground image comprising the silhouette image of the pedestrians with a plurality of foreground pixels is produced(S101). Each pixel comprising the foreground image is grouped to one or more groups of each pedestrian unit. A plurality of groups overlappingly mapped on one discrete pedestrian is merged. And after error groups which are not included in the discrete pedestrian are removed, the discrete pedestrian is distinguished(S103).

Description

Method for separating individual pedestrians by clustering foreground pixels}

1 is a flowchart illustrating a separation procedure, which is individual walking by clustering foreground pixels according to the present invention.

2 is a flowchart illustrating a detailed procedure of separation of individual walkers by clustering foreground pixels according to an exemplary embodiment of the present invention.

3 is a view showing an original image according to an embodiment of the present invention.

4 is a view showing a foreground image according to an embodiment of the present invention.

5 is a diagram showing a pedestrian silhouette model according to an embodiment of the present invention.

6 is a diagram illustrating a result of overgrouping a foreground image according to an exemplary embodiment of the present invention.

7 is a view showing a result of preprocessing an over-grouped image according to an embodiment of the present invention.

8 is a graph showing the score and the coupling force of each cluster according to an embodiment of the present invention.

9 is a view showing the final result of separating individual pedestrians by merging and removing respective clusters according to an embodiment of the present invention.

10 to 14 illustrate clustering results and final results for each original image according to an embodiment of the present invention.

The present invention relates to a method for separating individual pedestrians from an original image, and more particularly, to a method for separating individual pedestrians by clustering foreground pixels that separate individual pedestrians from foreground pixels separated from the original image for pedestrian behavior analysis. It is about.

As terrorism soared recently, countries around the world are realizing the need for safety and increasing security in the public sector. In this process, the importance of video security technology is increasing day by day. Video security systems built since the 1980s are evolving into today's digital video surveillance systems, which have been reduced to the cutting edge of technology due to improvements in recognition, cost and performance.

The conventional video security system has advantages in terms of ease of use and low price. However, there is still some inconvenience that the camera takes after reviewing the video captured by the video cassette recorder (VCR) or digital video recorder (DVR), and the administrator must watch directly on the monitor. Recently, unmanned security surveillance has also been enabled by DVRs and IP (Internet Protocol) cameras. However, even in the case of motion detection, the performance of the complete unmanned automation system does not meet expectations.

As a device that can reverse the evaluation, intelligent video security system is emerging as a new topic. Intelligent video security systems are actively being developed and released mainly in developed countries, and are expanding their scope based on high response in the market. In particular, the Motion Detection function, which detects and captures moving objects by itself, or the Video Content Analysis function, which automatically analyzes scenes and extracts information without human surveillance or supervision, It is well received.

Intelligent image detection consists of three phases: video analysis, content analysis, and appropriate response to content analysis. For this purpose, the technology of extracting a person using a difference image between frames and a difference image with a background from an input image input in real time, an object recognition technique distinguishing between a detected person, a car, an animal, etc., and a trace of a detected object Elemental skills, such as techniques for analyzing information and statistical information to detect abnormal behavior, are needed.

As such, most intelligent video surveillance systems require pedestrian behavior analysis. In order to interpret pedestrian behavior, it is important to track their movements. Since this tracking estimates the current position based on the previous position of the object, tracking the object requires knowing the initial position of the object. A simple way to know the initial position of an object is to combine background subtraction with connected component labeling.

That is, first, the foreground pixels constituting the pedestrian are divided using the background subtraction method, and the divided foreground pixels are grouped by area using a connected component classification method. However, when pedestrians overlap in the foreground image, this method cannot identify individual pedestrians. Therefore, there is a problem in that it is not possible to track pedestrians individually and also analyze the behavior of individual pedestrians.

Meanwhile, various approaches for detecting individual pedestrians have recently been proposed. For example, a gait detection method using a horizontal projection histogram of a foreground image has been proposed. In this method, since the head of the pedestrian appears in the form of a peak in the horizontal projection histogram, the position of each pedestrian is detected based on the head positions of the pedestrians found in this way. However, since the method assumes that several people are horizontally distributed in the image, there is a problem that it does not operate when such an assumption is not satisfied.

Therefore, there is a need for a method for effectively and accurately separating individual pedestrians from the foreground image.

SUMMARY OF THE INVENTION An object of the present invention is to provide a method for separating individual walks by clustering foreground pixels by separating foreground pixels from an original image and grouping the separated foreground pixels.

It is also an object of the present invention to provide a method for separating individual walks by clustering foreground pixels by overgrouping the separated foreground pixels and separating and separating individual pedestrians by merging and removing respective overgrouped clusters.

It is also an object of the present invention to over-group the separated foreground pixels and to separate individual pedestrians by merging and removing the individual pedestrians by merging and removing each of the over-populated clusters by the cohesion between the clusters and the normalized post score. To provide a separation method.

The method of the present invention for achieving the above object, in the method for separating one or more individual pedestrians from the original image consisting of a plurality of pixels, each of the pixels of the original image is classified into a background pixel and a foreground pixel Generating a foreground image constituting a silhouette image of pedestrians using the plurality of foreground pixels; Clustering the pixels constituting the foreground image into clusters of at least one pedestrian unit; And merging a plurality of clusters mapped to the individual pedestrians of one of the clusters and removing error clusters not included in the individual pedestrians to identify individual pedestrians.

At this time, the clustering is characterized in that over- clustering into a larger number of clusters than the actual number.

The method of classifying each pixel of the original image into a background pixel and a foreground pixel may include classifying a color distribution of each pixel as a mixed Gaussian model and classifying the background pixel and the foreground pixel based on the mixed Gaussian model. do.

The color values of the classified pixels may be reused to update the mixed Gaussian model, and the pixel classification and model update may be repeated every frame of the original image.

On the other hand, the clustering step, generating a silhouette model that is walking for each cluster; Initializing the number of clusters and parameters of the model for each cluster; And estimating an average vector representing the center position of the pedestrian in each model from the number of initialized clusters and model parameters of each cluster.

The pedestrian silhouette model is characterized in that the covariance matrix is defined as a two-dimensional Gaussian distribution geometrically fixed at each position, and the parameters of the model are composed of an average vector, a covariance matrix, and a prior probability. .

On the other hand, initializing the number of clusters and the parameters of the model for each cluster may include: dividing the plane into a grid separated by 1/2 times the average elongation from the flat floor on which the object is moving; Hypothesizing a central location of a pedestrian for each grid point of the grid; And verifying a center position of the pedestrian set as the hypothesis.

In this case, the verification method is characterized in that the normalized post-score for the cluster model at each grid point is initialized for a grid point larger than the first predetermined boundary value.

The estimation of the average vector is characterized by estimating a value converged by an expected value-maximization algorithm.

In addition, after the clustering, the method may further include merging clusters in which the center position of each cluster is closer than one pixel unit, calculating a normalized post score for each of the clusters, and And removing clusters whose calculated normalized post score is less than a second predetermined threshold value.

Meanwhile, the method of merging a plurality of clusters may include merging the two clusters when a bonding force of two selected clusters from the plurality of clusters is greater than a third boundary value. It is characterized by calculating and measuring the correlation coefficient between the clusters.

The step of identifying the individual pedestrian, characterized in that performing the merging process for the two clusters before the removal of the cluster, if the normalized post score of the cluster is greater than the fourth threshold value, the corresponding Characterize the cluster as one individual pedestrian.

In addition, the step of identifying the individual pedestrian, when the normalized post-score for each of the two selected from the clusters is less than the fourth threshold value, the correlation coefficient between the two clusters is greater than the third threshold value, It is characterized by merging the two clusters.

In this case, the merging between the two clusters is characterized by determining a cluster group to be merged using a depth-first search method.

On the other hand, the step of identifying the individual pedestrian, if the normalized post-score of the cluster is less than a fourth threshold value, the correlation coefficient with all other clusters of the cluster is smaller than the third threshold value, Characterized by removing the cluster.

At this time, the removal of the cluster, characterized in that the normalized post-score of each cluster is performed in ascending order from the smallest cluster.

On the other hand, the information on the separation method, which is an individual walk by clustering the foreground pixels, may be stored in a recording medium readable by a server computer. Such recording media includes all kinds of recording media on which programs and data are stored so that they can be read by a computer system. Examples include Read Only Memory (ROM), Random Access Memory (RAM), Compact Disk (CD), Digital Video Disk (DVD) -ROM, Magnetic Tape, Floppy Disk, Optical Data Storage, etc. It also includes implementations in the form of (eg, transmission over the Internet). In addition, these recording media can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Object detection is an essential element for tracking objects in an intelligent visual surveillance system. Accordingly, the present invention proposes a method of separating individual pedestrians by separating background pixels and foreground pixels from an original image and clustering the separated foreground pixels. do.

That is, in the present invention, when a silhouette image of various pedestrians is given, the locations of individual pedestrians are found through clustering. In this case, it is assumed that a plurality of walking models generate pixels constituting a silhouette, and the position of each pixel is used as a sample for clustering. In addition, the parameters of the model including the center coordinates are estimated using the clustering method.

1 is a flowchart illustrating a separation procedure, which is individual walking by clustering foreground pixels according to the present invention. Referring to FIG. 1, a foreground image is first generated by separating foreground pixels from an original image (S101), and the walking samples are clustered by over-clustering the foreground pixels of the generated foreground image (S102). Done. Then, the individual pedestrians are finally identified by merging and removing the clustered clusters to separate individual pedestrians (S103).

Meanwhile, in the present invention, an expectation-maximization algorithm (hereinafter, referred to as an 'EM algorithm') is used to cluster the foreground pixels. In addition, it is assumed that background subtraction may be used to separate foreground pixels from the original image, and the foreground pixels calculated by the background subtraction are generated by a plurality of walking models.

At this time, since the exact number of pedestrians present in the image is not known, the foreground pixels are clustered into a larger number of clusters than the actual number. In other words, this is called over-clustering. Then, the clusters are merged or removed according to a normalized posterior score and a coherence measure. Experimental results using real images show the justification of the method according to the invention.

DETAILED DESCRIPTION Hereinafter, a detailed description of a preferred embodiment of the present invention will be described with reference to the accompanying drawings. In the following description, detailed descriptions of well-known functions or configurations will be omitted when it is determined that the detailed descriptions of the known functions or configurations may unnecessarily obscure the subject matter of the present invention.

2 is a flowchart illustrating a detailed procedure of separating individual walkers by clustering foreground pixels according to an exemplary embodiment of the present invention.

Referring to FIG. 2, first, a silhouette of an object is divided by a background subtraction method in order to separate the foreground image from the background image (S201). At this time, the pedestrian silhouette model that will form the respective pedestrian clusters is generated (S202).

Then, the number of walking persons c and the model parameter μ _j are initialized (S203) with the boundary value θ _i with respect to the normalized posterior score S _j . The average vector μ _j of each cluster ω _j is calculated (S204) by the EM algorithm using the initialized number of pedestrians and model parameters. When the mean vector μ _j of each cluster converges to a specific value by the iterative operation of the EM algorithm, more initial clusters are obtained than actual numbers. That is, the entire foreground image is overgrouped into a plurality of clusters.

In this case, the preceding process may be selectively performed (S205) so that the merging and removing of the steps S206 and S207 can be more effectively performed. In other words, clusters closer to the center of two clusters than 1 pixel unit are merged, and a preprocessing process for removing clusters whose normalized post-score S _j is smaller than the boundary value θ _p can be selectively performed. have.

Then, when the coupling force ρ _jk of the two clusters ω _j and ω _k is greater than the boundary value θ _m , the two clusters are merged (S206). The determination of the group of clusters to be merged may use a depth-first search method.

Finally, to examine the post-score (S _j) normalizing for each cluster a cluster having a small post-normalized score (S _j) than a predetermined threshold value (θ _r) is removed (S207).

Hereinafter, each step described above with reference to FIG. 2 will be described in more detail with reference to FIGS. 3 to 14.

In the present invention, as described above, we propose a separation method for individual walk using the clustering method. In this case, the walking model may be represented by a two-dimensional Gaussian distribution. In this case, in order to effectively apply the pedestrian model, the camera is calibrated, and it is assumed that the pedestrians have a similar physique and move on a flat floor. Such geometric information is inherent in the walking model. After clustering, the clusters to merge and the clusters to remove are determined based on the analysis between each cluster.

First, a method of separating individual pedestrians by performing foreground division from an original image and then filtering and clustering each foreground pixel will be described.

If the camera is fixed, the silhouette of the object may be divided by using a background subtraction method. For example, the RGB distribution of each pixel may be modeled as a plurality of mixed Gaussians (hereinafter, referred to as 'MoG'). The MoG models are initialized through learning during the initial time. Each pixel is classified into a background or a foreground based on the MoG model. The RGB values of the classified pixels are then used to update the model. In this case, pixel classification and model updating are repeated every frame in the continuous image. By doing so, the foreground image of FIG. 4 can be obtained from the original image of FIG. 3.

If it is assumed that no object other than the pedestrian exists in the foreground image as shown in FIG. 4, all foreground pixels may be collected to form a silhouette image of the pedestrian. At this time, the pixels constituting the silhouette are grouped in units of gait.

On the other hand, in order to use the clustering method, a generation model for a pedestrian silhouette is required. In this case, it is assumed that all walkers have the same height h and the same width w, and move on a flat floor. If you know the camera projection matrix and the plane equation at the bottom, you can determine the orientation and size of the pedestrian at each location in the image. Thus, the walking silhouette model may be defined as a two-dimensional Gaussian distribution in which a covariance matrix is geometrically fixed at each position. In this case, the likelihood probability p (x | ω _j ) may be defined as a two-dimensional Gaussian probability distribution function as shown in Equation 1 below.

In Equation 1, x represents each pixel, and ω _j represents each cluster. In addition, μ represents a mean vector of the cluster, and Σ represents a covariance matrix of the cluster.

5 is a diagram illustrating a silhouette model of the walking person. In FIG. 5, the standard deviation of the long axis is 1/3 times the height h measured in the image, and the standard deviation of the short axis is 1/3 times the width w measured in the image. Thus, the total area of the walking silhouette has about 87% confidence interval.

On the other hand, since a two-dimensional Gaussian model is used for clustering, the model parameter is composed of an average vector μ _j , a covariance matrix Σ _j , and a prior probability P (ω _j ). In this case, it is assumed that the covariance matrix can be geometrically calculated at each position of the image, and the prior probability is the same regardless of the cluster.

Therefore, only the mean vector representing the center position of the pedestrian in each model is estimated, and the expectation-maximization algorithm (EM-MoG) of the mixed Gaussian may be applied to the learning method of the mean vector. The average vector μ _j of each cluster is repeatedly calculated in the same manner as in <Equation 2>.

In this case, the posterior probability P (ω _j | x _i ) in Equation 2 may be calculated as Equation 3 below.

On the other hand, in order to apply the above-described EM algorithm, it is necessary to know the exact number of clusters (c) and be able to initialize the parameters of each cluster (ie, the average vector μ _j ). In this case, although the number of clusters may be used and the initial value of the parameter may be arbitrarily determined, the initial value may be more effectively determined by using a pedestrian filtering method.

In the present invention, the number of pedestrians and model parameters are initialized by the following method. First, the plane divided by 1/2 times the average elongation h from the flat bottom on which the object moves is hypothesized, and the pedestrian's center position is hypothesized for each grid point. Then, each gait model hypothesis is verified by applying a boundary value θ _i to a normalized posterior score S _{j as} shown in Equation 4 below.

는 군집 ω_j에 속하는 전경 화소 표본의 개수를 의미한다. 그러나 카메라로부터 멀리 떨어진 화소일수록 실제 나타내는 면적은 커지므로, 상기 전경 화소 표본의 개수를 면적 A_j로 나누어 정규화(normalize)한다. 따라서, 상기 정규화된 사후 점수 S_j에 대해 경계값을 적용하여, 군집의 개수 c와 각 군집의 매개 변수 μ_j의 초기값을 결정하게 된다. 즉, 각 군집 ω_j에 대해 상기 정규화된 사후 점수 S_j가 기설정된 경계값(θ_i)보다 큰 군집만을 EM 알고리즘의 초기값으로 사용하게 된다.In Equation 4, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the cluster ω _j . Sum of posterior probabilities constituting the molecule of Equation 4

Denotes the number of foreground pixel samples belonging to the cluster ω _j . However, since the farther the pixel is from the camera, the area actually displayed becomes larger, the normalization is performed by dividing the number of the foreground pixel samples by the area A _j . Therefore, by applying a boundary value to the normalized post score S _j , the number of clusters c and the initial value of the parameter μ _j of each cluster are determined. That is, only the cluster whose normalized post score S _j is larger than the predetermined boundary value θ _i for each cluster ω _j is used as the initial value of the EM algorithm.

If the EM algorithm is repeated from the number c of clusters determined according to the above conditions and the initial value of the parameter μ _j of each cluster, a plurality of clusters are determined by gradually converging the average vector of each cluster to a specific value.

At this time, since the lattice points that are hypothetically selected at the time of pedestrian filtering are densely selected, more initial clusters are obtained than actual numbers. This clustering of samples into more clusters than the actual number is called 'over-clustering'.

As described above, when the clustering method is applied, each cluster is fitted to the silhouette image. At this time, since more clusters than actual numbers are applied, some clusters are clustered together, and some are fitted to outliers that are incorrectly obtained in the foreground image.

FIG. 6 illustrates a result of intermediate clustering of the foreground image of FIG. 4, which includes duplicate clusters and incorrect clusters. Therefore, there is a need for a method of merging duplicate clusters and removing erroneous clusters. That is, after the over clustering, clusters very close to each other should be merged, and clusters having a small number of foreground pixel samples should be removed.

On the other hand, before performing the actual exact merging and elimination process, it is possible to reduce the operation of the actual merging and elimination process by performing a rough merging and elimination process by performing the optional preprocessing.

For example, clusters having a center position closer than one pixel unit may be merged as the preprocessing process, and clusters having a normalized post score S _j smaller than a specific boundary value θ _p may be removed.

FIG. 7 illustrates a result of preprocessing the overgrouped foreground image of FIG. 6. In FIG. 7, the numbers indicate the numbers of each cluster, and each cluster is represented by a model of rectangular walking. At this time, the cross mark inside each rectangle means a center value of each cluster, that is, an average vector.

Although this pretreatment method can be used to significantly reduce the number of clusters, unnecessary clusters remain. Thus, final results are obtained by merging or removing clusters by the method described below.

Before describing a method of merging or removing clusters, a method of measuring coherence between two clusters required for merging clusters is described.

Hereinafter, a statistical method for measuring the binding force between two clusters is proposed. If the two clusters ω _j and ω _k are highly related, the posterior probabilities P (ω _j | x _i ) and P (ω _k | x _i ) of the samples have the same characteristics. For example, if P (ω _j | x _i ) increases, P (ω _k | x _i ) also increases, and conversely, if P (ω _j | x _i ) decreases, P (ω _k | x _i ) decreases. Thus, the use of posterior probability of the sample for each cluster in the feature vector (Feature vector) H _i, H _i the characteristic vector can be expressed as <Equation 5>.

At this time, in order to measure the coupling force between the two groups ω _j and ω _k , a correlation coefficient ρ _jk is calculated as shown in Equation 6 below.

Since the value of ρ _jk calculated in Equation 6 is a correlation coefficient, it has a distribution between [-1 and 1]. In this case, X is a random variable representing a coordinate value of a foreground pixel, which is a corresponding probability distribution for calculating an expectation value in order to obtain covariance and variance. On the other hand, the actual calculation of the correlation coefficient is performed by the following equation (7). In this case, n represents the total number of foreground pixel samples.

The parameters μ _j and μ _k of each cluster in Equation 7 are calculated as in Equations 8 and 9, respectively.

Looking at the characteristics of the correlation coefficient proposed in the present invention, because of constraints such as Σ _j P (ω _j | x) = 1, the correlation coefficient ρ _jk becomes negative if two clusters are not related. This means that if P (ω _j | x _i ) increases, P (ω _k | x _i ) decreases and conversely, if P (ω _j | x _i ) decreases, P (ω _k | x _i ) increases. do. As such, the correlation coefficient between the posterior probabilities used in the coupling force measurement has a property of evaluating the large and small overlapping areas between two clusters based on the distribution of the foreground pixel samples. Therefore, it is possible to determine whether two clusters should be combined by applying a boundary value to the correlation coefficient.

Hereinafter, a method of finally separating individual pedestrians by merging and removing clustered populations according to the present invention will be described.

That is, by combining the normalized post score S _j and the correlation coefficient ρ _jk , which is a measure of the binding force, the clusters can be divided into three cases according to the two values.

If the result of cluster ω _j is correct, then the normalized post score S _j has a large value. On the other hand, if two clusters ω _j and ω _k are to be merged, the two normalized post scores S _j and S _k have a small value, but the correlation coefficient ρ _jk between the two has a large value. On the other hand, if the cluster ω _j is to be removed, the normalized post score S _j of the cluster has a small value and the correlation coefficient ρ _jk with all other clusters also has a small value.

Table 1 below shows a method of performing merging and elimination based on the two values described above.

Occation Post Score S _j Correlation coefficient ρ _jk Perform Oh yeah ↑ - - lie ↓ ↑ absorption lie ↓ ↓ remove

FIG. 8 is a graph illustrating post-scoring scores S _j and values of the correlation coefficient ρ _jk for the clusters of FIG. 7.

Referring to FIG. 8, each index of the horizontal and vertical axes on the graph corresponds to the number of clusters shown in FIG. 7. In this graph, elements on the diagonal line (ie, a column having the same horizontal value and vertical value) represent a post score S _j for the corresponding value (ie, the corresponding cluster), and the remaining elements (squares) correspond to the horizontal axis values. Correlation coefficient ρ _jk between a cluster and a cluster corresponding to a vertical axis value is represented. According to the graph, clusters resulting from the same pedestrian are highly related and should be merged.

For example, cluster 1 and cluster 2 each have a post score of about 0.4, and the correlation coefficient is close to 1, so it is preferable to merge. It is preferable that the 13th cluster and the 14th cluster are similarly merged, and the 15th cluster and the 16th cluster are also preferably merged. However, clusters 17 and 18 have high post scores close to 1, respectively, so they can be judged as independent individual pedestrians. The remaining clusters can also be merged or removed by the same method.

On the other hand, the order is important because different results can be obtained according to the order of merging and removing. Thus, the merging and elimination algorithm consists of two stages, merging and elimination, in which only merging is performed. Since the merging must be performed for all clusters by selecting two clusters and analyzing the post scores and correlations, the cluster group to be merged is determined by using a depth-first search in order to effectively perform the merging. Can be.

In this case, the connectivity between two clusters ω _j and ω _k is determined by applying a boundary value θ _m to the correlation coefficient ρ _jk , which is a coupling force measurement element. That is, when the correlation coefficient p _jk is greater than a predetermined boundary value θ _{m while the} post score is small for each of the two clusters, the two clusters are merged.

The merging process is repeated until there are no more cluster groups to merge, and it is preferable to merge one cluster only once for the precision of the final result.

The second step is the removal step, which removes one cluster at a time with a post-score S _j that is less than a particular boundary value θ _r .

At this time, if one cluster is removed, the post score of the remaining clusters is increased by the constraint Σ _j P (ω _j | x) = 1. Therefore, cluster removal is preferably performed in ascending order with respect to the post score S _j .

9 shows the final result of merging and elimination. It can be seen that the location of all pedestrians was calculated correctly.

<Experiment Result>

10 to 14 show the results of experiments on various arrangements of pedestrians of the algorithm implemented according to the present invention as described above.

FIG. 10 is a result of selecting a 300th image from a continuous image, and FIG. 11 is a result of selecting a 500th image from a continuous image, and FIG. 12 is a result of selecting a 700th image from a continuous image. 13 is a result of selecting an 800th image from a continuous image, and FIG. 14 is a result of selecting a 900th image from a continuous image.

In each of the images of FIGS. 10 to 14, the image located on the left represents original images, and the image located in the middle represents a result of performing intermediate clustering. Also, the image located on the right side shows the final result of merging and removing.

Referring to FIG. 10, it can be seen that the final number of pedestrians was correctly calculated. In other words, since a pedestrian is out of the image in the left part of the image, the post score of the cluster is small and removed. Also, two pedestrians overlap in the upper left part of the image. At this time, the area of the two leg portions appeared larger than the area of the upper body portion in the result of the foreground division, and the center position of each pedestrian was estimated toward the leg.

In FIG. 11, the result was correctly calculated despite the overlap between the shadow and the pedestrians. The walking arrangement in FIG. 12 is easy to separate and has been correctly separated.

13 and 14, two pedestrians vertically overlap. In this situation, the silhouette image is too ambiguous to estimate the correct number of pedestrians. At this time, three pedestrians stacked vertically can also produce the same silhouette image. Therefore, the experimental result can be calculated differently according to the post score and the boundary value for the binding force.

The pedestrian separation method based on clustering according to the present invention was introduced. Geometric constraints are inherent in fixing the covariance matrix of the walking model. As described above, after the overcrowding, the pedestrians can be effectively and accurately separated by merging the duplicated clusters according to the post score and the cohesion measurement and removing the wrong clusters.

On the other hand, in the embodiment of the present invention has been described with respect to specific embodiments, various modifications are possible without departing from the scope of the invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the appended claims, but also by the equivalents of the claims.

According to the present invention, after each pixel for the foreground image is clustered into a plurality of clusters, the clusters can be effectively merged and removed, so that even if pedestrians overlap in the foreground image, individual pedestrians can be accurately separated. There is this.

In addition, when clustering, there is an advantage in that individual pedestrians can be accurately separated by over-aggregating samples into a larger number of clusters than actual clusters and merging and removing unnecessary clusters.

In addition, it is possible to effectively separate individual pedestrians by performing the merging and elimination by correlating each pixel from the foreground image with a correlation coefficient measuring a normalized post score and coupling force.

Finally, there is an advantage that more efficient merging and elimination can be performed by performing coarse merging and elimination by prior processing before performing substantial merging and elimination after the over-grouping.

Claims (37)

In the method for separating one or more individual walkers from the original image consisting of a plurality of pixels, Classifying the pixels of the original image into a background pixel and a foreground pixel to generate a foreground image constituting a silhouette image of pedestrians using the plurality of foreground pixels; Clustering the pixels constituting the foreground image into clusters of at least one pedestrian unit; And Clustering a plurality of clusters mapped to the individual pedestrians of one of the clusters, removing error clusters not included in the individual pedestrians, and identifying the individual pedestrians. Separation method of individual walkers. The method of claim 1, wherein the clustering, Separation method that is an individual walk by clustering the foreground pixels, characterized in that the clustered over a larger number than the actual number. The method of claim 1, wherein each pixel of the original image is classified into a background pixel and a foreground pixel. The gait separation method according to the clustering of the foreground pixels, characterized in that the color distribution of each pixel is modeled by a mixed Gaussian model, and the background and foreground pixels are classified based on the mixed Gaussian model. The color value of the background pixel and the foreground pixel classified based on the mixed Gaussian model, And separate gait by clustering foreground pixels, wherein the mixed Gaussian model is reused to update the mixed Gaussian model. The method of claim 4, wherein the pixel classification and model update, The separation method of the individual walk by clustering the foreground pixels, characterized in that repeated for every frame of the original image. The method of claim 1, wherein the clustering step, Generating a silhouette model that is walking for each cluster; Initializing the number of clusters and parameters of the model for each cluster; And And estimating an average vector representing the center position of the pedestrian in each model from the number of initialized clusters and model parameters of each cluster. According to claim 6, The walking silhouette model, A method of separation that is individual walk by clustering foreground pixels, wherein the covariance matrix is defined as a two-dimensional Gaussian distribution that is geometrically fixed at each location. The method of claim 7, wherein the two-dimensional Gaussian distribution, Separation method which is an individual gait by clustering foreground pixels, characterized by the following equation.

Where x is each pixel, ω _j is each cluster, μ is the mean vector of the cluster, and Σ is the covariance matrix of the cluster. The method of claim 6, wherein the parameter of the model, A separation method, which is an individual gait by clustering foreground pixels, comprising a mean vector, a covariance matrix, and a prior probability. The method of claim 6, wherein initializing the number of clusters and the parameters of the model for each cluster comprises: Dividing the plane into a grid separated by 1/2 times the average elongation from the flat bottom at which the object moves; Hypothesizing a central location of a pedestrian for each grid point of the grid; And And a step of verifying a center position of the pedestrian set as the hypothesis. The method of claim 10, wherein the verifying the central position of the pedestrian set as the hypothesis comprises: A method for separating individual walks by clustering foreground pixels, wherein the normalized post-score for the cluster model at each grid point is initialized for a grid point larger than a first predetermined boundary value. The method of claim 11, wherein the normalized post-mortem score, Separation method of the individual walk by clustering the foreground pixels, characterized in that calculated as shown in the following equation.

In this case, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the group ω _j . Sum of posterior probabilities constituting the molecule of the equation

Denotes the number of foreground pixel samples belonging to the cluster ω _j . The method of claim 6, wherein the estimation of the average vector, A method for separating individual walks by clustering foreground pixels, characterized by estimating a value converged by an expected value-maximization algorithm. The method of claim 13, wherein the estimation of the average vector, Separation method which is an individual gait by clustering foreground pixels, characterized by iteratively calculating by repeatedly performing the following equation.

Where μ _j is the mean vector of each cluster, x _i is each pixel, and P (ω _j | x _i ) represents the posterior probability. 15. The method of claim 14, wherein the posterior probability P (ω _j | x _i ) is calculated by the following equation.

Where c denotes the number of initialized clusters and p (x _i | ω _j ) denotes a two-dimensional Gaussian probability distribution of the cluster ω _j . The method of claim 1, wherein after the clustering step, The method of claim 1, further comprising merging clusters of which the center position of each cluster is closer than one pixel unit. The method of claim 1, wherein after the clustering step, Calculating a normalized post score for each of the clusters, and removing clusters of which the calculated normalized post score is smaller than a second predetermined threshold value. How to separate pedestrians. The method of claim 17, wherein the normalized post score, Separation method of the individual walk by clustering the foreground pixels, characterized in that calculated as shown in the following equation.

In this case, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the group ω _j . Sum of posterior probabilities constituting the molecule of the equation

Denotes the number of foreground pixel samples belonging to the cluster ω _j . The method of claim 1, wherein the method of merging a plurality of clusters comprises: Separation method according to the clustering of the foreground pixels, characterized in that for merging the two clusters when the coupling force of the two selected from the plurality of clusters is greater than the third boundary value. The method of claim 19, wherein the binding force of the two clusters, And separating and measuring a correlation coefficient between the two clusters. The method of claim 20, wherein the correlation coefficient, Separation method which is individual gait by clustering foreground pixels, which is calculated by the following equation.

In this case, ρ _jk is a correlation coefficient and has a distribution between [-1, 1]. X is a random variable representing a coordinate value of a foreground pixel, which is a corresponding probability distribution for calculating an expected value to obtain covariance and variance. The method of claim 20, wherein the correlation coefficient, Separation method which is individual gait by clustering foreground pixels, which is calculated by the following equation.

In this case, n represents the total number of foreground pixel samples. 23. The method of claim 22, wherein the parameters μ _j and μ _k of each cluster are calculated by the following equations, respectively.

,

The method of claim 1, wherein identifying the individual walker is: The method of claim 1, wherein the merging of the two clusters is performed prior to the removing of the clusters. The method of claim 1, wherein identifying the individual walker is: And if the normalized post score of the corresponding cluster is greater than a fourth boundary value, determining the corresponding cluster as one individual pedestrian. The method of claim 25, wherein the normalized post score is: Separation method of the individual walk by clustering the foreground pixels, characterized in that calculated as shown in the following equation.

In this case, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the group ω _j . Sum of posterior probabilities constituting the molecule of the equation

Denotes the number of foreground pixel samples belonging to the cluster ω _j . The method of claim 1, wherein identifying the individual walker is: The normalized post-score for each of the two selected clusters is less than a fourth boundary value, and when the correlation coefficient between the two clusters is greater than a third boundary value, the two clusters are merged. Separation method which is individual gait by clustering of pixels. The method of claim 27, wherein the merging between the two clusters, Separating method for individual walks by clustering foreground pixels, wherein the group of clusters to be merged is determined using a depth-first search method. The method of claim 27, wherein the normalized post score is: The separation method of the individual walk by clustering the foreground pixels, characterized in that calculated as follows.

In this case, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the group ω _j . Sum of posterior probabilities constituting the molecule of the equation

Denotes the number of foreground pixel samples belonging to the cluster ω _j . The method of claim 27, wherein the correlation coefficient, Separation method which is individual gait by clustering foreground pixels, which is calculated by the following equation.

In this case, n represents the total number of foreground pixel samples. 31. The method of claim 30, wherein the parameters μ _j and μ _k of each cluster are calculated by the following equation, respectively.

,

The method of claim 1, wherein identifying the individual walker is: If the normalized post score of the corresponding cluster is smaller than a fourth boundary value, and the correlation coefficient with all other clusters among the clusters is smaller than a third boundary value, the corresponding cluster is removed. Separation method of individual gait by clustering. 33. The method of claim 32, wherein removing the population is A method for separating individual walks by clustering foreground pixels, wherein the normalized post scores of the clusters are performed in ascending order from the smallest cluster. The method of claim 32, wherein the normalized post-mortem score, Separation method of the individual walk by clustering the foreground pixels, characterized in that calculated as shown in the following equation.

In this case, x _i represents each foreground pixel sample, and A _j represents the area of the pixel unit measured in the image of the group ω _j . Sum of posterior probabilities constituting the molecule of the equation

Denotes the number of foreground pixel samples belonging to the cluster ω _j . The method of claim 32, wherein the correlation coefficient, Separation method which is individual gait by clustering foreground pixels, which is calculated by the following equation.

In this case, n represents the total number of foreground pixel samples. 36. The method of claim 35, wherein the parameters μ _j and μ _k of each cluster are respectively calculated by the following equation.

,

A computer-readable recording medium containing a program capable of performing the method according to any one of claims 1 to 36.

Images