JPH06319108A - Scene change detector - Google Patents

Abstract

PURPOSE:To improve the detection accuracy of a scene change by expressing at which part of a picture and what kind of a color are in existence in terms of a distribution vector and calculating a distance from a picture element to the distribution vector so as to obtain a change in a content between frames with higher accuracy. CONSTITUTION:A color vector of a picture of a position vector generated at random is read out of a frame memory 101 storing in a current frame of a motion picture to combine a characteristic vector. Clustering is executed by using a distribution vector decision section 104 and a distribution vector revision section distribution vector storage memory 107 and a distribution parameter comprising a mean value and a covariance of colors and positions of a picture element subjected to clustering is obtained as the distribution vector. After a succeeding frame is read, a distance up to the distribution vector closest to the picture element selected at random is obtained and the number of picture element decided to have correlation is counted by a pixel count section 108 and the result is used for the correlation. Then a decision section 109 detects a scene change based on the correlation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a scene change detecting device capable of detecting a change in a scene of an image.

[0002]

2. Description of the Related Art Regarding a conventional scene change detecting device,
Japanese Unexamined Patent Publication No. 3-214364 discloses an apparatus that utilizes changes in a luminance histogram. FIG. 4 is a functional block diagram of the scene change detection device. The operation of the scene changing device based on the change in the luminance histogram will be described with reference to this figure. The brightness determination unit 401 analyzes the input signal and determines the brightness. The histogram creation unit 402 creates a histogram based on the determined brightness. The result is alternately input to the histogram storage unit 1 (403) or the histogram storage unit 2 (404). This alternate input is for storing the results of two adjacent moving image frames. The histogram difference absolute value calculation unit 405 stores the results of the histograms of two consecutive frames in the storage units 403 and 4.
The value is read from 04, and the absolute value of the difference between the pixel values for each luminance, that is, the pixel frequency is calculated. The histogram difference summation calculation unit 406 sums the difference absolute values,
If the total sum is larger than the threshold value set by the threshold value setting unit 408, the scene change determination unit 407 outputs a result indicating that the scene is a break. It should be noted that the histogram may use the brightness that is the gray value of the image, and the color image may use the combination of the three primary colors RGB.

[0003]

However, in the conventional configuration as described above, since only the change in the luminance histogram of the pixel is focused on, the same luminance, although the contents are different,
If the images of the color scheme are continuous, there is a possibility that the scene change will not be detected. That is, the degree of scene change may be underestimated.

On the other hand, as an improvement of the device based on the brightness histogram, there is a technique of dividing an image into a grid shape and paying attention to the change of the brightness histogram in each grid (for example, Nagasaka and Tanaka: in a color video image. Automatic Search Indexing and Object Search Method, IPSJ Transactions, Vol.33, N
o.4,1992.). However, in this case, if the histogram divided over the entire screen changes due to the camera swing, it may be erroneously detected as a scene change.

An object of the present invention is to provide a scene change detecting apparatus that is more faithful to changes in the content of a moving image, in consideration of the problems of the conventional scene changing apparatus as described above.

[0006]

According to the present invention, there is provided distribution vector generation means for clustering pixels of an image with feature amounts and positions of the pixels as feature vectors to generate a distribution vector set including distribution parameters of the feature vectors. , A plurality of pixels are extracted from the image, the feature amount and position of the pixel are used as the feature vector, and the correlation value of the image is calculated based on the distance corresponding to the error to the feature vector distribution of the pixel described by the distribution vector. By having the correlation calculating means for obtaining and applying the distribution vector generating means to the frames before and after the moving image, the distribution vectors are sequentially generated,
The scene change detecting device detects a change in image content by measuring a correlation value by the correlation calculating means.

[0007]

With respect to the frames in the moving image, the distribution vector generation means clusters the characteristic vectors consisting of the characteristic amounts and positions of the pixels to generate a distribution vector set consisting of distribution parameters of the characteristic vectors. For the next frame, the correlation calculation means reads out the feature vector including the feature amount and the position of the pixel, finds the distance corresponding to the error to the feature vector distribution of the pixel described by the distribution vector, Based on this, the correlation value of the image is obtained.

Accordingly, when the correlation value is low, it is considered that there is a scene change. When the entire image flows to the left and right due to the camera swing, the distance corresponding to the error in the feature vector distribution of the pixels described by the distribution vector increases, but it is smaller than the increase in the distance due to the scene change. Therefore, an effective scene change can be detected.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a functional block diagram of a scene change detection apparatus according to an embodiment of the present invention. FIG. 2 is a diagram for explaining the operation of the scene change detection device in this embodiment. FIG. 3 is an explanatory diagram of the feature distribution in this embodiment. First, the basic prerequisites of this embodiment will be described. In this embodiment, color is used as the feature amount of the pixel. Let this be x, which is a column vector of RGB. The pixel position is a column vector of horizontal position ξ and vertical position η p = (ξ, η) ^t
( ^T represents transposition of matrix and vector). Then, the feature vector of the pixel is set to (x, p). In the present embodiment, the images are clustered to be divided into a maximum of N clusters, and the images are represented by a set of distribution vectors composed of distribution parameters of feature vectors in each cluster. This concept is shown in FIG. An image is considered to be composed of a plurality of small areas, and the small areas are represented by a distribution vector. The distribution vector is the average of color vectors x
And the covariance X, the average p of the position vector p, and the covariance P. Let θ _{i be the} distribution vector set
= (X _i , X _i , p _{i i} , P _i ), 1 ≦ i ≦ N.

In FIG. 1, 101 is a frame memory,
Reference numeral 102 denotes a random address generation unit that randomly generates a position vector p of an image, and 103 uses the position vector p obtained by the random address generation unit 102 to read the color vector x of the pixel at that position from the frame memory 101, A feature vector synthesis unit that synthesizes and outputs the feature vector (x, p), and 104 is the feature vector (x, p
p), a distribution vector determination unit that determines a distribution vector, 105 a control unit that controls each of the units 106 and 108 described below, 106 a distribution vector change unit that changes the distribution vector, and 107 stores N distribution vector Vector storage memory with slots for 1 to N, 1
Reference numeral 08 is a correlation pixel counting unit that receives the output of the distribution vector determination unit 104 and counts correlation pixels, and 109 is a determination unit that determines using the correlation pixel count value. A distribution vector generation unit is configured by the feature vector synthesis unit 103, the distribution vector determination unit 106, the distribution vector storage memory 104, the distribution vector change unit 106, and the like.

With the above-described structure, this embodiment can
After performing clustering on frames, correlation calculation and clustering are performed before and after the second frame. The control unit (105) switches between the correlation calculation and the clustering.

That is, first, the pixels of the search image are classified from the feature vector (x, p). Figure 2
In the example shown in, clustering is performed on the first frame 201, and the first frame clustering result 20
2 has been obtained. In FIG. 2, one cluster is displayed as an ellipse. The result of clustered pixel clusters is not always guaranteed to form an elliptical region, but since position is used as a feature vector in addition to color, a cluster of pixels with similar colors and close distances is one cluster. To form. As a result, for example, one distribution vector is assigned to each of the black part of hair and the flesh-colored part of the face. In this way, by clustering the feature vectors of pixels in which the color vector and the position vector are combined, "where and what color distribution is present" are expressed. Although various methods have been proposed for this clustering, in this embodiment, they are realized by the competitive learning method. This is shown below.

Step 1: Each slot of the distribution vector memory 107 is associated with a valid flag which takes a binary value of 1,0. When 1 is valid, when 0 is invalid. First, the valid flags of c slots (c ≦ N) of the N slots are set to 1 and the others are set to 0. The distribution vector set of discrete time t is represented by θ _i (t) = (x− _i (t), X _i (t), p− _i (t), P _i
(T)), 1 ≦ i ≦ N. Initialize the memory contents of the distribution vector slot that is 1 of the valid flag. Initialization is performed assuming that the covariance matrices X _i (0) and P _i (0) are unit matrices,
The average P _i of the average x¯ _i (0) and the position vector p color vector x (0) is set at random. The discrete time t at the time of initialization is 0.

Step 2: The feature vector synthesizing unit 103 reads the color vector x of the pixel at that position from the position vector p obtained by the random address generating unit 102 and synthesizes the feature vector (x, p). Output.
The distribution vector determination unit 103 calculates the distance from the feature vector for the distribution vector in the slot in which the valid flag is 1 among the N slots. This distance is evaluated by (Equation 1).

[0016]

[Equation 1]

Here, | X | and | P | are determinants of each covariance matrix. (Equation 1), when the distribution of the pixel are independently color vector x and the position vector p are each an average x¯ _i, p¯ _i and covariance X _i, was assumed to be normally distributed with P _i, It is a statistical measure that can be obtained from the logarithm (log likelihood) of the probability density function. Of the N slots, the distance is calculated for the distribution vector whose effective flag is 1, and the number of the slot having the shortest distance is k, and the distribution vector determination unit 104 sets the number k and the sample to the distribution vector change unit 1.
Send to 06. The distribution vector changing unit 106 changes the distribution vector of the slot number k to be closer to the sample under the distance function of (Equation 1) (Equations 2, 3, 4, 5). (Equation 6) is an experimentally determined gain coefficient.

[0018]

[Equation 2]

[0019]

[Equation 3]

[0020]

[Equation 4]

[0021]

[Equation 5]

[0022]

[Equation 6]

The time t is incremented by 1.

The above step 2 is performed a predetermined number of times (for example, 1
By repeating (0000 times), pixel clustering by self-organization is performed. That is, one cluster is formed by modifying the distribution parameter data so that the distribution vector closest to the randomly selected feature vector is closer to the sample. Random address generation is to prevent bias when changing the successive distribution vector.

During the repetition of step 2, the distribution vector changing unit 106 satisfies the condition 1: (total number of distribution vector slots N)-(currently effective distribution vector slot number c)> 0, and condition 2: ln | X _i | + ln | P _i |
Is greater than or equal to a predetermined threshold, the cluster represented by the distribution vector θ _i is divided. For this purpose, the distribution vector θ _i is duplicated in the distribution vector slot j whose valid flag is 0 (invalid), and θ _j = θ _i is created. And each,
The average vector is adjusted by ε. The value of ε is experimentally determined.
This is shown in (Equations 7, 8, 9, 10).

[0026]

[Equation 7]

[0027]

[Equation 8]

[0028]

[Equation 9]

[0029]

[Equation 10]

Then, the effective flag of the slot j is set to 1, the effective distribution vector slot number c is increased by 1, and used for the repetition of self-organization thereafter. As a result, ln |
A cluster having a large X _i | + ln | P _i | is divided.

Further, during the repetition of step 2, the valid flag of the slot of the distribution vector for which the randomly selected sample vector has never been obtained for a predetermined period, that is, the shortest distance is not set, is set to 0. , The effective distribution vector slot number c is decremented by 1. This is necessary to cancel the parameters of the distribution vector that have converged to the wrong value.

By the above processing, a distribution vector set consisting of N distribution vectors including invalid distribution vectors is generated for the image to be searched. In the first frame clustering result 202 of FIG. 2, the image is represented by 14 distribution vectors.

Next, the second frame is set to the frame memory 101.
To enter. The control unit 105 includes the distribution vector changing unit 10
The operation of 6 is stopped, and the operation of the correlated pixel counting unit 108 is activated. The distribution vector determination unit 104 performs the same operation as at the time of clustering. That is, in N slots, the distance is calculated for the distribution vector whose effective flag is 1, and the distribution vector k which is the shortest distance is obtained. And (Equation 1)
Among these, only the component shown in (Equation 11) excluding the value of the determinant of covariance is sent to the correlated pixel counting unit 108 as the distance χ.

[0034]

[Equation 11]

Here, the correlation pixel counting unit 108 counts pixels having the shortest distance χ <threshold value T. Specifically, the initial value of the correlation value is set to 0, and when the shortest distance equal to or less than the threshold value T is input from the distribution vector determination unit 104, the number is increased by 1. This is repeated a predetermined number of times (for example, 10,000 times). here,
Distance χ from the feature vector, the mean X - _k, a Mahalanobis distance is normalized distance to p¯ _k. The Mahalanobis distance follows a χ ² distribution, assuming that the feature vector is normally distributed. Therefore, as the threshold value T, the threshold value used for the χ ² test can be used. For example, in a χ ² distribution with 5 degrees of freedom, assuming that T = 16.749, it is expected that 99.5% of pixels are counted as the threshold value T or less in the same image. The counted number of pixels is sent to the determination unit 109 as a correlation value. If this correlation value is less than or equal to a predetermined threshold value, it is considered that the correlation between consecutive frames is weak, and a scene change signal is output.

As described above, at the time of the correlation calculation, the correlation value is calculated by repeating the reading of the random feature vector, the determination of the distribution vector, the calculation of the normalized distance χ, and the counting of the correlated pixels by the threshold processing of the normalized distance χ. Determines the scene change. After that, the distribution vector description of the current frame by clustering, the correlation calculation with the input of the next frame,
Repeat.

The second frame 203 in FIG. 2 has a correlation with the first frame 201, and the correlation value is large. After the second frame, the clustering result of the previous frame is used as the initial value of clustering. In FIG. 2, the second frame 203 is clustered to obtain a clustering result 204.
Is getting Then, the same correlation calculation is performed in the third frame 2
I am going to 05. In this case, since there is a change in the scene between the second frame 203 and the third frame 205,
The correlation value becomes small and is detected as a scene change.

According to the present embodiment described above, by expressing the existence of what kind of color in the image by a distribution vector and calculating the distance to the distribution vector, it is possible to calculate the distance between frames. Changes in content can be obtained more precisely. Moreover, since the correlation value conforms to the χ ² distribution, the threshold value can be set as a statistical parameter.

Although the feature amount of the pixel of the present invention uses color in the above embodiment, it is not limited to this.

Further, the clustering of the present invention is not limited to the system of the above embodiment, and other clustering system can be adopted.

Further, each means of the present invention can be realized by software using a computer, or can be realized by using a dedicated hardware circuit having the respective functions.

[0042]

As is apparent from the above description,
According to the present invention, the following effects can be obtained.

(1) Where in the image, for example, what kind of color exists, is expressed by a distribution vector, and the distance to the distribution vector is calculated, so that the change in the content between frames can be further reduced. It can be obtained with precision and can detect scene changes more accurately than with conventional luminance histogram based devices. For example, even if images with the same color arrangement are consecutive, if the positions are different, it can be detected as a change in the scene.

(2) The distribution vector represents the distribution of features depending on the position. Therefore, by adding this result to each frame of the moving image, it can be applied to processing such as image retrieval.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of a scene change detection device of the present invention.

FIG. 2 is an operation explanatory diagram in the above embodiment.

FIG. 3 is an explanatory diagram of a feature vector distribution in the above embodiment.

FIG. 4 is a block diagram of a conventional scene change detection device.

[Explanation of symbols]

 101 frame memory 102 random address generator 103 feature vector combiner 104 distribution vector determiner 105 controller 106 distribution vector changer 107 distribution vector storage memory 108 correlation pixel counter (correlation calculation means) 109 judgment unit 201 first frame 202 1-frame clustering result 203 Second frame 204 Second-frame clustering result 205 Third frame

Claims (2)

[Claims] 1. A distribution vector generation unit that clusters pixels of an image using a feature amount and a position of the pixel as a feature vector to generate a set of distribution vectors composed of distribution parameters of the feature vector; Correlation calculation for obtaining the correlation value of the image based on the distance corresponding to the error to the feature vector distribution of the pixel described by the distribution vector by using the feature amount and the position of the pixel as the feature vector. Means for sequentially generating distribution vectors by the distribution vector generation means for the preceding and succeeding frames of the moving image, and measuring the correlation value by the correlation calculation means to detect a scene change of the image. Characteristic scene change detection device. 2. The scene change detection device according to claim 1, wherein the characteristic amount of the pixel is a color.