JPH0779431A - Scene change detection circuit for digital picture signal - Google Patents

Abstract

PURPOSE:To detect a scene change of a digital picture signal with high accuracy. CONSTITUTION:A picture is deframed by an interleave circuit 2 and a least square method arithmetic operation circuit 3 decides a coefficient minimizing a square sum of errors when the picture of a deframed noticed frame is estimated by linear coupling of data between and preceding and succeeding frames. Coefficients corresponding to the preceding frame in the coefficients are added by an adder 21 and the sum SUMp is fed to a comparator 24. Coefficients corresponding to the succeeding frame are added by an adder 22 and the sum SUMn is fed to a comparator 25. When the SUMp is large and a SUMn is small through the threshold level discrimination of the comparators 24, 28. The coefficient is decided to be an object of a scene change. When each of the absolute values of the succeeding coefficients is confirmed to be smaller, an object of the scene change is decided as the occurrence of the scene change.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a scene change detection circuit for digital image signals.

[0002]

2. Description of the Related Art A scene change of a digital image signal means that the correlation of images in the time direction is broken.
Therefore, scene change detection is required in predictive coding, Y / C separation for separating a luminance signal and a chrominance signal, and processing of a digital image signal such as a noise removing circuit. Further, in an image recording device such as a digital VTR or a disc recording device, it is possible to store an image immediately after a scene change and use it as an index of the recorded image.

A conventional scene change detection circuit finds the difference between the values of a plurality of pixels belonging to the current frame and the values of a plurality of pixels belonging to the previous frame, and determines the absolute value of the difference. A method is known in which a sum of values (or a sum of squares) is calculated, and when it is larger than a threshold value, it is detected that a scene change has occurred.

[0004]

The conventional scene change detection circuit has a problem in that when the overall brightness of an image changes due to the influence of illumination or the like, it is erroneously detected as a scene change. In addition, it is difficult to set the threshold value correctly.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a scene change detection circuit which can detect a scene change more accurately.

[0006]

SUMMARY OF THE INVENTION According to the present invention, in a scene change detection circuit for a digital image signal, an input digital image of a target frame is obtained by linearly combining pixels and coefficients included in frames before and after the target frame. Calculating means for expressing the signal and calculating the coefficient of the linear first-order combination by the least-squares method, and detecting the ratio of the calculated coefficient of each of the previous frame and the subsequent frame, which contributes to the estimation of the frame of interest. Then, the scene change detection circuit comprises means for determining the occurrence of a scene change by judging the detected ratio by a threshold value.

[0007]

The image of the frame of interest in the center is represented by linear linear combination of the data and the coefficients of the preceding and following frames by using the images of three frames which are temporally continuous. The coefficient with the smallest sum of squared error is determined. In a continuous scene, since temporal and spatial correlations exist, the contributions of preceding and succeeding frames are approximately equal to the estimation of the frame of interest. on the other hand,
When a scene change occurs, this correlation is broken, so that the contribution ratios of the preceding and following frames are significantly different. This difference can be identified from the coefficient in the previous frame and the coefficient in the subsequent frame.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A scene change detection circuit according to the present invention will be described below. To facilitate understanding of the present invention, description of an image by a spatiotemporal model will be described first with reference to FIG.

FIG. 1 shows a model of 1/2 frame drop,
T0, T1, and T2 indicate three temporally consecutive frames. The frame T1 at the center is a frame on which frames are dropped. As will be described later, the frame T1 is a frame of interest to be detected as a scene change. This frame T
The pixel value y included in 1 is represented by a linear linear combination of the pixel value and the coefficient of the previous frame T0 and the subsequent frame T2.

More specifically, frames T0 and T2
From (3), the (3 × 3) regions at the same spatial position are cut out. Two regions form one three-dimensional block. As will be described in more detail below, the value y of the center pixel of frame T1 is linearly proportional to the pixel value x _i
In order to minimize the square of the error of the data expressed by the linear combination model and expressed by the linear linear combination with respect to the actual data,
The coefficients are determined by the method of least squares. One frame defines one set of coefficients.

In the spatiotemporal model shown in FIG. 1, a block including two regions contains a total of 18 pixels. The value of this pixel is x _i (i = 1, 2, ..., 1
8). The coefficient to be multiplied to each pixel represents the w ₁ to w _18. If the value of the central pixel of the block of the frame T1 is y, this value is set to another frame T0,
It is represented by a linear primary combination x _i w _i of the pixel of T2 and the coefficient.
That is, the value y at the center position of the frame T1 is thus the linear linear combination w ₁ x ₁ + w ₂ of the input pixels of 18 taps.
It is represented by x ₂ + ... + w ₁₈ x ₁₈ . This linear 1
Regarding the coefficient w _i in the second-order coupling model, the coefficient that minimizes the residual between the actual value and the value represented by the linear first-order coupling is determined.

In order to determine the undetermined coefficient w _i , the pixel value x _i (i of the block shown in FIG. 1 when the input image is shifted by one pixel in the spatial direction (horizontal direction and vertical direction)
= 1, ..., N) and the actual value of the pixel to be interpolated y _j (j
= 1, ..., M) are respectively substituted to create a linear linear combination formula. In the example here, (n = 18).
For example, when obtaining one set of coefficients for one frame,
A large number of equations, that is, simultaneous equations (referred to as observation equations) of the number of pixels (= m) in one frame are created by shifting the cutout of the block pixel by pixel for the image of one frame. . A minimum of (m = 18) simultaneous equations are required to determine the 18 coefficients. The number m of equations can be appropriately selected in consideration of the problem of accuracy and the processing time. The observation equation is XW = Y (1). Here, X, W, and Y are the following matrices, respectively.

[0013]

[Equation 1]

The coefficient w that minimizes the error from the actual value is obtained by the least squares method. For this purpose, the following residual equation is created by adding the residual matrix E to the right side of the observation equation. That is, in the least squares method, the coefficient matrix W that minimizes the square of the elements of the residual matrix E in the residual equation, that is, the squared error is obtained.

[0015]

[Equation 2]

Next, the condition for finding the most probable value of each element w _i of the coefficient matrix W from the residual equation (3) is to square the m residuals corresponding to the pixels in the block and sum them. It suffices if the condition for minimizing is satisfied. This condition is expressed by the following equation (4).

[0017]

[Equation 3]

It suffices to enter _{n conditions} and find the undetermined coefficients w ₁ , w ₂ , ..., W _n that are the elements of the coefficient matrix W that satisfy these conditions. Therefore, from the residual equation (3),

[0019]

[Equation 4]

[0020] The condition of Expression (4) is i = 1, 2, ...
.., n)

[0021]

[Equation 5]

Is obtained. The following normal equation is obtained from the equations (3) and (6).

[0023]

[Equation 6]

The normal equation (7) is a simultaneous equation having exactly n unknowns. As a result, each undetermined coefficient w _i that is the most probable value can be obtained. To be precise, if the matrix of w _i in equation (7) is regular, it can be solved. Actually, the undetermined coefficient w _i is obtained by using the Gauss-Jordan elimination method (also known as the sweeping method). In this way, one set of coefficients for representing the pixel of the frame of interest is determined in one frame.

This embodiment uses the determined coefficients to detect between frames T1 and T2 (ie the rear scene change). FIG. 2 shows an example of coefficients determined as described above in the case of continuous scenes, and FIG. 3 shows an example of coefficients when a scene change occurs on the rear side. w _{1 to} w ₉ are the pixel data x _{1 of the} previous frame
Is a coefficient to be multiplied respectively ~x _9, a coefficient w ₁₀ to w ₁₈ is multiplied to the pixel data x ₁₀ ~x ₁₈ of the rear frame. Since the scene change breaks the correlation between the spatiotemporal images, as can be seen from FIG. 3, when the scene change occurs, the coefficient for the pixel data of the subsequent frame T2 is substantially zero. The present invention is intended to detect a scene change based on such a characteristic having a coefficient.

FIG. 4 is a block diagram of an example of the scene change detection circuit according to the present invention. The digital image data from the input terminal 1 is supplied to the thinning circuit 2 to be thinned out. One example is the ½ frame drop shown in FIG. Frames T0 and T from thinning circuit 2
Two pieces of data are supplied to the arithmetic circuit 3 of the least square method.

The actual data of the target frame T1 is also supplied from the input terminal 1 to the arithmetic circuit 3. In the least-squares arithmetic circuit 3, for a spatiotemporal model as shown in FIG. 1, for example, one set of coefficients w _i is determined in one frame by an algorithm of the least-squares method. From the arithmetic circuit 3,
The deterministic coefficient is output. This coefficient is stored in the coefficient memory 4. Using the coefficients stored in the memory 4, scene change detection processing is performed as described later. The coefficient from the arithmetic circuit 3 may be treated as output data of high efficiency coding. That is, it is possible to transmit data and coefficients that are not decimated, and the receiving side can interpolate the decimated data from these data.

Here, FIG. 5 shows an arithmetic circuit 3 of the least squares method.
A more detailed configuration of the above is shown. Data that is supplied with an input digital image signal and that constitutes a spatiotemporal model, that is, the actual data y of the target pixel and the data x used for linear linear combination
A time series conversion memory 11 for synchronizing _i is provided. The data from the time series conversion memory 11 is supplied to the multiplier array 12. An addition memory 13 is connected to the multiplier array 12. The multiplier array 12 and the addition memory 13 form a normal equation generation circuit.

The multiplier array 12 multiplies each pixel, and the addition memory 13 is composed of an adder array to which the multiplication result from the multiplier array 12 is supplied and a memory array. FIG. 6 is a specific configuration of the multiplier array 12. In FIG. 6, as one of them is enlarged and shown, a square cell represents a multiplier. The multiplication of each pixel is performed in the multiplier array 12, and the result is supplied to the addition memory 13.

As shown in FIG. 7, the adder memory 13 has an adder array 13a and a memory (or register, the same applies hereinafter) array 13b connected in series.
The output of b is fed back to the adder array 13a. A product-sum operation is performed by these multiplier array 12, adder array 13a, and memory array 13b. Looking at the term of the product-sum operation related to w _i in the above-mentioned normal equation (7), if the upper right term is inverted, it becomes the same as the lower left. Therefore, (7)
It suffices to divide the equation diagonally by the line from the upper left to the lower right and calculate only the terms included in the upper triangular portion. From this point, the multiplier array 12, the adder array 13a, and the memory array 13b are, as shown in FIGS. 6 and 7, a multiplication cell or a memory which is required to calculate the terms included in the upper triangular portion. It has a cell.

As described above, the product-sum operation is performed as the input image arrives, and the normal equation is generated. The result of each term of this normal equation is stored in the memory array 13b, and then, as shown in FIG. 5, each term of this normal equation is calculated by the CPU arithmetic circuit 14 of the sweep method. Normal equations (simultaneous equations) are solved by calculation using the CPU, and the coefficient that is the most probable value is obtained. This coefficient is output.

Returning to FIG. 4, the scene change detection circuit which is a feature of the present invention will be described. The coefficients stored in the coefficient memory 4 are read out and supplied to the adders 21, 22 and the absolute value conversion circuit 23. The adder 21 sums the sums SUMp (= coefficients w ₁ to w ₉ related to the previous frame T0.
w ₁ + w ₂ + ... + w ₉ ) and the adder 22 generates
Sum SUMn of coefficients w ₁₀ to w ₁₈ related to the rear frame T2
_{(= W 10 + w 11 +} ··· + w 18) to generate. Absolute value circuit 23 converts the value of the coefficient w ₁₀ to w ₁₈ to an absolute value.

As in the example of FIG. 2 described above, in consecutive scenes, SUMp and SUMn are 0.5 respectively.
Often shows a value close to. On the other hand, as in the example of FIG.
When a scene change occurs on the rear side of the frame of interest, SUMp shows a value close to 1 and SUMn shows a value close to 0 in many cases. This is because the scene change breaks the spatiotemporal correlation with the previous image.

Therefore, as the first stage processing, the adder 2
The outputs of 1 and 22 are supplied to comparators 24 and 25,
Compare with thresholds Th1 and Th2. As an example,
Th1 = 0.8 and Th2 = 0.2 are selected. The output of the comparator 24 (high level when SUMp ≧ Th1) and the inverted output of the comparator 25 (high level when SUMn ≦ Th2) are supplied to the AND gate 26. The flag 0 from the AND gate 26 is supplied to the register 27, and the register 1 generates the flag 1 delayed by one clock.

When the flag 0 (flag 1) is at the high level, (SUMp ≧ Th1) and (SUMn ≦ Th
Means 2). If this condition is satisfied, it is determined as a scene change candidate. If either one of the conditions is not satisfied, it is not considered as a scene change candidate.

Next, a second judgment process for confirming whether or not the scene change candidate is really a scene change is performed. This is because the absolute value of each coefficient corresponding to the subsequent frame T2 is checked, and all values have a certain threshold value Th3 (for example, 0.
If it does not exceed 1), it means that there is a scene change. Therefore, the output of the absolute value conversion circuit 23 is supplied to the comparator 28 and compared with the threshold value Th3. The flag 1 from the register 27 is supplied to the comparator 28 as an enable EN. When EN is at a high level, that is, when the result of the first determination process means a candidate for a scene change, the comparator 28 is active.

The inverted comparison output of the comparator 28 is supplied as a reset signal to the register 30 via the register 29. The register 30 receives the flag 1 from the register 27 and outputs the final flag 2.
This flag 2 indicates the occurrence of a scene change at a high level and indicates a continuous scene at a low level. Therefore,
If the flag 1 is at the high level and the register 30 is not reset, the flag 2 is at the high level, and the flag 2 indicates the occurrence of the scene change.

The inverted output of the comparator 28 described above is (| w ₁₀
It is at a low level when | ˜ | w ₁₈ |> Th3). The output of low level resets the register 30 and sets the flag 2 to low level. Therefore, | w ₁₀ | ≦ Th3and | w ₁₁ | ≦ Th3and | w ₁₂ | ≦
_{Th3and ···· and | w 17 | ≦} Th3and | w 18 | when ≦ Th3 is satisfied, the register 30 is not reset, a result, the scene change candidate is output as a flag 2.

In this example, the occurrence of a scene change on the rear side is detected. By sequentially performing the processing in which each frame is set as the frame of interest, all the scene changes on the rear side can be detected. The detection result of the scene change is delayed with respect to the input signal by one frame or more. This compensation can be compensated by a memory or a delay circuit.

It is also possible to adopt a configuration in which every other frame is set as a frame of interest and both the front and rear scene changes are detected. To detect the occurrence of a scene change on the rear side,
This is done in the same manner as in the above-described embodiment. The occurrence of the scene change on the front side is detected when the following conditions are satisfied. SUMp ≦ Th2 (eg 0.2) and SUMn ≧ T
h1 (eg 0.8)

Further, the spatiotemporal model is not limited to that shown in FIG. By increasing the number of taps of the linear linear combination, the spatial spread is increased, and the possibility that the motion of the image is erroneously detected as a scene change can be reduced. However, an increase in the number of taps causes a problem of increasing the calculation time. To deal with this problem, subsampling may be performed.

FIG. 8 shows an example of performing sub-sampling in a five-eye grid pattern in the front and rear frames T0 and T2. The number of pixels is halved by this sub-sampling
The spatial spread can be increased while suppressing the increase in the calculation time. Of course, 1/4 subsampling, 1/8 subsampling, or the like can also be adopted.

[0043]

As is apparent from the above description, according to the present invention, a scene change can be detected more accurately without being affected by the change in brightness due to illumination or the like. Further, the present invention can provide a scene change detection circuit having good matching in a system in which sub-sampling is performed in order to compress the amount of transmission data and transmission pixel data and coefficients are transmitted.

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining a spatiotemporal model according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing an example of coefficients of linear linear combination in the case of continuous scenes.

[Figure 3] Linear 1 when a scene change occurs on the rear side
It is an approximate line figure showing an example of the coefficient of the next combination.

FIG. 4 is a block diagram of an embodiment of the present invention.

FIG. 5 is a block diagram of an example of a least square method arithmetic circuit.

FIG. 6 is a schematic diagram for explaining a multiplier array included in a least square method arithmetic circuit.

FIG. 7 is a schematic diagram for explaining an adder array and a memory array included in a least square method arithmetic circuit.

FIG. 8 is a schematic diagram of another example of the spatiotemporal model to which the present invention can be applied.

[Explanation of symbols]

 2 Thinning circuit 3 Least square method arithmetic circuit 21, 22 Adder 23 Absolute value conversion circuit 24, 25, 28 Comparator

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI technical display location H04N 5/91

Claims (4)

[Claims] 1. A scene change detection circuit for a digital image signal, wherein an input digital image signal of the frame of interest is expressed by linear linear combination of pixels and coefficients included in frames before and after the frame of interest, and the linear With respect to the calculating means for calculating the coefficient of the linear combination by the least squares method and the calculated coefficient of each of the previous frame and the subsequent frame, the ratio contributing to the estimation of the frame of interest is detected and detected. A scene change detection circuit comprising means for determining the occurrence of a scene change by determining a percentage as a threshold value. 2. The scene change detection circuit according to claim 1, wherein the scene change determining means further comprises means for determining the occurrence of a scene change when all the coefficients of the preceding or succeeding frames are almost zero. Scene change detection circuit having. 3. The scene change detection circuit according to claim 1, wherein the scene change determining means determines that the coefficient of one frame is greater than the predetermined value when the sum of the coefficients of one of the preceding and succeeding frames is greater than a predetermined value. A scene change detection circuit adapted to determine that the contribution to estimation is large. 4. The scene change detection circuit according to claim 1, wherein the scene change determining means determines that the coefficient of the subsequent frame has a coefficient when the total value of the coefficients of one of the preceding and succeeding frames is smaller than a predetermined value. A scene change detection circuit adapted to determine that the proportion contributing to the estimation is small.