KR100729660B1 - Real-time digital video identification system and method using scene change length - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital video recognition system using a scene change length, comprising: a scene change detector for receiving a digital video, extracting a difference between frames, and calculating a scene change length using the extracted difference; A digital video database storing their corresponding scene transition lengths, and a scene transition length calculated by the scene transition detector, query the digital video database for a query type query and return the response to the query. By including a digital video comparator that compares the scene transition length, there is an effect of enabling digital video to be searched and identified in real time.

Video properties, transition detection, transition length, video recognition system

Description

Digital video recognition system and method using scene change length {REAL-TIME DIGITAL VIDEO IDENTIFICATION SYSTEM AND METHOD USING SCENE CHANGE LENGTH}

1 is a diagram illustrating a configuration of a digital video recognition system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating the configuration of the scene change detector of FIG. 1 in more detail.

3 is a diagram representing a frame of digital video with respect to the time axis,

4 is a view for explaining the concept of a scene change used in the digital video recognition system according to an embodiment of the present invention,

5 is a diagram illustrating an example of a scene change occurring in actual digital video.

FIG. 6 is a diagram illustrating a signal input to the interframe difference extractor of FIG.

FIG. 7 is a diagram illustrating a simple process of the scene change detection filter of FIG. 2.

8 is a diagram illustrating an example of establishing a scene change length in a digital video database according to an embodiment of the present invention.

9 is a flowchart illustrating a processing flow of a digital video recognition method using a scene change length according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

10: Scene Transition Detector 20: Digital Video Database

30: digital video comparator

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to digital video recognition technology, and more particularly, to a digital video recognition system using a scene change length. More particularly, a real-time search and identification of digital video by effectively constructing a database using the scene change length of digital video. The present invention relates to a digital video recognition system using a scene change length to enable the operation.

The development of digital video acquisition technology and processing technology and the spread of various video tools have made it possible for multimedia users to easily manipulate digital video. In addition, many multimedia users are demanding high quality and high capacity digital video as network and storage media advances.

However, in multimedia services that must process high quality and high capacity digital video, fast search and accurate comparison of digital video are recognized as an important problem. For example, in the case of a system for monitoring commercials, it is impossible to search for commercials that are currently being broadcast in the database through comparison between frames, because a large number of commercials must be stored in a database and provide services in real time. .

Therefore, in such a database of digital video, it is necessary to manage these digital videos by using attribute information of each digital video. The present invention relates to a technique of using scene change length (shot length) of digital video among various attribute information. For example, every digital video has a scene, which is a set of semantic frames, and different digital videos have different lengths of scenes constituting the digital video. The present invention relates to a technology for retrieving or identifying digital video using the above properties, and implements a digital video recognition system using a scene change length.

On the other hand, with respect to technology using scene change, a number of prior patents have already been published.

Korean Patent Application No. 10-70567 (filed date: November 24, 2000) includes a method for detecting a scene change in a video stream compressed by the MPEG video compression coding standard. Is described. According to the method, a Discrete Cosine Transform (DCT) used to remove spatial redundancy is first applied, and then a scene change portion is detected using an AC coefficient. By determining the histogram of the edge image of the moving picture video stream with reference to the AC coefficient and referring to the distribution thereof, scene change false detection due to lighting change or the like can be reduced.

In the "scene switching detection apparatus" of Korean Patent Application No. 10-86096 (filed date: December 27, 2001), a histogram is obtained using a frame which is continuously restored as an input. Then, the scene transition can be detected by obtaining their cumulative histogram, making the pixel values corresponding to 20%, 40%, 60%, and 80% as a pixel value list and comparing them. In order to reduce the error in scene change detection, the model of brightness change of the image according to the lighting change is adopted, and the difference between the histogram and the threshold between the two frames is compared to investigate whether the lighting is affected. Processing is performed.

Z. Rasheed, Y. Sheikh and M. Shah's paper "On the Use of Computable Features for Film Classification" (IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 1, pp. 52-64, 2005 In order to categorize movies by genre, various calculable clues are proposed. Among these clues, the method using a scene change describes converting a color space into a Hue Saturation Value (HSV) and making them a histogram having 8, 4, and 4 bins, respectively. According to this method, a scene change can be detected by obtaining an intersection of histograms of consecutive frames and applying an anisotropic diffusion algorithm to the obtained results.

In addition, Xavier Naturel and Patrick Gros' article "On the Use of Computable Features for Film Classification" (Proceedings of the 2nd ACM SIGMOD International Workshop on Computer Vision meets Databases, June 2005) describes a system for retrieving broadcast video streams. In this case, a simple operation is used to provide such a search in real time. This paper describes searching for a scene change part by calculating a histogram of brightness of successively reconstructed frames and finding the difference between them.

However, the prior art described above does not specifically present a technical solution for searching and identifying digital video in real time in a large digital video database, that is, a technical solution for simple detection and fast detection of scene change. have.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and obtains a transition length using a transition part between frames of the digital video, and determines the transition length of the other digital video stored in the database in advance. It is an object of the present invention to provide a digital video recognition system that makes it possible to search and identify digital video in real time by comparing scene transition lengths.

Digital video recognition system using the scene change length of the present invention for achieving the above object,

A scene change detector which receives a digital video, extracts a difference between frames, and calculates a scene change length using the extracted difference;

A digital video database storing a plurality of digital videos and their corresponding scene transition lengths; And

And a digital video comparator that receives the scene change length calculated by the scene change detector and inquires a query in the form of a query to the digital video database, and compares the response to the query with the calculated scene change length. do.

In addition, the digital video recognition method using the scene change length of the present invention for achieving the above object,

(a) extracting a difference between frames of the input digital video by using a brightness change rate equal to or greater than an interframe threshold of the input digital video as a parameter;

(b) calculating a frame length corresponding to the scene change portion of the digital video as the scene change length by using the interframe difference extracted in the step (a);

(c) querying the database of pre-built digital video as a query the scene change length calculated in step (b);

(d) comparing the calculated scene change length with the registered scene change length corresponding to the identification number of the digital video output as a result of the query in step (c); And

(e) determining whether the calculated scene change length is the same as the scene change length of the digital video registered in the database within a threshold range, and determining whether to register the database of the currently input digital video. It features.

Hereinafter, a digital video recognition system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The digital video recognition system of the present invention is applicable to a large capacity multimedia service requiring real time processing such as searching and monitoring of digital video. 1 is a diagram illustrating a configuration of a digital video recognition system according to an embodiment of the present invention.

As shown in FIG. 1, the digital video recognition system according to an exemplary embodiment of the present invention includes a scene change detector 10, a digital video database 20, and a digital video comparator 30.

The scene change detector 10 receives digital video, extracts a difference between frames, and calculates a scene change length using the difference. The digital video database 20 stores a plurality of digital video and corresponding scene transition lengths, which will be described later in detail. The digital video comparator 30 receives the scene change length calculated by the scene change detector 10 and compares it with the scene change length stored in the digital video database 20, and outputs a comparison result of the same. do. From the comparison result, it is possible to determine whether there is the same thing as the digital video input into the digital video database 20.

FIG. 2 is a diagram illustrating the configuration of the scene change detector 10 of FIG. 1 in more detail. As shown in FIG. 2, the scene change detector 10 includes an interframe difference extractor 11 and a scene change detection filter 12. The interframe difference extractor 11 receives the digital video and extracts the interframe difference of the digital video, and the scene change detection filter 12 detects the scene change part by using the extracted inter frame difference and detects the scene change. Use the part to calculate the transition length.

3 is a diagram illustrating a frame of digital video with respect to a time axis, and FIG. 4 is a diagram illustrating a concept of a scene change used in a digital video recognition system according to an embodiment of the present invention.

As shown in FIG. 3, digital video is a collection of consecutive frames and has many temporal and spatial redundancies. In FIG. 3, the horizontal axis represents time t.

4 illustrates a scene that is a set of frames connected according to semantic content of digital video. In this case, each scene scene _{i- 1} and scene _i contain screen configurations and contents having different semantic contents from each other, and a scene change (shot) portion exists in a boundary region between these scenes. For example, in FIG. 4, the boundary regions of scenes scene _{i - 1} and scene _{i and} the boundary regions of scenes scene _i and scene _{i +1} become scene transition parts, and the positions on the frames of the scene transition parts are respectively location (shot). _{i -1} ) and location (shot _i ). At this time, as shown in FIG. 4, the scene change length indicates the frame distance between the scene change parts, and thus becomes length (scene _i ).

5 is a diagram illustrating an example of scene change in an actual frame. That is, the frame of the part where a scene change occurs in the video stream regarding "table tennis" is shown. As can be seen from Fig. 5, there is almost no change in the movement of the person in the frame of the same scene. Scene transitions are an attribute that represents the inherent characteristics of digital video and are now widely used in the field of digital video search and recognition technology.

However, it is ambiguous to define scenes containing special effects such as overlap, fade-in, fade-out, and cross-fade as transition parts. Thus, this type of scene is defined as a continuous scene change. Seamless scene transitions may occur depending on the intention of the creator of the digital video, but may also occur due to a change in the frame rate of the digital video itself.

Searching for all these seamless scene transitions increases processing capacity and complexity, so it is not appropriate to apply the concept to a digital video recognition system. As a result, an appropriate trade-off is required between the search accuracy of the seamless transition and the real-time processing. In view of this, the present inventors search for an appropriate level of the seamless transition, It is proposed a method to enable detection.

Detection of scene transitions in the present invention is based on the method of using reconstructed frames rather than using a specific video compression domain.

First, the operation of the interframe difference extractor 11 shown in FIG. 2 will be described in detail. In the inter-frame difference extractor 11, one of a method using a sum of absolute values of brightness differences between frames, a change rate of brightness over a frame threshold, a sum of differences between histograms of frames, and block-based correlation may be applied. .

The sum of the absolute values of the brightness differences between the frames has a large value in the scene change part, but has a disadvantage of being insensitive when a large value change occurs in a small area or a small value change occurs in a large area.

In general, as the scene changes, so does the distribution of brightness values. However, if the object moves within the frame, the change in distribution of brightness is small. Thus, the sum of the histogram differences between frames represents a small change in the same scene. The block-based interframe correlation is similar to the concept of finding a motion vector used in the video compression coding scheme. The assumption that the motion is very small in the same scene is applied. Therefore, block-based interframe correlation helps to reduce object movement and camera manipulation. However, the methods for calculating the sum of the histogram differences between frames and the block-based interframe correlation require a relatively large amount of computation. Therefore, the present invention uses the rate of change of brightness above the inter-frame threshold in consideration of the real-time aspect and detection performance.

If the brightness values of the positions x and y at time t are I _{x , y} (t), the brightness difference ΔI _{x, y} (t) between frames may be defined by the following equation (1).

Assuming that n () represents the number of elements in the set, and b is the number of bits used to express the brightness of the frame, the rate of change in brightness over the interframe threshold may be expressed as in Equation 2 below.

In Equation 2, 2 ^b-4 is an experimentally obtained threshold value. If b is 8, the brightness value is between 0 and 255, and the threshold value is 16. Since the rate of change of brightness above the threshold between frames has a non-linear relationship with the difference in brightness, it is easy to search for scene transitions.

The parameters for scene change detection described above, that is, the sum of the brightness change rate over the interframe threshold and other absolute values of the interframe brightness difference, the sum of the histogram differences between the frames, and the block-based correlation Are both small in the same scene, but large in the boundary area between the two scenes. The scene change portion may be detected by defining a portion having a value greater than or equal to the threshold value as the scene change portion in the calculated large value. However, these methods have images with intermediate frame rates (digital video), slow images taken by cameras with high shutter speeds, animations with fewer frames, strong lights (lightning, explosions, camera flashes, etc.). It has a high false detection rate in images including images, seamless scene transitions. Therefore, it is necessary to reduce the false detection rate by performing scene change detection filtering on the brightness change rate over the interframe threshold, that is, the parameter extracted above. In the present invention, as the input of the scene change detection filter 12 in Fig. 2, a brightness change rate (Rate _{| FD |} ) equal to or more than an inter-frame threshold that simplifies calculation and enables real-time processing is used. The obtained Rate _{| FD |} Is used as the input to the scene change detection filter 12 which is a combination of a local maximum value filter and a minimum value filter.

A filter for generating a maximum value and a minimum value as an output in a specific interval may be defined by Equations 3 and 4 below.

Next, using Equations 3 and 4, the operation of the scene change detection filter 12 may be represented by Equation 5 below.

FIG. 7 is a diagram illustrating a simple process of the scene change detection filter 12 of FIG. 2.

FIG. 7A illustrates an example of an input rate of the scene change detection filter 12 and illustrates a rate of change of brightness above an interframe threshold. Next, in order to calculate the scene change filtering result SCDF (t) represented by Equation 5, the MX ₄ , MN ₄ , MN ₂ , and MX ₂ filters are applied to the input shown in FIG. In order, the results of FIGS. 7B to 7E are calculated. Finally, the final filtering result SCDF (t) can be obtained by calculating the difference between the result of FIG. 7C and the result of (E). The output of the SCDF (t) is mostly close to zero and has a relatively high value in the scene change part, and the scene change part can be detected using a threshold value. The threshold value used here is an experimental value. If the input value is greater than 0.2, the scene is detected as a scene change part.

When the scene change part is detected, as shown in FIG. 4, the scene change length can be obtained using Equation 6 below. If we define location (x) to represent the frame position of x, the length of scene scene _i is the difference between the distances of scene transition shot _i and shot _{i- 1} . This is defined as the transition length for scene scene _i .

The scene change length thus obtained is input to the digital video comparator 30 shown in FIG. The digital video comparator 30 queries a digital video database 20 using a general-purpose database system to search for the existence of the same digital video as the currently input digital video. If the digital video comparator 30 accesses the digital video database 20 based on the evaluation criteria and evaluates the data, the real-time processing becomes difficult. Therefore, it is necessary to make full use of the performance of the database by using a general-purpose database (MySQL, Oracle, etc.) that is optimized for all operations (insert, delete, search, etc.) of the database.

First, in order to build a database in the digital video database 20 using the scene change length, when storing each digital video, the scene change length information is stored together with the digital video. If the cutaway length is regarded as an entity and stored, it is difficult to retrieve the cutaway length from the database. In addition, calculations must be made for each entity stored by the evaluation criteria, and this process must be performed in the digital video comparator 30, not in the database, which creates a large load on the overall system. Therefore, since the digital video database 20 simply stores information and provides it according to an external request, in order to solve this problem, the scene transition length is divided into N attributes and overlapped with each other. It was arranged as 8. In addition, the key value of the database is defined as a video ID. Building a digital video database like this requires a relatively large amount of storage space in the database, but contributes to the real-time processing of the system.

The digital video requested to be recognized by the present system is obtained by the scene change detector 10 in real time. The scene transition length thus obtained is queried by the digital video comparator 30 as a query with a bundle of N scene transition lengths, which is an entity unit stored in the digital video database 20, and in response, the digital video database 20 The video ID searched within the threshold per scene change length is output. Since the frames at the beginning and the end may be different from those registered in the digital video database 20 for editing or the like, the threshold should be set slightly larger than the middle at the beginning and the end of the digital video. If the digital video comparator 30 is equal to the scene change length registered in the digital video database 20 within the threshold range based on the returned video identification number (Video ID), the digital video comparator 30 transmits to the digital video database 20. It will be recognized as registered digital video. Through this series of processes, it is possible to determine whether or not the input digital video exists in the digital video database 20.

Next, a digital video recognition method using a scene change length according to an embodiment of the present invention will be described with reference to FIG. 9.

When the operation is started, the digital video is input to the scene change detector 10 to extract the difference between the frames (S11). As described above, the frame-to-frame difference of the input digital video is extracted by using a brightness change rate of at least a threshold value between frames as a parameter. Next, the frame length corresponding to the scene change portion is calculated as the scene change length using the difference between the frames extracted in step S11 (S12). The equation for calculating the scene change length is expressed by Equation 6 above.

On the other hand, in order to provide a large number of digital videos for retrieval, a large number of digital videos are built in advance as a database. This digital video database has a structure in which the key value is defined as an identification number of the corresponding digital video, and the scene change length is divided into N attributes and stored. In this way, the database of the pre-built digital video is queried as a query for the scene change length calculated in step S12 (S13). In response, the database outputs an identification number of the digital video searched within the threshold for each scene transition length, and compares the scene transition length registered for this identification number with the calculated scene transition length (S14).

From the comparison result of step S14, if the calculated scene change length is equal to the scene change length of the digital video stored in the database within the threshold range, it is determined that the currently input digital video is already registered in the database. (S15). Through this series of processes, it is possible to determine in real time whether the input digital video exists in the database.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

As described above, according to the present invention, the scene transition length is obtained by using the scene transition portion between the frames of the digital video, and the scene transition length and the obtained scene transition length of other digital video stored in the digital video database in advance are obtained. By comparison, it is possible to provide a digital video real time recognition system that enables searching and identifying digital video in real time.

In the present invention, unlike the prior art, in order to enable recognition and retrieval of digital video in real time using a scene change length, an interframe difference extractor simply calculates a rate of change of brightness above an interframe threshold and maximizes a specific interval. A scene transition detection filter using a combination of a value filter and a minimum value filter is used to retrieve scene transitions, and a method of storing N scene transition lengths in a digital video database and searching within a threshold rather than a specific evaluation is presented. . In addition, three consecutive frames were used as inputs to the interframe difference extractor to provide some performance against seamless scene transitions.

Claims (6)

A scene change detector which receives a digital video, extracts a difference between frames, and calculates a scene change length using the extracted difference; A digital video database storing a plurality of digital videos and their corresponding scene transition lengths; And And a digital video comparator that receives the scene change length calculated by the scene change detector and inquires a query in the form of a query to the digital video database, and compares the response to the query with the calculated scene change length. Digital video recognition system. The method of claim 1, The scene change detector, An interframe difference extractor which receives the digital video and detects a scene change portion of the digital video by a predetermined parameter; And And a scene change detection filter for performing a scene change detection filtering on the parameter and calculating a scene change length using the parameter. The method of claim 2, And wherein said parameter is a rate of change of brightness above an interframe threshold. The method of claim 1, The digital video database is constructed such that when a plurality of digital videos and corresponding scene transition lengths are stored in the digital video database, a continuous bundle of N scene transition lengths is stored as one entity. Recognition system. (a) extracting a difference between frames of the input digital video by using a brightness change rate equal to or greater than an interframe threshold of the input digital video as a parameter; (b) calculating a frame length corresponding to the scene change portion of the digital video as the scene change length by using the interframe difference extracted in the step (a); (c) querying the database of pre-built digital video as a query the scene change length calculated in step (b); (d) comparing the calculated scene change length with the registered scene change length corresponding to the identification number of the digital video output as a result of the query in step (c); And (e) determining whether the calculated scene change length is the same as the scene change length of the digital video registered in the database within a threshold range, and determining whether to register the database of the currently input digital video. Digital video recognition method. The method of claim 5, And the database is constructed in advance by defining identification numbers of a plurality of digital videos as key values and storing N scene transition lengths as attributes for each identification number.

Images