KR100683501B1 - An image extraction device of anchor frame in the news video using neural network and method thereof - Google Patents

Abstract

In the anchor screen extraction apparatus for news video using neural network technique, news videos are classified into anchor screens having structural characteristics of anchor screens and non anchor screens. When the news image data is input from the outside, the scene change is detected, and the anchor screen is detected from the input pattern by comparing the input pattern corresponding to the scene change detected based on the neural network technique with a plurality of classification patterns corresponding to the anchor screen. do. The classification pattern is modeled into a plurality of models according to anchor configuration variables including at least one of the number of anchors, the presence or absence of a news icon, and the location of a news icon.

Therefore, the neural network technique can accurately and efficiently extract the anchor screen from the news video.

Neural network, news video, anchor screen, cutaway

Description

An image extraction device of anchor frame in the news video using neural network and method

1 is a diagram showing the spatial structure of a news video according to an embodiment of the present invention.

2 is a structural diagram of an anchor screen extracting apparatus according to an embodiment of the present invention.

3 is a diagram illustrating a model of the spatial configuration of an anchor screen according to an embodiment of the present invention into four neural network input models.

5A and 5B are diagrams for modeling a spatial configuration of a non-anchor screen as an error input model of a neural network to extract an anchor screen according to an exemplary embodiment of the present invention.

6 is a flowchart illustrating an anchor screen extracting method according to an embodiment of the present invention.

7 is a diagram illustrating a scene change extraction process according to an embodiment of the present invention.

8 is an exemplary diagram illustrating a result of detecting a scene change according to the process illustrated in FIG. 7.

9 is a structural diagram of an ADALINE network of a new network model according to an embodiment of the present invention.

10 is a diagram illustrating a result of extracting an anchor screen using a neural network technique according to an exemplary embodiment of the present invention.

11 is a screen structure diagram of a program for verifying validity according to an embodiment of the present invention.

12 is a structural example of a digital news video archive system to which an anchor screen extracting apparatus according to an embodiment of the present invention can be applied.

The present invention relates to an image extracting apparatus and a method thereof, and more particularly, to an anchor screen extracting apparatus and method of a news video for implementing a digital news video archive system.

In the case of video, among the various knowledge assets, analog production and storage / distribution are common, and digitalization is rapidly progressing in recent years. Examples include digitalization of broadcasts, replacement of VHS tapes by DVD, the spread of digital camcorders, and digitalization of the security surveillance market.

Digital images can be stored in magnetic disks, magnetic tapes, optical disks, etc., and for easy utilization of digital images stored in various storage media, easy and quick search for desired scenes is required. In addition, with the shift to digital broadcasting, the search demand for digital images is inevitably generated. As digital broadcasting becomes more common in the future and intellectual assets in the form of digital images and images from other corporations, agencies, and government agencies in addition to the broadcasting industry are increasing rapidly, efficient storage, management, and retrieval solutions for such digital images are essential. It is becoming.

In response to these demands, digital news video archive systems have been developed to enable easy database and retrieval of various and digitized news information.

In the digital news video archive system, news video must be stored and searched on a per event basis. To do this, the anchor screen, which is the beginning of the event, must be detected. The anchorperson frame is the section in which the anchor introduces the event in the news video. The anchor screen is not only important for structurally dividing and storing the news because it means the beginning of an event, but also includes news icons and subtitles that compress and represent the event to be reported next. Extraction of anchor screens should be an essential prerequisite. In addition, in order to extract a main word or index word and use it as a future search word, extraction of an anchor screen has an important meaning.

In order to extract the anchor screen, it is necessary to analyze the news video and segment the news video based on the content. The most popular method is to detect where the content change occurs, that is, a scene change, and detect the boundary. How to split news video into scene units. Since the scene transitions occur in a large amount of change in brightness or color in a continuous screen, the change amount of the screen such as the difference between the brightness or color value of the current frame and the previous frame, or the difference in the brightness or color histogram is expressed as a single figure. By calculating in units that can be expressed, the point where the change amount is large can be detected as a scene change.

Conventionally, a method of detecting an anchor screen by using scene change detection is to consider a screen having the same color as a specific background color of a screen in which an anchor appears during a scene change as an anchor screen, or a screen in which a face color appears by calculating an anchor face color. May be regarded as an anchor screen or a method for detecting an anchor screen by recognizing an anchor face. However, these methods generate a lot of errors in detecting a screen with many colors similar to the background or face color.

Another method is to consider a screen with a small amount of movement as an anchor screen under the assumption that the anchor movement is small. However, it is difficult to accurately extract the anchor screen using only the movement amount.

Therefore, the technical problem to be achieved by the present invention is to solve the conventional problems, and to accurately extract the anchor screen from the news video using a neural network technique.

In particular, the technical problem to be achieved by the present invention is to accurately extract the anchor screen in consideration of the structural characteristics of the anchor screen in the news video.

In order to achieve the above technical problem, the anchor screen extraction apparatus according to an aspect of the present invention, in the device for extracting the anchor screen from the news image data corresponding to the news video, the image input unit for receiving the news image data applied from the outside; A scene change detector for detecting a scene change from the news image data and outputting an input pattern corresponding to the detected scene change; And a classification pattern storage module storing a plurality of classification patterns corresponding to the anchor screen, and a pattern comparison module configured to detect the anchor screen from the input pattern by comparing the input pattern with the stored classification pattern based on a neural network technique. Includes an anchor screen detector comprising a.

In the present invention having such a feature, in order to extract the anchor screen using the neural network technique, the classification pattern may include an anchor screen configuration variable including at least one of the number of anchors, the presence or absence of a news icon, and the location of a news icon. It consists of a number of classification patterns modeled on the basis. The classification pattern may be classified into one of an anchor screen and a non-anchor screen, and the anchor screen may be modeled with at least four models, and the non-anchor screen may be modeled with at least eight models.

In addition, the anchor screen extraction method according to another aspect of the present invention, in the anchor screen extraction method of the device for extracting the anchor screen from the news image data corresponding to the news video, the news image data from the outside--the news image data is I When the input is made of a motion picture expert group (MPEG) bit stream including a frame, the device detects a scene change only on a screen corresponding to an I frame from the news image data, and corresponds to the detected scene change. Outputting an input pattern; And detecting the anchor screen from the input pattern by comparing the input pattern with a plurality of classification patterns corresponding to the anchor screen based on a neural network technique, wherein the classification pattern includes a number of anchors and a news icon. It consists of a plurality of classification patterns modeled based on the anchor screen configuration variable including at least one of the presence, and the location of the news icon.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

The news is regularly organized in terms of the time the anchor speaks and the screen described by the reporter, a non-anchor section that follows, with a typical news video shown in FIG. As shown in FIG. 1, the news video includes an anchor section, a subtitle, a news icon, and the like. In order to efficiently search for such news videos, news must be structurally divided and stored by event. For this purpose, the anchor section must be accurately detected in the news video.

Therefore, the embodiment of the present invention extracts an anchor section, that is, an anchor screen, which is an essential task for implementing a digital news video archive system belonging to a wide range of digital video archive systems, using neural network techniques.

FIG. 2 is a diagram illustrating a structure of an anchor screen extracting apparatus (hereinafter, referred to as an "anchor screen extracting apparatus" for convenience of description) of a news video using a neural network technique according to an exemplary embodiment of the present invention. 3 illustrates a detailed structure of the scene change detection unit and the anchor screen extracting unit shown in FIG. 2.

As shown in FIG. 2, an anchor screen extracting apparatus according to an embodiment of the present invention may include an image input unit 10, a buffer 20, a scene change detector 30, an anchor screen extractor 40, and an encoder ( 50, and an output unit 60.

The image input unit 10 receives news image data applied from the outside and buffers the news image data in the buffer 20, where the news image data is image processed data so as to be photographed by an camera or the like and electrically processed. For example, news image data is data processed in the form of a bit stream encoded according to the Moving Picture Experts Group (MPEG) standard.

The scene change detection unit 30 detects a scene change from the news image data, and divides the news image data into scene units based on the scene change.

As illustrated in FIG. 3, the scene change detection unit 30 according to an embodiment of the present invention performs a binarization on a DC image generation module 31 and a DC image for obtaining a DC image for scene change detection from news image data. Binarization module 32 includes a normalization module 33 for normalizing the binarized DC image.

The scene change detection unit 30 having such a structure detects the scene change using not only the brightness component of the image data but also the histogram accumulation difference caused by the combination of the color components. The detected scene change is normalized and output as an input pattern for anchor screen detection.

Meanwhile, the anchor screen extractor 40 extracts the anchor screen from the detected scene change, and the encoder 50 encodes the extracted anchor screen and outputs or stores the encoded anchor screen through the output unit 60.

In order to extract the anchor screen using the neural network technique, the anchor screen extractor 40 according to an embodiment of the present invention stores data corresponding to a classification pattern according to the neural network technique for extracting the anchor screen as shown in FIG. 3. A pattern for extracting the anchor screen by comparing the classification pattern stored in the pattern storage module 41 and the stored classification pattern with the input pattern output by the scene change extraction unit 30 to determine which classification pattern the input pattern corresponds to Comparison module 42.

A neural network is a technique widely used in the field of pattern classification, and classifies input patterns by assigning unknown input patterns to predetermined predetermined pattern classes most similar to the input patterns. In an embodiment of the present invention, the anchor screen is modeled as a plurality of anchor models in order to detect the anchor screen using a neural network.

Next, a method of generating a plurality of classification patterns according to modeling of an anchor screen according to an embodiment of the present invention will be described.

In order to extract the anchor screen, first, we analyze the spatial structure characteristics of the screen in which the anchor exists in the news image data, set prior knowledge of the anchor's face area, and then detect the anchor's face and accurately recognize it as an anchor. You must set the screen as a key frame.

In detail, the anchor screen may use spatial features such that the face of the anchor appears with a certain size, the movement of the face is very fine, and the background that appears throughout the video is similar. Therefore, this spatial feature can be modeled as a pattern class of neural network.

The anchor screen may or may not have a news icon that represents a compressed news report. However, you can see that the anchor's face area is always constant, regardless of the presence of a news icon. Therefore, in order to set all these cases as neural network pattern class, four models of anchor screens were modeled according to the spatial configuration of anchor screen based on the number of anchors, the presence or absence of news icons, and the location of news icons. Hereinafter, for convenience of description, the number of anchors, the presence or absence of a news icon, and the location of the news icon representing the spatial configuration of the anchor screen are referred to as "anchor screen configuration variables".

4 is an exemplary view illustrating modeling an anchor screen into four models according to an anchor screen configuration variable according to an exemplary embodiment of the present invention.

As shown in the accompanying FIG. 4, the anchor screen is a single anchor, according to the anchor screen configuration variable, the first model M1 without a news icon, a single anchor, and a news icon in the upper right (or lower) part of the anchor. The second model M2, a single anchor and a third model M3 having a news icon in the upper left (or lower) part of the anchor, and a fourth model M4 in which a plurality of anchors appear.

Meanwhile, a non-anchor screen, that is, a screen without an anchor, may be modeled according to the distribution of brightness of the screen, and FIGS. 5A and 5B illustrate a non-anchor screen according to an exemplary embodiment of the present invention. As shown in FIGS. 5A and 5B, the non-anchor screen may be modeled into eight error models according to the distribution of screen brightness.

As described above, in the embodiment of the present invention, the pattern class is largely set to two classes, the anchor screen and the non-anchor screen, and each class is configured into four and eight subclasses to train the neural network. . Then, by searching for a pattern class most similar to the input pattern according to the news image data input later, it is determined whether the input pattern is an anchor screen or a non-anchor screen to extract an anchor screen.

Next, the operation of the anchor screen extracting apparatus according to an embodiment of the present invention will be described in more detail based on the above-described structure.

6 is a flowchart illustrating an anchor screen extracting method according to an embodiment of the present invention.

When news image data transmitted from an external broadcast system or taken by a camera or the like and stored in a separate recording medium (eg, a video tape or a DVD) is input through the image input unit 10, the news image data is buffered 20. Is buffered (S100). Such news video data is data composed of an MPEG bit stream.

In general, MPEG uses prediction and interpolation to compensate for motion by using a similarity between adjacent frames, rather than compressing all frames into individual purified images. However, due to various reasons such as VCR-like control such as random access, MPEG frames should regularly insert frames that can be recovered by only their own information.

Therefore, MPEG video data is composed of a frame I (Intra-coded) frame compressed into a clean image, a P (Predictive-coded) frame with only prediction, and a B (Bidirectional-coded) frame with bidirectional interpolation. The I-frame here can come anywhere in the data stream and is used for random access of the data and encoded without reference to other images. The I-frame is divided into 8x8 blocks designated in the macroblock, and then encoded in block units. The set of consecutive pictures starting with an I-frame is called a GOP (Group Of Picture).

Since at least one I frame is encoded once in the GOP, the scene change detection unit 30 according to an embodiment of the present invention detects a scene change only for a screen corresponding to the I frame from the news image data of the MPGE. That is, partial decoding is performed to decode only the bit stream having necessary information without completely decoding the MPEG bit stream, thereby reducing the time required for decoding. 7 illustrates an example of scene change detection according to an embodiment of the present invention.

Specifically, with reference to FIG. 7, the DC image generating module 31 of the scene change detection unit 30 targets only I frames in the news image data, and pixel values of predetermined blocks (8 × 8 blocks) in the I frames. A DC image (40 × 30) is obtained by partially decoding using only the DC coefficients representing (S110 to S120).

Next, the binarization module 32 of the scene change detection unit 30 performs a binarization process on the obtained DC image (S130). In particular, the average value of the brightness value of the DC image obtained above is used as a threshold value, and binarization is performed by comparing the brightness value and the threshold value of the pixels constituting the DC image. For example, if the brightness value of a predetermined pixel is greater than the threshold, a value corresponding to white (eg, +1) is given. If the brightness value of the predetermined pixel is less than the threshold, a value corresponding to black (eg, -1) is given. Grant. The threshold is not limited to the average value and may be set differently according to a given environment.

The normalization module 33 normalizes the binarized image as described above to generate a normalization pattern, that is, an input pattern as illustrated in FIG. 7 (S140). In this example, the binarized image is normalized to 10 × 10, but is not limited thereto. An example of scene change detected from news image data input to FIG. 8 is shown.

As described above, when a scene change is detected, an input pattern corresponding to the detected scene change is classified into two classes, an anchor screen and a non-anchor screen, using neural network techniques.

In the embodiment of the present invention, the input pattern is classified into two classes using the ADALINE neural network, but is not necessarily limited thereto. Figure 9 shows the structure of the ADALINE neural network used in the embodiment of the present invention. In general, the ADALINE neural network includes an input layer and an output layer, as shown in FIG. 9, and the input layer includes an input node and a bias node for each input to the network. And the output layer contains only the ADALINE nodes that produce the output for the network. Each node in the input layer is connected to the ADALINE node by one link.

Since the input pattern input to this ADALINE neural network should have components normalized to a predetermined range (typically -1.0 to 1.0 is used), the image data for the detected scene transition is normalized as described above. This normalization prevents one component of the input pattern from prevailing and disturbing the operation of the network.

The outputs required by the ADALINE neural network must always be one of two values for the two classification classes. Therefore, in an embodiment of the present invention, the output value for the input pattern should be either an anchor screen or a non-anchor screen. Here, the number of input vectors of the ADALINE neural network is set to 100. If the pixel value corresponds to 0 in the normalized input pattern, the ADALINE neural network has an input pattern of -1 and a pixel value of +1 corresponding to 255.

Based on the neural network network as described above, the pattern comparison module 41 of the anchor screen extractor 40 classifies the input pattern provided from the scene change extractor 30 and the classification stored in the storage module 42. The classification classes for the input patterns are determined by comparing the patterns (S150 to S160). For example, based on the ADALINE neural network, the product of the input pattern input through the input node and the corresponding link value is summed, and then the threshold function is applied to the weighted sum so that the output is the value of one of the two classification classes. To force the The threshold function converts a positive sum to a value associated with classification class A (anchor picture), and any negative sum to a value associated with classification class B (non-anchor picture). .

This ADALINE neural network learns by modifying weight values associated with links during the training process. Training usually adjusts link values with a learning rule known as a delta rule.

The delta rule uses the error produced by ADALINE to correct the link values (sorting one pattern into the wrong category) so that the correct answer is produced the next time the input pattern is presented.

The weighting value calculated according to the delta rule according to an embodiment of the present invention may be calculated according to Equation 1 below.

는 가중값을 나타내며,

는 학습율,

는 에러 그리고

는 입력값(입력패턴의 소정 화소값)을 나타낸다. here

Represents the weighting value,

Is the learning rate,

Is an error and

Indicates an input value (a predetermined pixel value of the input pattern).

10 illustrates a screen structure of a program for validating an anchor screen extracting apparatus according to an embodiment of the present invention, which operates as described above, and FIG. 11 uses a neural network technique according to an embodiment of the present invention. The extracted anchor screen result is illustrated. 10 and 11, it can be seen that the anchor screen is accurately detected in the news video.

The anchor screen extracting apparatus according to this embodiment may be applied to and used in a digital news video archive system that may be implemented in a structure as shown in FIG. 12.

Although the present invention has been described with reference to the most practical and preferred embodiments, the present invention is not limited to the above disclosed embodiments, but also includes various modifications and equivalents within the scope of the following claims.

As described above, according to an embodiment of the present invention, in order to implement a digital news video archive system for efficient storage and retrieval of news video, an anchor screen may be accurately detected using a neural network technique.

The present invention can be applied to the field of image search and editing, and is also an essential technology for the implementation of digital news archives, which contributes to the early construction of digital broadcasting stations, as well as other digital image archives such as digital newspapers and movies. It is expected to be able.

Claims (8)

In the device for extracting the anchor screen from the news image data corresponding to the news video, An image input unit which receives the news image data applied from the outside; A scene change detector for detecting a scene change from the news image data and outputting an input pattern corresponding to the detected scene change; And A classification pattern storage module for storing a plurality of classification patterns corresponding to an anchor screen, and a pattern comparison module for comparing the input pattern with the stored classification pattern based on a neural network technique and detecting an anchor screen from the input pattern. Anchor screen detection unit including Including, The scene change detection unit, A DC image generation module for obtaining a DC image for scene change detection from the news image data; A binarization module for performing binarization on the DC image; And Normalization module for outputting an input pattern by normalizing the binarized DC image Anchor screen extracting device comprising a. The method of claim 1 And the classification pattern comprises a plurality of classification patterns modeled based on an anchor screen configuration variable including at least one of the number of anchors, the presence or absence of a news icon, and the location of a news icon. The method according to claim 1 or 2 The classification pattern is classified into one of an anchor screen and a non-anchor screen, and the classification pattern corresponding to the anchor screen is a single anchor, a first model without a news icon, a single anchor, and a news icon on the right and above or below the anchor. And a second model having a single anchor, a third model having a news icon on the left and top or bottom portions of the anchor, and a fourth model showing a plurality of anchors. The method of claim 1 The binarization module of the scene change detection unit uses the average value of the brightness values of the DC image obtained as the threshold value, and performs the binarization by comparing the brightness value and the threshold value of each pixel constituting the DC image. Extraction device. The method according to claim 1 or 4 And the news image data comprises an MPEG bit stream including an I-frame, and the scene change detection unit detects a scene change only for a screen corresponding to an I frame from the news image data of the MPEG. The method of claim 3, The classification pattern corresponding to the anchor screen is modeled with one of the first to fourth models, and the classification pattern corresponding to the non-anchor screen is modeled with one of a plurality of preset error models. In the anchor screen extraction method of the device for extracting the anchor screen from the news image data corresponding to the news video, When the news image data from outside, the news image data consists of an MPEG bit stream including an I-frame, is input, the device obtains a DC image for the screen corresponding to the I frame from the news image data, Binarizing and then normalizing the obtained DC image to output an input pattern corresponding to a scene change; And Detecting the anchor screen from the input pattern by comparing the input pattern with a plurality of classification patterns corresponding to the anchor screen based on the neural network technique; Including; And the classification pattern comprises a plurality of classification patterns modeled based on an anchor screen configuration variable including at least one of the number of anchors, the presence or absence of a news icon, and the location of a news icon. The method of claim 7, The classification pattern is a single anchor, a first model without a news icon, a single anchor, a second model with a news icon on the right and top or bottom of the anchor, modeled according to the anchor screen configuration variables, a single anchor and anchor And a classification pattern corresponding to any one of a third model having a news icon on a left side and an upper part or a lower part of the fourth model, and a fourth model in which a plurality of anchors appear.

Images