KR20040102962A - Apparatus for generating highlight stream in PVR and method for the same - Google Patents

Abstract

PURPOSE: A highlight stream producing apparatus of a PVR(Personal Video Recorder) and a method thereof are provided to divide a scene of a program stored to a recording medium by using a scene shift detecting algorithm and to extract main scenes under the quantitative analysis of audio stream of each scene period. CONSTITUTION: A scene shift discriminating member(102) analyzes video stream of a broadcasting program recorded to a storage medium and separates the program into plural periods according to the scene shift. A period volume estimating member(104) estimates a degree of importance by analyzing a voice signal of each scene period. A highlight generating member(105) selects scene periods needed to produce the highlight stream on the basis of the degree of importance and stores information on positions of the selected scene periods to the storage medium.

Description

Apparatus for generating highlight stream in PVR and method for the same}

The present invention provides an apparatus for automatically generating a highlight stream for a broadcast program recorded in a personal video recorder (PVR) equipped with a hard disk (HDD) in a digital TV and providing the same to a user. It is about a method.

The PVR provides a function of storing a digital broadcast on a hard disk by mounting a hard disk on a digital TV. The PVR provides not only the recording function as a main function but also a time-shift, instant replay, and other trick play functions, allowing the user to enjoy a variety of digital TVs. to provide. Moreover, the advantages of storing HD-quality high-definition digital broadcasting at any time on the hard disk and various additional service functions are very attractive to users. However, the viewing time of the user is limited compared to the vast amount of program content. In an environment with vast amounts of video data stored, users must spend a lot of time watching all their recorded programs. This demands the ability to quickly and easily access or retrieve the desired content within a limited time, and to solve this problem, an effective way to implicitly summarize the video data is to provide an overview of the program in a short time. need.

Among these methods, a function called highlight may provide a user with implicit information about a recorded program by representing the entire video stream using only a part of the entire data. Highlight video is the recording or playback of specific sections of the entire video and audio stream separately. When a user wants to select and view one of several programs stored on the digital video recorder for a limited time, the user can only highlight the highlight stream of each program. By playing back, it is possible to save time for searching for the contents of the desired program. Especially in programs such as sports broadcasting, highlight video has the advantage of compressing the entire game scene into one summarized video. That is, only a main scene such as a scoring scene can be selectively arranged to provide a single summarized video to the user.

However, the conventional methods lack the automatic method for generating the highlight stream for the video and audio data, and also lacks the method for separately extracting the portions to generate the highlight from the video and audio stream. In other words, highlights need to be extracted separately to represent the contents of the program. Therefore, selecting a section to generate highlights is a very difficult task. Therefore, conventionally, a program provider uses a method of separately creating a highlight stream. As a result, the user had to rely entirely on the content producer to get the highlight stream for the program. This results in a problem that, in the case of a program that does not provide a highlight, when a user records a plurality of programs at the same time, there is no way to learn information about each program within a limited time.

Therefore, to solve the above problem, we need a way to automatically create these highlights. In order to generate the highlights in an automatic way, the video must be divided into scene concepts that are clearly distinguished in terms of the given video data, and the quantitative measure of importance for each segmented section for the selection of the main scenes. How is it needed? However, there is a problem in the conventional method is insufficient function to solve this problem.

In addition, the conventional highlight generation method does not provide an effective method of storing the separately extracted highlight stream. That is, the highlight is configured by separately storing some stream data constituting the highlight in the recorded program. However, this method extracts and re-stores some data already stored on the hard disk again, resulting in a waste of storage space by storing redundant data.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to perform scene segmentation on a program stored in a storage medium using a scene change detection algorithm, and then based on quantitative analysis of an audio stream for each scene section. The present invention provides an apparatus and method for generating a highlight stream by extracting a main scene.

Another object of the present invention is to provide an apparatus and method for generating a highlight in a PVR which minimizes the size of data additionally generated by generating a highlight stream for a recorded program.

1 is a block diagram showing an embodiment of a highlight stream generating apparatus according to the present invention;

2 is a flowchart illustrating an embodiment of a process of generating and storing a highlight stream according to the present invention.

FIG. 3A is a diagram illustrating an embodiment of a section selected as a highlight in each scene section; FIG.

3B is a diagram illustrating an embodiment of storing a pointer set HS of a highlight section selected as in FIG. 3A.

4 is a flowchart illustrating an exemplary embodiment of a trick play for reproducing a highlight using the pointer information of FIG. 3.

Explanation of symbols for main parts of the drawings

101: image comparison unit 102: scene change determination unit

103: audio decoding unit 104: interval volume estimation unit

105: highlight generator 106: HDD interface

In the PVR according to the present invention for achieving the above object, a scene change determination unit for analyzing the video stream of the broadcast program recorded on the storage medium to divide the program into a plurality of sections according to the scene change And a section volume estimator for estimating the importance of each scene section by analyzing a voice signal of each scene section, selecting scene sections having high importance as scene sections for generating a highlight stream, and location information of the selected scene sections. It characterized in that it comprises a highlight generating unit for storing in the storage medium.

And a video comparator configured to receive a video stream of a recorded broadcast program from the storage medium and compare a quantitative difference between two video frame signals and output a frame error value to the scene change determiner. A frame error value is compared with a predetermined threshold value to determine whether to switch scenes of the video stream, and based on the scene change, the video stream is divided into a plurality of scene sections and outputs information of each scene section. do.

The section volume estimating unit receives the decoded audio signal, calculates an average volume value of each scene section, and uses the same to measure the importance of each scene section.

The location information of the scene section selected as the highlight in the highlight generator is pointer information of the start frame and the end frame of the scene section selected as the section constituting the highlight.

Highlight stream generation method in the PVR according to the present invention,

(a) analyzing a video stream of a broadcast program recorded on a storage medium and dividing the program into several sections according to scene change;

(b) estimating importance of each scene section by analyzing a voice signal for each scene section; And

(c) selecting scene sections of high importance as scene sections required for generating a highlight stream, and storing location information of the selected scene sections on the storage medium.

In the step (b), the average volume value of each scene section is calculated and output as the importance of each scene section.

Other objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings.

Hereinafter, with reference to the accompanying drawings illustrating the configuration and operation of the embodiment of the present invention, the configuration and operation of the present invention shown in the drawings and described by it will be described as at least one embodiment, By the technical spirit of the present invention described above and its core configuration and operation is not limited.

According to the present invention, a scene change detection is performed on MPEG video streams stored in a storage medium such as an HDD, divided into several sections, and then the sections to be included in the highlight stream are determined based on the audio signal analysis of each section. have.

FIG. 1 is a block diagram illustrating an exemplary embodiment of a highlight stream generating apparatus according to the present invention, which receives a video signal of a recorded program from a storage medium and compares quantitative differences between two frame signals to output a frame error value. The image comparator 101 compares the frame error value output from the image comparator 101 with a preset threshold value to determine whether to switch scenes, and converts the stored video stream into a plurality of scene sections based on the scene change. After the division, the scene change determination unit 102 outputting information of each scene section, the audio decoding unit 103 which receives the audio signal of the recorded program from the storage medium, extracts the volume information of the audio, and the volume information of the audio. Section volume estimation for calculating and outputting average volume value for each scene section based on And a highlight generator 105 for selecting a scene section necessary for generating a highlight stream from the volume information of each scene section and outputting information of the selected scene section.

Here, the information of the scene section selected as the section constituting the highlight is pointer information in the stream for the start point and the end point of each scene section. In the embodiment of FIG. 1, the highlight section 105 is output from the highlight generator 105. The highlight pointer information is stored in the HDD via the HDD interface 106.

As another example of storing the highlight pointer information, a method of transmitting to the memory in the system using a memory interface may be used.

2 is a flowchart illustrating an example of a method of generating a highlight stream according to the present invention, in which the scene change determiner 102 detects a scene change with respect to a video stream input through the image comparator 101 and detects the scene change. Based on the scene change, the video stream is divided into a plurality of scene sections (step 201).

Here, the scene change detection is a method of calculating a frame error for consecutive I pictures of an MPEG video stream of a stored program, or calculating a histogram error by extracting a histogram for Y / Cb / Cr components of each I picture. Can be used. In addition, a method of comparing a histogram with respect to a macroblock type with respect to a P picture or a B picture may be used to more accurately detect a scene change occurrence point.

At this time, by performing scene change detection, the position of the frame in which the scene change occurs in the video stream can be known. The frame position at which the scene change occurs is used as the highlight section pointer information to be described later. In this case, video frames included in one point where a scene change occurs from a point where the next scene change occurs constitute one scene section.

In operation 201, when a plurality of sections are divided according to the scene change, the audio decoding unit 103 decodes an audio signal corresponding to each section for each section, and then uses the decoded audio signal to determine the section volume estimating unit ( In 104, the average volume of the section is calculated (step 202). At this time, the volume information for the i-th section is denoted by A (i).

In the present invention, the average volume value is used as an example for measuring the importance of each section. Particularly in the case of sports relay broadcasting, when a major scoring scene or a decisive situation occurs, the loudness of the shout of the crowd or the voice of the announcer is increased, thereby increasing the volume of the audio signal of the corresponding section. Therefore, it may be determined that a deterministic event occurs in a section with a large volume. In this invention, the volume value is used as a criterion for measuring the importance of each section.

Meanwhile, when the average volume of each section is obtained for each section in step 202, the highlight generator 105 determines the section HS to be included in the highlight stream based on the average volume value of each section (step 203). In this case, the highlight generator 105 compares a volume value of each section with a preset threshold in order to determine a section to be included in the highlight stream. In this case, the threshold is set in an adaptive manner according to the audio characteristics of the recorded stream. Each scene section is defined by the position of the start frame and the position of the end frame. Therefore, these frame positions are defined as the start pointer and the end pointer of each section, and the sections necessary for reproducing the highlight stream can be expressed using these pointer information. In this case, the HS shown in FIG. 2 represents a set composed of pointer information of a section constituting the highlight stream.

When the section to be included in the highlight stream is determined through the above process, the set HS of the highlight section pointer is stored in a hard disk or other memory (step 204).

Meanwhile, as another embodiment of the present invention, step 201 and step 202 of FIG. 2 may be combined and executed. That is, the scene change detection operation for the video signal and the volume accumulation operation for the audio signal corresponding to each video frame may be performed in parallel. In this process, the sum of the video frame number and the volume value for the current scene section is calculated.When a scene transition occurs, the sum of the volume values calculated so far divided by the number of video frames is determined as the average volume value. Store in internal buffer or external memory. For example, it is assumed that a scene change occurs when 20 video frames are counted in the current scene section, and the volume value accumulated until 20 video frames are counted is 40. Then, the average volume value is 2 in the current scene section. Then, after storing the average volume value, a counter circuit (not shown) for counting the sum of the volume values and the number of video frames may perform a reset operation to calculate a new average volume value for the next section.

3A is a diagram illustrating an embodiment of a process of generating a highlight pointer set HS in the apparatus and method for generating a highlight stream according to the present invention. Referring to FIG. 3A, a program recorded by the scene change detection process described above is divided into five scene sections. In this case, the total length of the recorded stream is 3600 frames, and the frame number corresponding to the start of each section is 1, 600, 1600, 2000, 2800. That is, the interval 2 is composed of data corresponding to 600th to 1599th frames of the stream. In this case, the highlight generator 105 compares the average volume value for each section calculated by the section volume estimator 104 to determine section 3 and section 5 as sections to be included in the highlight stream.

According to an embodiment of the present invention, the set HS including the pointer of the section to configure the highlight stream is shown in the table shown in FIG. 3B. At this time, in the embodiment of the present invention, the highlight pointer set HS is composed of a frame number in the stream indicating the number of sections included in the highlight and the start and end points of each section. Thus, the number 2 in the first row means that the highlight consists of two scene sections, and the values in the second and third rows (ie, `` 1600,1900 '' and `` 2800,3600 '') represent Represents a start and end pointer.

In this case, the method of configuring the highlight pointer set HS may be variously modified in a range capable of effectively expressing a highlight section, such as a form consisting of a set of header information and a pointer, in addition to the embodiment of FIG. 3.

4 is a flowchart illustrating an example of a process of reproducing a highlight using a highlight pointer set HS in the method of generating a highlight stream according to the present invention. It is assumed that the highlight pointer set is given by the above-described embodiment of FIG. 3. do. That is, in the highlight playback process, the highlight stream is composed of sections 3 and 5, and when the user selects the highlight playback mode, the system plays sections 3 and 5.

To this end, first, the controller moves to the 1600th frame corresponding to the start pointer of the section 3 and performs playback on the section (step 401). That is, the 15th to 1599th frames are skipped without reproducing. When playback to the 1999th frame, which is the last frame of the interval 3, is finished (step 402), the process moves to the 2800th frame corresponding to the first frame of the interval 5 (step 403). Then, playback of the highlight stream is completed by reproducing from the 2800 th frame to the 3600 th frame (step 404). Here, the audio corresponding to the interval 3 and the interval 5 is also reproduced when the interval 3 and the interval 5 are reproduced.

At this time, in order to accurately move to the frame number indicated by the pointer of each section, when recording a program, an index file indicating a physical location on the hard disk where each frame of the stream is stored is separately managed. Assume that

As a result, the present invention stores only the pointer information indicating the position in the stream which is already stored without separately storing the video and audio data constituting the highlight stream, and then controls the trick play process using the pointer when the highlight is played. By using, it is possible to minimize the consumption of additional storage space by the creation of the highlight stream.

As described above, the present invention can be applied to a summary of contents recorded on a PVR equipped with a hard disk in a digital TV, a quick search for a plurality of recorded programs, and a search in a database system composed of a large amount of digital video. It can be applied to applications related to summary video generation for convenience of work and the like to provide an effective and automatic highlight stream to the user.

As described above, according to the apparatus and method for generating a highlight stream in a PVR according to the present invention, a video stream of a recorded program is divided into a plurality of scene sections by applying a scene change detection algorithm to a video stream of a program recorded in the PVR. Next, the importance level is calculated for each scene section and the highlight stream is generated based on the selected scene section based on the importance level, thereby automatically selecting a part to be preferentially viewed by the user for the stored program.

In addition, the present invention has an effect of presenting a quantitative criterion for selecting a section having a meaningful content in the context of automatic highlight stream generation by calculating the importance level using volume information for each scene section.

In addition, the present invention stores only the pointer information of the start and end points of the scene section constituting the highlight without directly storing the video stream of the section selected as the highlight, and then uses the pointer information at the time of highlight playback to record the original recorded program. By selecting and playing back only the section set as the highlight, there is an effect of saving the space required for storing the highlight.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention.

Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims.

Claims (9)

A scene change determination unit analyzing a video stream of a broadcast program recorded on a storage medium and dividing the program into a plurality of sections according to a scene change; A section volume estimator for estimating the importance level of each scene section by analyzing a voice signal for each scene section; And And a highlight generator configured to select scene sections necessary for generating a highlight stream based on the importance, and store location information of the selected scene sections in the storage medium. The method of claim 1, And a video comparator configured to receive a video stream of a recorded broadcast program from the storage medium, compare a quantitative difference between two video frame signals, and output a frame error value to the scene change determiner. The scene change determining unit determines whether to change the scene of the video stream by comparing the frame error value with a predetermined threshold value, and divides the video stream into a plurality of scene sections based on the scene change, and then displays information of each scene section. Highlight stream generation device in the PVR, characterized in that for outputting. The method of claim 1, And a video frame included from one point at which a scene change occurs to just before a point at which a next scene change occurs in the scene change determining unit constitutes one scene section. The apparatus of claim 1, wherein the section volume estimating unit An apparatus for highlight stream generation in a PVR, comprising receiving a decoded audio signal and calculating an average volume value of each scene section to use for measuring the importance of each scene section. The method of claim 1, The location information of the scene section selected as the highlight by the highlight generator is pointer information of the start frame and the end frame of the scene section selected as the section constituting the highlight. (a) analyzing a video stream of a broadcast program recorded on a storage medium and dividing the program into several sections according to scene change; (b) estimating importance of each scene section by analyzing a voice signal for each scene section; And (c) selecting scene sections necessary for generating a highlight stream based on the importance level, and storing position information of the selected scene sections on the storage medium. The method of claim 6, wherein step (b) A method of generating a highlight stream in a PVR, wherein the average volume value of each scene section is calculated and output as the importance of each scene section. The method of claim 6, wherein step (c) And a pointer information of a start frame and an end frame of a scene section selected as a highlight constituting section as position information of a scene section selected as a highlight. The method of claim 8, And controlling a trick play by referring to a corresponding highlight section pointer in a video stream of a broadcast program recorded on the storage medium when the highlight is played back.