KR20090108876A - System and method for detecting a synchronization image - Google Patents

Abstract

PURPOSE: A synchronism screen detector system and a method thereof for improving the comparison accuracy of a detected feature screen are provided to detect a special screen when manufacturing PPL information for synchronizing the screen. CONSTITUTION: A synchronism screen detector system and a method thereof for improving the comparison accuracy of a detected feature screen includes a digital broadcast reception terminal, a synchronism server(130), and a service providing server(120). The digital broadcast reception terminal displays and receives the broadcasting contents and additional information. The synchronism server extracts each video frame from the broadcasting contents. The synchronism server produces the color variation of the front-to-back frame. The synchronism server extracts the feature screen and compares the color variation with the critical value. The synchronism server removes the similar frame on each feature screen.

Description

System and method for detecting a synchronization image

The present invention relates to a synchronization screen detection system and method for synchronization between content of digital multimedia broadcasting and product placement information. More particularly, the present invention relates to broadcast content and PPL information provided to a digital broadcast receiving terminal. For synchronization, the feature screen is extracted according to the amount of color change between before and after video frames of the broadcast content, similar frames are removed so that the feature screen becomes the only screen over the entire section, and the image frame is removed according to the brightness value. The present invention relates to a synchronization screen detection system and method for enabling a synchronization screen to be detected.

In general, DMB (Digital Multimedia Broadcasting) service is a mobile multimedia broadcasting service in which broadcasting and communication are converged. It is a digital broadcasting service that provides high quality video service.

Satellite (D) and terrestrial DMB services, which are commercially available and serviced in Korea, are a new concept of mobile broadcasting services that overcome the spatial limitations of existing broadcasting and provide differentiated broadcasting services as mobile terminals.

Here, the satellite DMB is a digital multimedia broadcasting through a satellite broadcast signal, it is possible to provide an interactive service. However, data transmission of two-way service using satellite signals has limitations in capacity and diversity of information, and in order to compensate for these limitations, it is possible to provide a service that meets the needs of viewers more extensively through the Internet connection using a mobile communication network. .

The service provided through the mobile communication network is provided by subdividing the viewer's viewing channel, the program broadcasting in the channel, the information that can be provided in the program, and the information that can be searched in the scene currently being watched for such information. can do.

Such additional information, for example, product placement (PPL) information, may be produced in advance before being broadcast to a viewer and stored with time information for each program. In order to provide data to pre-produced information to any viewer, when the viewer requests the information, the service provider needs to know which scene of which broadcast the viewer has viewed and requested the information.

The pre-produced information may be information about a person, a product, and the like for a specific program. Since the information is about a feature scene in the program, in order to provide such information, an accurate scene requested by the viewer or data at that time is required. .

By the way, the existing technology is no automated technology capable of synchronizing the pre-produced information and the scene at the time of the viewer request. Accordingly, there is a demand for a synchronous technology capable of providing the viewer with pre-produced information at an appropriate time after judging by the human eye a point of view of a program currently being broadcast.

In addition, for this synchronization technique, for example, when producing PPL information, it is more efficient to detect and compare a small number of screens that can be compared than frame comparison of every video. This is required.

In order to solve the above requirements, the present invention extracts a feature screen according to the amount of color change between the front and rear image frames of the broadcast content for synchronization between the broadcast content provided to the digital broadcast receiving terminal and the PPL information, and the feature screen is the entire section. It is an object of the present invention to provide a synchronous screen detection system and method for removing a similar frame so as to be the only screen over and detecting a synchronous screen by removing an image frame according to a brightness value.

In accordance with another aspect of the present invention, there is provided a synchronization screen detecting system comprising: a digital broadcast receiving terminal configured to receive and display broadcast content and additional information; Extracting each image frame from the broadcast content, calculating a color change amount between front and rear frames for each of the extracted image frames, and extracting a feature screen and a playback time by comparing the color change amount with a threshold value; A synchronization server for removing a similar frame and detecting a synchronization screen by removing a frame according to a brightness value from the feature screen, and providing the detected synchronization screen; And a service providing server for transmitting additional information corresponding to the synchronization screen provided from the synchronization server and the synchronization screen to the digital broadcast receiving terminal, wherein the digital broadcast receiving terminal is received from the service providing server. The broadcast content and the additional information are synchronized based on the synchronization screen.

In addition, the synchronization server extracts each video frame by designating a comparison screen detection section for the broadcast content.

In addition, the sync server compares each video frame with a different video frame for the similar frame to be removed from the feature screen, and compares the pixel value of each corresponding video frame with respect to the pixel value of the entire video frame. The difference in color is calculated, and the similarity between the two image frames is determined by adding the difference values.

In addition, the synchronization server divides each color into 256 steps when the brightness of the screen is 8-bit color based on the black screen as "0" when removing the frame according to the brightness value, and for each color. If the sum is less than the lower 10% or more than the upper 10%, the sum is determined as a black screen or a white screen and is removed from the feature screen.

The threshold may be designated according to a source state of a video and a broadcast reception state.

Meanwhile, a digital broadcast receiving terminal for receiving and displaying broadcast content according to the present invention for achieving the above object, a service providing server for providing additional information on the broadcast content, and synchronization between the broadcast content and the additional information. A synchronization screen detection method of a system including a synchronization server for providing information, the method comprising: (a) the synchronization server extracting each video frame from the broadcast content, and calculating a color change amount between front and rear frames with respect to each of the extracted video frames; And extracting a feature screen by comparing the amount of color change with a threshold value, removing a similar frame from each feature screen, and removing a frame according to a brightness value from the feature screen to detect a sync screen, and detecting the sync screen. Transmitting a screen to the service providing server; (b) the service providing server transmitting additional information corresponding to the synchronization screen received from the synchronization server and the synchronization screen to the digital broadcast receiving terminal; And (c) synchronizing the broadcast content with the additional information based on the synchronization screen received from the service providing server.

In addition, in the step (a), the sync server extracts each video frame by designating a comparison screen detection section for the broadcast content.

In addition, in the step (a), for the similar frame for the synchronization server to remove from the feature screen, each image frame is compared with each other image frame, and corresponding to each pixel value of the entire image frame. The color difference between pixel values of another image frame is calculated, and the similarity between the two image frames is determined by adding the difference values together.

Further, in the step (a), when the synchronization server removes the frame according to the brightness value, each color is divided into 256 steps when the brightness of the screen is 8 bit color based on the black screen as "0". When the sum value for each color is less than the lower 10% or the upper 10% or more, it is determined as a black screen or a white screen and removed from the feature screen.

The threshold may be designated according to a source state of a video and a broadcast reception state.

On the other hand, the communication unit for transmitting the synchronization information to the service providing server for providing additional information to the digital broadcast receiving terminal for receiving and displaying the broadcast content according to the present invention for achieving the above object; Extracting each image frame from the broadcast content, calculating a color change amount between front and rear frames for each of the extracted image frames, and extracting a feature screen and a playback time by comparing the color change amount with a threshold value; A synchronization surface detector for removing a similar frame and detecting a synchronization screen by removing a frame according to a brightness value from the feature screen; A control unit controlling an operation of the synchronization screen detector to detect the synchronization screen from the broadcast content; And a synchronization information storage unit for storing the extracted feature screen, the reproduction time, and the detected synchronization screen.

The controller may be further configured to compare each image frame with a different image frame with respect to the similar frame for removal from the feature screen through the synchronization surface detector, and to correspond to each corresponding pixel value of the entire image frame. The color difference between the pixel values of the image frame is calculated, and the similarity between the two image frames is determined by adding up the difference values.

The controller may divide each color into 256 levels when the brightness of the screen is 8 bit color based on the black screen as “0” when the frame according to the brightness value is removed through the synchronization surface detector. When the sum value for each color is less than the lower 10% or the upper 10% or more, it is determined as a black screen or a white screen and removed from the feature screen.

The control unit may designate a comparison screen detection section for the broadcast content through the synchronization surface detector to extract the respective video frames.

의 수식으로 산출하며, 프레임

과

는

크기이며 각각 n번째, n-1번째 영상을 나타내며, i와 j는 각각 가로와 세로의 위치 값을 나타낸다.The controller may be further configured to adjust an average change amount MAFD of one pixel through the synchronization plane detector.

Calculated by the formula of, frame

and

Is

It is the size and represents the nth and n-1th images, respectively, and i and j represent the position values of the horizontal and vertical, respectively.

와 현재 프레임의 차이

의 차이 값이 임계 값보다 크고

도 임계 값보다 크게 되면 장면이 전환되었다고 판단하여 상기 특징 화면으로 추출하게 된다.And, the control unit, the difference between the previous frame through the synchronization surface detection unit

And the difference between the current frame

Is greater than the threshold

If it is larger than the threshold value, it is determined that the scene has been changed and extracted to the feature screen.

A synchronization screen detection method of a synchronization server for detecting a synchronization screen for synchronization between the additional information and a broadcast content of a service providing server for providing additional information according to the present invention for achieving the above object, (a) from the broadcast content Extracting each image frame; calculating a color change amount between front and rear frames for each of the extracted image frames; (c) extracting a feature screen and a reproduction time by comparing the change amount with a threshold value; (d) removing similar frames from each feature screen; And (e) extracting a synchronization screen by removing a frame according to a brightness value from the feature screen from which the similar frame is removed.

In addition, in the step (a), a comparison screen detection section is designated for the broadcast content to extract the respective video frames.

의 수식으로 산출하고, 프레임

과

는

크기이며 각각 n번째, n-1번째 영상을 나타내며, i와 j는 각각 가로와 세로의 위치 값을 나타낸다.Further, in the step (b), the average change amount MAFD of one pixel is

Calculated by the formula of the frame

and

Is

It is the size and represents the nth and n-1th images, respectively, and i and j represent the position values of the horizontal and vertical, respectively.

와 현재 프레임의 차이

의 차이 값이 임계 값보다 크고

도 임계 값보다 크게 되면 장면이 전환되었다고 판단하여 상기 특징 화면으로 추출하게 된다.In addition, the step (b), the difference of the previous frame

And the difference between the current frame

Is greater than the threshold

If it is larger than the threshold value, it is determined that the scene has been changed and extracted to the feature screen.

In addition, the threshold value may be specified according to a source state of a video and a broadcast reception state.

Further, in the step (d) and the step (e), the similar frame is compared with the different image frame for each image frame, and the pixel value of each corresponding other image frame with respect to the pixel value of the entire image frame. The difference in color is calculated, and the similarity between the two image frames is determined by adding the difference values.

In the step (e), removing the frame according to the brightness value divides each color into 256 steps when the brightness of the screen is an 8-bit color based on the black screen as "0", and for each color If the sum is less than 25, which is the lower 10%, or 230 or more, which is the upper 10%, the sum is determined as a black screen or a white screen and removed from the feature screen.

On the other hand, the recording medium according to the present invention for achieving the above object, (a) extracting each image frame from the broadcast content; calculating a color change amount between front and rear frames for each of the extracted image frames; (c) extracting a feature screen and a reproduction time by comparing the change amount with a threshold value; (d) removing similar frames from each feature screen; And (e) a program for processing a step of extracting a synchronization screen by removing a frame according to a brightness value from the feature screen from which the similar frame is removed.

In addition, in the step (a), a comparison screen detection section is designated for the broadcast content to extract the respective video frames.

의 수식으로 산출하고, 프레임

과

는

크기이며 각각 n번째, n-1번째 영상을 나타내며, i와 j는 각 각 가로와 세로의 위치 값을 나타낸다.Further, in the step (b), the average change amount MAFD of one pixel is

Calculated by the formula of the frame

and

Is

It is the size and represents the nth and n-1th images, respectively, and i and j represent the position values of the horizontal and vertical, respectively.

와 현재 프레임의 차이

의 차이 값이 임계 값보다 크고

도 임계 값보다 크게 되면 장면이 전환되었다고 판단하여 상기 특징 화면으로 추출하게 된다.In addition, the step (b), the difference of the previous frame

And the difference between the current frame

Is greater than the threshold

If it is larger than the threshold value, it is determined that the scene has been changed and extracted to the feature screen.

In addition, the threshold value may be specified according to a source state of a video and a broadcast reception state.

Further, in the step (d) and the step (e), the similar frame is compared with the different image frame for each image frame, and the pixel value of each corresponding other image frame with respect to the pixel value of the entire image frame. The difference in color is calculated, and the similarity between the two image frames is determined by adding the difference values.

In the step (e), removing the frame according to the brightness value divides each color into 256 steps when the brightness of the screen is an 8-bit color based on the black screen as "0", and for each color If the sum is less than 25, which is the lower 10%, or 230 or more, which is the upper 10%, the sum is determined as a black screen or a white screen and removed from the feature screen.

According to the present invention, it is possible to provide a technique for detecting a feature screen when producing PPL information for synchronization through scene comparison.

In addition, the comparison accuracy of the detected feature screen may be increased.

In addition, synchronization can be performed without changing the existing satellite broadcast signal, and a technique for detecting a synchronization screen to be compared to find and synchronize a similar screen between two screens can be provided.

Details of the object and technical configuration of the present invention and the resulting effects thereof will be more clearly understood by the following detailed description based on the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram schematically showing an example of a mobile communication broadcast system to which a synchronization screen detection method according to an embodiment of the present invention is applied.

Referring to FIG. 1, a mobile communication broadcast system 100 to which the present invention is applied includes a broadcast center 110, a service providing server 120, a synchronization server 130, a content server 140, a mobile communication network 150, and the like. DMB terminal 160 is included.

The broadcast center 110 provides satellite DMB content to the DMB terminal 160 via satellite or provides terrestrial DMB content to the DMB terminal 160 through a terrestrial transmission station.

The service providing server 120 holds additional information on broadcast content including satellite DMB content and terrestrial DMB content, and provides the DMB terminal 160 with additional information about broadcast content, for example, product placement (PPL) information. to provide.

In addition, when the service providing server 120 receives the synchronization information from the synchronization server 130, the service providing server 120 transmits additional information and synchronization information corresponding to the synchronization information to the DMB terminal 160. That is, the service providing server 120 generates additional information associated with the second broadcast content provided from the content provider CP not shown in FIG. 1 and the first broadcast content broadcast in real time, and the additional information and the first information. The synchronization information and the additional information for synchronizing the broadcast content are transmitted to the DMB terminal 160.

In addition, the service providing server 120 is a GUIP (Graphic User Interface in Picture) server that provides information on the type of additional information that can be provided for each object displayed on the digital multimedia broadcasting screen.

Here, the additional information includes plot information of a specific program, character information of an actor, background music information, PPL information about a product appearing in a program, and the like. Hereinafter, the additional information provided by the service providing server 120 is referred to as PPL information. It demonstrates as follows.

That is, the service providing server 120 enables the user to select a specific object on the broadcast screen through the linkage with the DMB terminal 160 receiving the PPL information, and the user to receive additional information content on the selected object. As a supporting server, for example, when the user presses any GUIP start key while watching a broadcast, the contours of the objects (eg, a singer, a singer's costume, etc.) appearing on the broadcast screen after that point in time can be provided for each object. Information about the type of information (for example, the profile of the singer, the record of the singer, the lyrics of the song the singer sings, the information of the costume of the singer, etc.), and the information, that is, the download address (URL) of the content is transmitted.

In this case, the contour information is information for allowing a user to select an object. When the contour information is synchronized with the broadcast screen, the DMB terminal 160 can display an outline of an object selected according to a user's key input. Is provided as guide information that allows a particular object to be selected.

If the DMB terminal 160 has a touch screen, it is not necessary to provide guide information separately for object selection. In this case, the DMB terminal 160 sends coordinate information of a point selected by the user to the service providing server 120. By transmitting information, information on an object corresponding to the selected point may be directly provided.

Therefore, in such a case, the service providing server 120 does not have to provide all PPL information for subsequent broadcasting in advance, and information on the type of additional information that can be provided to the object of the corresponding point when selecting a specific point and content URL information. It is only necessary to provide (hereinafter referred to as GUIP information).

As such, when the DMB terminal 160 selects an object (a person, a product, etc.) appearing on the broadcast screen while watching a broadcast, the service providing server 120 according to the present invention enables the user to check information available for the selected object. A URL is provided so that content about the information can be downloaded from the content server 140.

In order to provide the synchronization information to the service providing server 120, the synchronization server 130 extracts each image frame from the broadcast content, measures the amount of color change between the front and rear frames for each extracted image frame, and then displays a feature screen larger than a threshold value. Is detected and the detected feature screen is stored together with the reproduction time of the feature screen.

At this time, the synchronization server 130 extracts each image frame by designating a comparison screen detection section for the broadcast content, calculates a change graph by measuring the amount of color change between front and rear frames for each image frame, and calculates a change graph from the threshold value. The large feature screen is detected. Here, the synchronization server 130 is connected to the broadcast center 110 and extracts an image frame from the broadcast content received from the broadcast center 110.

In addition, the synchronization server 130 performs a process of removing similar screens and frames with a high probability of error in order to increase the comparison accuracy with respect to the detected feature screen. That is, the synchronization server 130 removes the similar frame so that one feature frame becomes the only screen over the entire interval with respect to the extracted entire feature frame, and removes the frame that is not suitable for screen comparison according to the brightness value to remove the sync screen. Will be detected.

The content server 140 holds content matched with PPL information, and when PPL information for the DMB terminal 160 is selected from the service providing server 120, the content server 140 displays content matched with the selected PPL information. Will be downloaded.

The DMB terminal 160 receives and displays broadcast content and PPL information related to digital multimedia, and synchronizes and displays the broadcast content and PPL information based on the synchronization information. Here, the DMB terminal 160 is a digital broadcast receiving terminal including a mobile communication terminal and a PDA that can receive and display DMB broadcasting.

The DMB terminal 160 is a terminal that receives satellite DMB broadcasting contents from the broadcasting center 110 via satellite or receives terrestrial DMB broadcasting contents through a terrestrial transmission station. The DMB terminal 160 is provided through the packet network 150 and the mobile communication network 150. A terminal accessible to the service providing server 120 and the content server 140.

Herein, the mobile communication network 150 includes a base station transmitter (BTS), a base station controller (BSC), and a mobile switching center (MSC), and thus a detailed description thereof will be omitted. In addition, the present invention is not limited to such a mobile communication network but can be equally applied to a wired communication network and a satellite communication network.

2 is a configuration diagram schematically showing an internal configuration of a service providing server according to an embodiment of the present invention.

Referring to FIG. 2, the service providing server 120 may include a service control unit 211, a subscriber database 212, an additional information search unit 213, an additional information database 214, an additional information generation unit 215, and a GUIP. The information providing unit 216 and the communication unit 270 are configured.

The service providing server 120 not only provides a synchronization service for transmitting synchronization information on synchronization between the broadcast content and the additional information to the DMB terminal 160, but also provides the user with the broadcast content displayed on the DMB terminal 160. When an arbitrary object is selected by the user, a GUIP service for transmitting PPL information of the selected object to the DMB terminal 160 is also provided.

Therefore, in the synchronous service mode, the service control unit 211 transmits a synchronization information request message for a specific broadcast program to the synchronization server 130, receives synchronization information from the synchronization server 130, and receives the DMB terminal 160. Will be sent to.

In addition, in the GUIP service mode, the service control unit 211 receives a GUIP start request message including channel information and program information of a broadcast currently being viewed from the DMB terminal 160 and includes the received GUIP start request message in the received GUIP start request message. The GUIP start response message including the received information is transmitted to the DMB terminal 160 so that the information can be confirmed by the user or the terminal.

The GUIP start request message is a service request message that is automatically generated and transmitted from the DMB terminal 160 as the user selects the GUIP start key from the DMB terminal 160. The GUIP start request message includes: MTS number, PTS information provided to know the current broadcast time; GUIP service type information, terminal type information (laptop, mobile phone, etc.), broadcast channel information, broadcast program information may be included.

PTS (Presentation Time Stamp) is a time stamp included in a DMB broadcast data packet. When PTS of the packet header is used, the PTS can find a screen when a user requests a service from a corresponding broadcast program. 120 provides PPL information or GUIP information for the time or after the time.

The GUIP service type information is information for distinguishing the download method, and the method of downloading the PPL information by the service providing server 120 includes the full download method, the partial download method, and the ON. There is an ONDEMAND method.

Here, the full download method and the split download method may download PPL information including contour information, types of additional information, and content URLs of a plurality of objects for subsequent broadcasts of the corresponding broadcast program when the PPL information request message is received. In this case, the overall download method is to download the PPL information for the entire broadcast program at once, and the split download method periodically divides the PPL information into a suitable size to distribute the network traffic load at regular intervals. This is how you download sequentially.

In addition, the on-demand method is applied to the DMB terminal 160 with a touch screen. When a specific point is received and a GUIP information request message including coordinates is received, only the type of the additional information and the content URL of the corresponding object are received. This method downloads GUIP information.

Such a GUIP service type may be preset in the DMB terminal 160, may be selected when the GUIP start key is pressed, or may be stored in the subscriber database 212 of the service providing server 120.

On the other hand, if the GUIP service is provided only to the service subscriber, the service control unit 211 checks whether the information of the subscriber exists in the subscriber database 212 when receiving the GUIP start request message, performs the subscriber authentication and then includes the authentication result Will reply to the GUIP start response message. To this end, the subscriber database 212 stores subscriber information.

Next, when the additional information retrieval unit 213 receives the PPL information request message including the channel information and the program information of the broadcast from the DMB terminal 160, the additional information database 214 is based on the information included in the PPL information request message. PPL information about the broadcast program is searched for and the URL information for downloading the PPL information is provided to the DMB terminal 160.

At this time, the additional information retrieval unit 213 should retrieve the PPL information for the corresponding broadcast after the DMB terminal 160 requests the PPL information, and thus is included in the PPL information request message received from the DMB terminal 160. The PPL information for the part to be watched later is searched using the PTS information indicating the reproduction time of the broadcast program.

The additional information database 214 includes guide information for selecting objects displayed on the broadcast screen, PPL information including types of additional information that can be provided for each object, and content URL information for downloading each additional information content. Is stored for each broadcast program.

For example, in the case of a music broadcast program, an object of the PPL information may be a singer displayed on the screen. Examples of additional information that may be provided for the singer include profile information of the singer, record information of the singer, and the current information. There may be a song being played. In addition, if the user selects a song, the corresponding sound source content may be provided as additional information content. In this case, the download URL information of the sound source content is included in the PPL information.

In addition, the guide information is information for allowing a user to select one of the objects displayed on the broadcast screen. As described above, the guide information may be outline information for displaying the outline of the object, and may be variously designed in addition to the outline. . In addition, the guide information is synchronized with the broadcast data in the DMB terminal 160 to display guide information such as the outline of the corresponding object according to the key input so that the user can select the desired object through the key input while watching the broadcast.

When the additional information search unit 215 provides URL information for downloading the PPL information searched to the DMB terminal 160 by the additional information search unit 213, when the DMB terminal 160 accesses the corresponding URL, Download the corresponding PPL information.

As mentioned above, since the PPL information download method includes a full download method and a split download method, the additional information generator 215 may include a full download method that is a download method of PPL information in a PPL information request message, and a split download method. When the information on any one of the download method is included, or when the information is stored in the subscriber database 212, when the full download method according to the information, the entire PPL information is downloaded at once, and the split download method The PPL information to be downloaded is divided into appropriate sizes, and each split PPL information is sequentially downloaded.

Next, the GUIP information providing unit 216 is a configuration for a service to the DMB terminal 160 having a touch screen, and the channel information of the broadcast currently being viewed from the DMB terminal 160 as the user selects a specific point of the broadcast screen. When a GUIP information request message including program information, coordinate information of a selected point, and PTS information is received, referring to the additional information database 214 based on the information included in the GUIP information request message, The GUIP information response message including the type of additional information that can be provided for the object and the content URL information for downloading each additional information content is transmitted to the DMB terminal 160.

In this case, the additional information database 214 should further store the coordinate information of the object displayed through the broadcast screen, and exactly the information on the range of coordinates for the screen area occupied by the object.

On the other hand, in the case of the additional information generation unit 215 is to give down the PPL information for the broadcast after the time the user requests the service, since the corresponding PPL information includes guide information for selecting an object, the DMB terminal 160 After downloading the PPL information, the user can identify the object selected by the user and receive the corresponding additional information content. In addition, the additional information generator 215 generates additional information associated with the second broadcast content provided from the content provider CP. In this case, the second broadcast content is the same broadcast content as the first broadcast content that is broadcast in real time.

On the other hand, the GUIP information providing unit 216 receives the coordinate information selected by the user in real time, identifies the object corresponding to the coordinates and gives only relevant information.

Meanwhile, in FIG. 2, when the PMB information is downloaded to identify the object in the DMB terminal 160 and when the coordinates are transmitted in the DMB terminal 160, the service providing server 120 identifies the object and provides only GUIP information. Although the service providing server 120 is configured to perform both cases, the GUIP information providing unit 216 may be omitted if only the former service is provided, and the additional information retrieval unit may be omitted if only the latter service is provided. 213 and the additional information generator 215 may be omitted.

3 is a configuration diagram schematically showing an internal configuration of a synchronization server according to an embodiment of the present invention.

Referring to FIG. 3, the synchronization server 120 according to the present invention includes a communication unit 310, a synchronization surface detector 320, a controller 330, and a synchronization information storage unit 340.

The communication unit 310 transmits or receives information or data, and transmits synchronization information to the service providing server 120 providing additional information.

The synchronization surface detector 320 extracts each image frame from the broadcast content, calculates a color change amount between the front and rear frames for each extracted image frame, and extracts a feature screen and a playback time by comparing the change amount with a threshold value. The synchronization screen is detected by removing similar frames from the screen and removing frames according to brightness values from the feature screen.

The controller 330 controls the operation of the synchronization surface detector 320 to detect the synchronization screen from the broadcast content.

That is, the controller 330 compares each image frame with each other image frame with respect to the similar frame to be removed from the feature screen through the synchronization surface detector 320, and corresponds to each pixel value of the entire image frame. The color difference between pixel values of different image frames is calculated, and the similarity between the two image frames is determined by adding the difference values together.

In addition, when removing the frame according to the brightness value, the controller 330 divides each color into 256 steps when the brightness of the screen is an 8-bit color based on the black screen as "0", and adds the sums for each color. If the value is less than 25, which is the lower 10%, or 230 or more, which is the upper 10%, the black screen or the white screen is judged to be removed from the feature screen, and the synchronization screen is detected.

The synchronization information storage unit 340 stores and stores the extracted feature screen and reproduction time and the detected synchronization screen.

4 is a flowchart illustrating a method of detecting a sync screen according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the synchronization server 130 according to the present invention extracts each image frame from the broadcast content through the synchronization surface detector 330 (S410).

At this time, the synchronization server 130 designates a comparison screen detection section for the broadcast content, and extracts each image frame within the section.

Subsequently, the synchronization server 130 calculates a color change amount between the front and rear frames for each extracted image frame (S420).

At this time, the mean variation amount (MAFD: Mean Absolute Frame Differences) of one pixel is calculated as in Equation 1 below.

과

는

크기이며 각각 n번째, n-1번째 영상을 나타내며, i와 j는 각각 가로와 세로의 위치 값을 나타낸다.Where frame

and

Is

It is the size and represents the nth and n-1th images, respectively, and i and j represent the position values of the horizontal and vertical, respectively.

와 현재 프레임의 차이

의 차이 값이 임계 값보다 크고

도 임계값보다 크게 되면 장면이 전환되었다고 판단하여 동기화면 검출부(320)를 통해 특징 화면으로 추출하게 된다.The control unit 330 is the difference of the previous frame

And the difference between the current frame

Is greater than the threshold

If it is larger than the threshold value, it is determined that the scene has been changed and extracted by the synchronization screen detector 320 as a feature screen.

In this case, the threshold may be designated according to the source state of the video and the broadcast reception state.

Subsequently, the synchronization server 130 extracts the feature screen and the reproduction time as shown in FIG. 5 by comparing the change amount calculated as described above with a threshold value (S430).

5 is a diagram illustrating a feature screen extracted by a synchronization server according to an exemplary embodiment of the present invention. The feature screens shown in FIG. 5 are screens extracted by the synchronization surface detector 320 through the above-described process.

However, the extracted feature screen includes similar screens between frames and screens that are inappropriate for comparison. Therefore, the synchronization server 130 performs the following process to increase the accuracy of the comparison.

Subsequently, the synchronization server 130 removes the similar frame from each feature screen (S440).

Accordingly, the synchronization server 130 obtains the feature screen a from which similar frames are removed from each feature screen through the synchronization surface detector 320.

Here, since each feature screen is extracted as a change amount with respect to the front and rear frames, the likelihood of a similar frame coming out in a section other than the front and rear frames is very high. Accordingly, the purpose of the present invention is to allow the feature frame to be the only screen over the entire section by removing the similar frame with respect to the entire feature frame.

In this case, the controller 330 compares each image frame with each other for the similar frame, and calculates a color difference of the pixel value of each corresponding image frame with respect to the pixel value of the entire image frame, and the difference value. The similarity between the two image frames is determined using the sum of the two values.

Subsequently, the synchronization server 130 extracts the synchronization screen as shown in FIG. 6 by removing a frame according to the brightness value from the feature screen from which the similar frame is removed (S450).

6 is a diagram illustrating a feature screen from which a similar frame extracted by a sync server is removed and a sync screen from which a frame is removed according to a brightness value according to an embodiment of the present invention.

At this time, the controller 320 divides each color into 256 steps when the brightness of the screen is an 8-bit color based on the black screen as "0" for removing the frame according to the brightness value, and adds the sums for the respective colors. If the value is less than 25, which is the lower 10%, or 230 or more, which is the upper 10%, the black screen or the white screen is determined, and the corresponding black is removed from the feature screen (a) in which the similar frame is removed through the synchronization screen detector 320. The synchronization screen (b) is extracted by removing the screen or the white screen.

The synchronization server 130 that detects the synchronization screen by the above-described process generates the synchronization information based on the synchronization screen and the reproduction time, and transmits the synchronization information to the service providing server 120. The service providing server 120 synchronizes the synchronization information. Will be received and stored.

Meanwhile, the service providing server 120 may receive the GUIP request message from the DMB terminal 160 while keeping the synchronization information.

That is, when the user viewing the DMB content through the DMB terminal 160 selects the GUIP start key, the DMB terminal 160 transmits the GUIP request message to the service providing server 120, and thus the service providing server. 120 receives a GUIP request message including channel information, broadcast program information, PTS information, and the like.

The service providing server 120 transmits the GUIP request response message to the DMB terminal 160 by including the information included in the received GUIP request message for the DMB terminal 160 or the confirmation of the subscriber.

The DMB terminal 160 transmits a PPL information request message including a channel, a program, and PTS information to the service providing server 120. Accordingly, the service providing server 120 corresponds to the channel, program, and PTS information based on the corresponding channel, program, and PTS information. The user searches for PPL information about a broadcast being watched and includes URL information for downloading the PPL information in the PPL information response message.

Upon receiving the PPL information response message, the DMB terminal 160 receives the PPL information automatically or accesses the corresponding URL to download the PPL information.

The DMB terminal 160 synchronizes the broadcast content and the PPL information based on the start PTS included in the synchronization information message stored in the memory.

Accordingly, the DMB terminal 160 synchronizes the broadcast content with the PPL information to support the user's object selection and additional information selection. When the user selects an outline or a point of a specific object while watching a broadcast, the additional information on the object is displayed. If one of the selected is selected, the user accesses the content URL matched with the additional information and downloads the content desired by the user.

As described above, according to the present invention, in order to synchronize the broadcast content provided to the DMB terminal with the PPL information, the feature screen is extracted according to the amount of color change between the front and rear image frames of the broadcast content, and the feature screen is the only screen over the entire section. The synchronization screen detection system and method may be realized to remove the similar frame so that the synchronization screen may be detected by removing the image frame according to the brightness value.

As those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features, the embodiments described above should be understood as illustrative and not restrictive in all aspects. Should be. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

The present invention can be applied to a system for providing synchronization information to a DMB terminal that receives and displays terrestrial or satellite DMB broadcasts. The present invention also provides a mobile communication broadcast system that provides broadcast contents provided from the Internet to a mobile communication terminal through a mobile communication network. Applicable

1 is a configuration diagram schematically showing a configuration of a mobile communication broadcasting system to which a synchronization screen detecting method according to an embodiment of the present invention is applied;

2 is a configuration diagram schematically showing an internal configuration of a service providing server according to an embodiment of the present invention;

3 is a configuration diagram schematically showing an internal configuration of a synchronization server according to an embodiment of the present invention; and

4 is an operation flowchart for explaining a synchronization screen detection method according to an embodiment of the present invention;

5 is a view showing a feature screen extracted by a synchronization server according to an embodiment of the present invention; and

6 is a diagram illustrating a feature screen from which a similar frame extracted by a sync server is removed and a sync screen from which a frame is removed according to a brightness value according to an embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

100: mobile communication broadcasting system 110: broadcasting center

120: service providing server 130: synchronous server

140: content server 150: mobile communication network

160: DMB terminal 211: service control unit

212: subscriber DB 213: additional information search unit

214: additional information DB 215: additional information generating unit

216: GUIP information provider 217: communication unit

310: communication unit 320: synchronization surface detection unit

330: control unit

340: synchronization information storage unit

Claims (30)

A digital broadcast receiving terminal for receiving and displaying broadcast content and additional information; Extracting each image frame from the broadcast content, calculating a color change amount between front and rear frames for each of the extracted image frames, and extracting a feature screen and a playback time by comparing the color change amount with a threshold value; A synchronization server for removing a similar frame and detecting a synchronization screen by removing a frame according to a brightness value from the feature screen, and providing the detected synchronization screen; And And a service providing server for transmitting additional information corresponding to the synchronization screen received from the synchronization server and the synchronization screen to the digital broadcast receiving terminal. And the digital broadcast receiving terminal synchronizes the broadcast content and the additional information based on the sync screen received from the service providing server. The method of claim 1, And the synchronization server extracts the respective video frames by designating a comparison screen detection section for the broadcast content. The method of claim 1, The synchronization server compares each video frame with a different video frame for the similar frame to be removed from the feature screen, and compares the color of the pixel value of each corresponding video frame with respect to the pixel value of the entire video frame. And calculating the difference and determining the similarity between the two image frames by adding the difference values together. The method of claim 1, When the frame according to the brightness value is removed, the sync server divides each color into 256 levels when the brightness of the screen is an 8-bit color based on the black screen as "0", and adds the sum values for each color. If the lower 10% or less than the upper 10% or more, the screen is judged as a black screen or a white screen, characterized in that to remove from the feature screen. The method of claim 1, And said threshold value is specified according to a source state of a video and a broadcast reception state. A synchronization screen of a system including a digital broadcast reception terminal for receiving and displaying broadcast content, a service providing server for providing additional information on the broadcast content, and a synchronization server for providing synchronization information between the broadcast content and the additional information. As a detection method, (a) the synchronization server extracts each image frame from the broadcast content, calculates a color change amount between front and rear frames for each extracted image frame, extracts a feature screen by comparing the color change amount with a threshold value, and Removing a similar frame from each feature screen and removing a frame according to a brightness value from the feature screen to detect a sync screen, and transmitting the detected sync screen to the service providing server; (b) the service providing server transmitting additional information corresponding to the synchronization screen received from the synchronization server and the synchronization screen to the digital broadcast receiving terminal; And (c) synchronizing the broadcast content with the additional information based on the synchronization screen received from the service providing server by the digital broadcast receiving terminal; Sync screen detection method comprising a. The method of claim 6, In the step (a), the synchronization server detects each video frame by designating a comparison screen detection section for the broadcast content. The method of claim 6, In the step (a), for each similar frame for the synchronization server to remove from the feature screen, the respective image frames are compared with each other image frame, and each corresponding other image is corresponding to the pixel value of the entire image frame. And calculating the color difference between the pixel values of the frame and determining the similarity between the two image frames by adding the difference values together. The method of claim 6, In the step (a), when the synchronization server removes the frame according to the brightness value, each color is divided into 256 steps when the brightness of the screen is an 8-bit color based on the black screen as "0". And determining that the black screen or the white screen is removed from the feature screen when the sum of the colors is less than the lower 10% or more than the upper 10%. The method of claim 6, The threshold value is specified according to a source state and a broadcast reception state of the video. A communication unit for transmitting synchronization information to a service providing server providing additional information to a digital broadcast receiving terminal for receiving and displaying broadcast content; Extracting each image frame from the broadcast content, calculating a color change amount between front and rear frames for each of the extracted image frames, and extracting a feature screen and a playback time by comparing the color change amount with a threshold value; A synchronization surface detector for removing a similar frame and detecting a synchronization screen by removing a frame according to a brightness value from the feature screen; A control unit controlling an operation of the synchronization screen detector to detect the synchronization screen from the broadcast content; And A synchronization information storage unit for storing the extracted feature screen and the reproduction time and the detected synchronization screen; Synchronous server comprising a. The method of claim 11, The control unit compares each image frame with a different image frame with respect to the similar frame to be removed from the feature screen through the synchronization screen detector, and corresponds to another image frame corresponding to pixel values of the entire image frame. And calculating the color difference between the pixel values of and determining the similarity between the two image frames by adding the difference values together. The method of claim 11, The control unit divides each color into 256 levels when the brightness of the screen is 8 bit color based on the black screen as “0” when the frame according to the brightness value is removed through the synchronization surface detector. And if the sum of the colors is less than the lower 10% or more than the upper 10%, determine that the black screen or the white screen is removed from the feature screen. The method of claim 11, And the controller is configured to extract each image frame by designating a comparison screen detection section for the broadcast content through the synchronization screen detector. The method of claim 11, The controller may be configured to adjust the average change amount MAFD of one pixel through the synchronization surface detector.

Calculated by the formula of, frame

and

Is

And a size, each representing an nth and n-1th image, i and j respectively indicating a horizontal and vertical position value. The method of claim 11, The control unit, the difference between the previous frame through the synchronization surface detection unit

And the difference between the current frame

Is greater than the threshold

If it is larger than the threshold value, it is determined that the scene has been changed, the synchronization server, characterized in that extracted to the feature screen. A synchronization screen detection method of a synchronization server for detecting a synchronization screen for synchronization between the additional information of a service providing server providing additional information and broadcast content, (a) extracting each image frame from broadcast content; calculating a color change amount between front and rear frames for each of the extracted image frames; (c) extracting a feature screen and a reproduction time by comparing the change amount with a threshold value; (d) removing similar frames from each feature screen; And (e) extracting a synchronization screen by removing a frame according to a brightness value from the feature screen from which the similar frame is removed; Method of detecting a sync screen of a sync server comprising a. The method of claim 11, In the step (a), the method for detecting a synchronization screen of a synchronization server, characterized in that for extracting the respective video frames by specifying a comparison screen detection section for the broadcast content. The method of claim 17, In the step (b), the average change amount MAFD of one pixel is

Calculated by the formula of the frame

and

Is

And a size, each of which represents an nth and n-1th image, and i and j each indicate a horizontal and vertical position value. The method of claim 19, Step (b), the difference of the previous frame

And the difference between the current frame

Is greater than the threshold

Also, if it is larger than the threshold value, it is determined that the scene has been changed and extracted into the feature screen. The method of claim 20, The threshold value is specified according to a source state and a broadcast reception state of the video. The method of claim 17, In the steps (d) and (e), the similar frame is compared with the different image frames for the respective image frames, and the color of the pixel value of each corresponding other image frame relative to the pixel value of the entire image frame. And calculating the difference and determining the similarity between the two image frames by adding the difference values together. The method of claim 17, In the step (e), removing the frame according to the brightness value divides each color into 256 steps when the brightness of the screen is 8-bit color based on the black screen as "0", and adds the sum values for the respective colors. And if the lower 10% is less than 25 or the upper 10% is 230 or more, it is determined as a black screen or a white screen and removed from the feature screen. (a) extracting each image frame from broadcast content; calculating a color change amount between front and rear frames for each of the extracted image frames; (c) extracting a feature screen and a reproduction time by comparing the change amount with a threshold value; (d) removing similar frames from each feature screen; And (e) extracting a synchronization screen by removing a frame according to a brightness value from the feature screen from which the similar frame is removed; The recording medium on which the program for processing the recorded data. The method of claim 24, Step (a), the recording medium recording the program, characterized in that for extracting the respective video frame by specifying a comparison screen detection section for the broadcast content. The method of claim 24, In the step (b), the average change amount MAFD of one pixel is

Calculated by the formula of the frame

and

Is

And a size, each representing an nth and n-1th image, i and j respectively indicating a horizontal and vertical position value. The method of claim 26, Step (b), the difference of the previous frame

And the difference between the current frame

Is greater than the threshold

And recording the program to the feature screen when it is determined that the scene has been changed. The method of claim 27, The threshold value is specified according to the source state and the broadcast reception state of the video recording medium recording program. The method of claim 24, In the steps (d) and (e), the similar frame is compared with the different image frames for the respective image frames, and the color of the pixel value of each corresponding other image frame with respect to the pixel value of the entire image frame. And calculating a difference and determining the similarity between the two image frames by adding the difference values together. The method of claim 24, In the step (e), removing the frame according to the brightness value divides each color into 256 steps when the brightness of the screen is 8-bit color based on the black screen as "0", and adds the sum values for the respective colors. And if the lower 10% is less than 25 or the upper 10% is 230 or more, the program is recorded as a black screen or a white screen and removed from the feature screen.

Images