KR20160055525A - Video display apparatus and operating method thereof - Google Patents

Abstract

Disclosed is an operating method of a video display apparatus comprising a display unit. The operating method of a video display apparatus according to the present invention comprises the steps of: receiving broadcasting signals including main content and metadata assisting the main content from a content provider; identifying a user by receiving login information; searching data for customized service corresponding to the identified user; converting an output form of the metadata based on the search result; and inputting the converted metadata. The operating method of a video display apparatus according to the present invention can provide a user with customized additional services, and reduce an amount of data to be transmitted.

Description

TECHNICAL FIELD [0001] The present invention relates to a video display device and an operation method thereof,

The present invention relates to an image display apparatus and an operation method thereof.

[0004] 2. Description of the Related Art [0005] A digital broadcasting service is provided and with the development of digital technology, a technique for supplying a higher quality broadcasting service is attracting attention.

In addition, there is an increasing demand for users who want contents and broadcasting service that actively reflects the needs of users together with existing broadcasting services.

However, a system for collecting user information of the current broadcasting system and processing the collected user information is not constructed, and thus it is difficult to provide a broadcasting service suitable for each user.

The present invention provides an image display apparatus and an operation method thereof that recognize a non-compressed audio-visual content type and provide an additional service related to the recognized content to a user.

In addition, the present invention provides an image display apparatus and an operation method thereof that convert and provide supplementary services related to recognized contents based on information acquired from a user.

In particular, among the additional services related to the recognized content, the subtitle service is provided in the image display apparatus according to the user characteristics.

A method of controlling an image display apparatus according to an exemplary embodiment of the present invention includes receiving a broadcast signal including a main content and metadata supporting a main content from a content provider; Receiving login information and identifying a user; Retrieving data for a personalization service corresponding to the identified user; Converting an output format of the metadata based on the search result; And outputting the converted metadata.

The embodiment of the present invention grasps user characteristics through questionnaires and enables additional services that match user characteristics.

In addition, if only one meta data is transmitted without the content provider needing to send all the additional services corresponding to the user characteristics, the amount of data to be transmitted and converted according to the user characteristics in the video display device can be reduced.

In addition, according to the embodiment of the present invention, a menu providing a service related to content can be provided through a second screen rather than a screen of a display device, thereby minimizing the inconvenience between views that a user feels.

1 is a block diagram illustrating a network topology according to an embodiment of the present invention.
2 is a block diagram illustrating a watermark-based network topology in accordance with an embodiment of the present invention.
3 is a ladder diagram illustrating data flow within a watermark-based network topology in accordance with an embodiment of the present invention.
FIG. 4 shows a watermark-based content recognition timing according to an embodiment of the present invention.
5 is a block diagram illustrating a fingerprint-based network topology in accordance with an embodiment of the present invention.
6 is a ladder diagram illustrating data flow within a fingerprint-based network topology in accordance with an embodiment of the present invention.
FIG. 7 shows an XML schema diagram of an ACR-Resulttype that contains a query result according to an embodiment of the present invention.
Figure 8 is a block diagram illustrating a watermark and fingerprint-based network topology in accordance with an embodiment of the present invention.
9 is a ladder diagram illustrating data flow within a watermark and fingerprint-based network topology in accordance with an embodiment of the present invention.
10 is a block diagram of an image display apparatus according to an embodiment of the present invention.
11 is a diagram illustrating a digital broadcasting system according to an embodiment of the present invention.
12 is a block diagram showing the configuration of the PDI interface.
13 is a block diagram illustrating a configuration of a personalization service according to an embodiment of the present invention.
FIG. 14 is a flowchart showing a process in which the image display apparatus 100 acquires PDI data from a user.
15 is a diagram showing an XML schema table of a PDI questionnaire.
16 is a block diagram illustrating a configuration of a content providing service for each user according to an embodiment of the present invention.
17 is a flowchart illustrating a personalization service providing process based on user-specific PDI data according to an embodiment of the present invention.
18 is a block diagram illustrating a structure of a fingerprint-based image display apparatus according to another embodiment of the present invention.
19 is a block diagram illustrating a structure of a watermark-based image display apparatus according to another embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffixes "engine", "module", and "portion" for components used in the following description are given or mixed in consideration of ease of specification only and do not have their own distinct meanings or roles .

Next, a network topology according to an embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG.

1 is a block diagram illustrating a network topology according to an embodiment of the present invention.

1, a network topology according to an exemplary embodiment of the present invention includes a content providing server 10, a content recognition service providing server 20, a multi-channel video distribution server 30, an additional service information providing server 40, a plurality of additional service providing servers 50, a broadcast receiving apparatus 60, a network 70, and an image display apparatus 100.

The content providing server 10 may correspond to a broadcasting station and broadcast a broadcasting signal including main audio-visual content. The broadcast signal may further include an additional service. The supplementary service may or may not be associated with the main audiovisual content. The supplementary service may include service information, metadata, additional data, a compiled executable file, a web application, an HTML (Hypertext Markup Language) document, an XML document, a cascading style sheet File, an ATSC 2.0 content, an address such as a URL (Uniform Resource Locator), and the like. There may be one or more content providing servers.

The content recognition service providing server 20 provides a content recognition service that enables the video display device 100 to recognize the content based on the main audiovisual contents. The content recognition service providing server 20 may or may not modify the main audiovisual contents. There may be one or more content recognition service providing servers.

The content recognition service providing server 20 may be a watermark server that inserts a visible watermark such as a logo on the main audiovisual contents by transforming the main audiovisual contents. The watermark server may watermark the content provider's logo in the upper left or upper right of each frame of the main audiovisual content.

In addition, the content recognition service providing server 20 may be a watermark server for inserting the content information into the main audiovisual contents as an invisible watermark by transforming the main audiovisual contents.

The content recognition service providing server 20 may be a fingerprint server for extracting and storing feature information from a part of frames or a part of audio samples of the main audiovisual contents. This feature information is also called a signature.

The multi-channel video distribution server 30 receives broadcast signals from a plurality of broadcasting stations and multiplexes them, and provides the multiplexed signals to the broadcast receiving apparatus 60. In particular, the multi-channel video distribution server 30 performs demodulation and channel decoding on the received broadcast signal to extract main audio-visual content and supplementary services, and then channel-encodes the extracted main audio-visual content and the extracted supplementary service It is possible to generate a multiplexed signal for distribution. At this time, the multi-channel video distribution server 30 may exclude the extracted supplementary service and may add another supplementary service, so that the broadcasting station can not provide the service driven by the broadcasting station. There may be one or more multi-channel video distribution servers.

The broadcast receiving apparatus 60 tunes the channel selected by the user, receives the signal of the tuned channel, performs demodulation and channel decoding on the received signal, and extracts the main audiovisual contents. Then, the broadcast receiving apparatus 60 transmits the extracted main audio-visual contents to H.264 / MPEG-4 AVC (Moving Picture Experts Group-4 advanced video coding), Dolby AC-3, MPEG- Audio Coding) algorithm to generate uncompressed main AV content. The broadcast receiving apparatus 60 provides the generated uncompressed main audiovisual contents to the image display apparatus 100 through an external input port of the image display apparatus 100 or the like.

The additional service information providing server 40 provides additional service information for one or more available additional services related to the main audiovisual contents in response to a request of the video display device. There may be one or more supplementary service address providing servers. The additional service information providing server 40 may provide additional service information for the additional service having the highest priority among a plurality of available additional services.

The additional service providing server 50 provides one or more additional services that can be used in association with the main audiovisual contents in response to a request of the video display device. There may be one or more additional service providing servers.

The image display apparatus 100 may be a device including a display unit such as a television, a notebook, a mobile phone, a smart phone, or the like. The video display apparatus 100 may receive the uncompressed main audiovisual content from the broadcast receiving apparatus 60 or may transmit the main audiovisual content encoded from the content providing server 10 or the multi- Signal may be received. The video display apparatus 100 can receive the content recognition service from the content recognition service providing server 20 through the network 70 and can receive the main audio-visual content and the audio-visual content from the additional service information providing server 40 through the network 70 And receive one or more additional services that can be used in connection with the main audiovisual contents from the additional service providing server 50. [

At least two of the content providing server 10, the content recognition service providing server 20, the multi-channel video distribution server 30, the supplementary service information providing server 40, and the plurality of additional service providing servers 50, Or may be operated by a single operator.

2 is a block diagram illustrating a watermark-based network topology in accordance with an embodiment of the present invention.

As shown in FIG. 2, the network topology according to an embodiment of the present invention further includes a watermark server 21.

The watermark server 21 as shown in FIG. 2 inserts content information into the main audiovisual contents by applying a transformation to the main audiovisual contents. The multi-channel video distribution server 30 receives and distributes the broadcasting signal including the modified main audio-visual contents. In particular, the watermark server can use the digital watermarking technique as described below.

A digital watermark is a process of inserting information into a digital signal in a way that is difficult to delete. For example, the digital signal may be audio, picture, or video. When this digital signal is copied, the inserted information is also copied. One digital signal can carry several different watermarks at the same time.

In visible watermarking, the information to be inserted is visually identifiable in the picture or video. Typically, the embedded information is a text or logo that identifies the owner of the media. If a television station adds its logo to the corner of the video being transmitted, this is a visually identifiable watermark.

In invisible watermarking, information is added to audio, pictures, or video as digital data, but the fact that a certain amount of information is hidden is not recognizable, although it can be detected. Secret messages can be conveyed through this visually unidentifiable watermarking.

One application of watermarking is in a copyright protection system to prevent illegal copying of digital media. For example, the cloning device may obtain a watermark from digital media prior to cloning of the digital media, and may determine whether to clone based on the content of the watermark.

Another application of watermarking is to track the origin of digital media. At each point on the distribution path, a watermark is embedded in the digital media. If such a digital media is found later, a watermark can be extracted from the digital media and the source of the distribution can be grasped from the contents of the watermark.

The description of digital media is another application of watermarking which is not visually recognizable.

The file format for digital media may contain additional information called metadata, which is distinguished from metadata in that it is delivered to the audiovisual signal itself of the digital media.

Watermarking methods include spread spectrum, quantization, and amplified modulation.

If the signal to be marked is obtained by further modification, the watermarking method corresponds to a spread spectrum. Although spread spectrum watermarks are known to be fairly robust, they do not provide much information because they interfere with the host signal on which the watermark is embedded.

If the signal to be marked is obtained by quantization, the watermarking method corresponds to the quantization type. The quantization watermark is low in robustness, but it can hold quite a lot of information.

If the signal to be marked is obtained in the spatial domain by an additional correction method similar to the spread spectrum, the watermarking method corresponds to an amplified modulation.

3 is a ladder diagram illustrating data flow within a watermark-based network topology in accordance with an embodiment of the present invention.

First, the content providing server 10 transmits a broadcast signal including main audio-visual contents and an additional service (S101).

The watermark server 21 receives a broadcast signal provided by the content providing server 10, inserts a visible watermark such as a logo on the main audiovisual contents by applying a transformation to the main audiovisual contents, (Watermark information) as an invisible watermark, and provides watermarked main audiovisual contents and an additional service to the MVPD 30 (S103).

The watermark information inserted through the invisible watermark may include one or more of watermark usage, content information, supplementary service information, and available supplementary services. The watermark application may indicate one of prevention of piracy, investigation of rating, and acquisition of supplementary service.

The content information includes identification information of a content provider providing main audiovisual contents, main audiovisual content identification information, main audiovisual content rating information, time information of a content section used for acquiring content information, The logo of the channel on which the audiovisual contents are broadcast, the description of the channel in which the main audiovisual contents are broadcast, the utilization information reporting address, the utilization information reporting period, the minimum usage time for acquiring usage information, And may include one or more.

If the image display apparatus 100 uses a watermark to acquire content information, the time information of the content section used for acquiring the content information may be the time information of the content section in which the watermark is used. If the image display apparatus 100 uses the fingerprint to acquire the content information, the time information of the content section used for obtaining the content information may be the time information of the content section from which the feature information is extracted. The time information of the content section used for acquiring the content information includes a start time of the content section used for acquiring the content information, a duration of the content section used for acquiring the content information, an end time of the content section used for acquiring the content information ≪ / RTI >

The usage information reporting address may include one or more of the main audio-visual content viewing information reporting address and the supplementary service use information reporting address. The usage information reporting period may include at least one of a main audiovisual content viewing information reporting period and a supplementary service use information reporting period. The minimum usage time for acquiring usage information may include at least one of a minimum viewing time for acquiring main audio-visual content viewing information and a minimum usage time for extracting additional service usage information.

The video display apparatus 100 acquires the viewing information of the main audiovisual content based on the case where the main audiovisual content is viewed for a minimum viewing time or more and displays the viewing information extracted from the main audiovisual content viewing information reporting address in the main audiovisual content viewing information reporting period Can be reported.

The video display apparatus 100 may acquire the additional service utilization information based on the use of the additional service for the minimum usage time and may report the utilization information extracted by the additional service utilization information reporting period in the additional service utilization information reporting cycle .

The additional service information includes information on whether an additional service exists, an address of an additional service address server, an acquisition path of each available additional service, an address for each available additional service, a start time of each available additional service, The end time of each available supplementary service, the lifetime of each available supplementary service, the acquisition mode of each available supplementary service, the request period for each available supplementary service, Priority information, description of each available supplementary service, category of each available supplementary service, utilization information reporting address, utilization information reporting period, and minimum usage time for acquiring utilization information have.

The acquisition path of available supplementary services may represent an IP or Advanced Television Systems Committee (ATSC M / H). If the acquisition path of the available supplementary service is ATSC M / H, the supplementary service information may further include frequency information and channel information. The acquisition mode of each available supplementary service may indicate Push or Pull.

On the other hand, the watermark server 21 can embed watermark information as a watermark (invisible watermark) in the logo of the main audiovisual contents.

For example, the watermark server 21 can insert a barcode at a predetermined position of the logo. At this time, the predetermined position of the logo may correspond to the lower one line of the area where the logo is displayed. The image display apparatus 100 may not display the bar code when receiving the main audiovisual contents including the logo having the bar code inserted therein.

In addition, the watermark server 21 can embed watermark information in the form of a logo's metadata. At this time, the shape of the logo can be maintained.

In addition, the watermark server 21 can insert N-bit watermark information into each of the logos of M frames. That is, the watermark server 21 can insert M * N pieces of watermark information through M frames.

The MVPD 30 receives the broadcast signal including watermarked main audiovisual contents and an additional service, generates a multiplex signal, and provides the multiplex signal to the broadcast receiver 60 (S105). At this time, the multiplexed signal may exclude the received additional service or may include a new additional service.

The broadcast receiving apparatus 60 tunes a channel selected by the user, receives a signal of a tuned channel, demodulates the received broadcast signal, channel-decodes the received broadcast signal, and performs AV decoding to obtain an uncompressed main audiovisual After generating the content, the generated uncompressed main audio-visual content is provided to the image display device 100 (S106).

Meanwhile, the content providing server 10 also broadcasts a broadcasting signal including the main audiovisual contents through a wireless channel or the like (S107).

In addition, the MVPD 30 may directly transmit the broadcast signal including the main audiovisual contents to the video display device 100 without going through the broadcast receiving device 60 (S108).

The video display apparatus 100 can receive uncompressed main audio-visual contents through the set-top box 60. [ Alternatively, the video display apparatus 100 may receive a broadcast signal through a wireless channel, demodulate and decode the received broadcast signal, and obtain main audio-visual contents. Alternatively, the video display device 100 may receive a broadcast signal from the MVPD 30, demodulate and decode the received broadcast signal, and receive main audio-visual content. The video display device 100 extracts watermark information from audio samples of some frames or partial intervals of the acquired main audiovisual contents. If the watermark information corresponds to the logo, the image display apparatus 100 confirms the watermark server address corresponding to the logo extracted from the correspondence between the plurality of logos and the plurality of watermark server addresses. When the watermark information corresponds to the logo, the video display device 100 can not identify the main audiovisual contents with only the logo. Also, even when the watermark information does not include the content information, the video display device 100 can not identify the main audiovisual content, but the watermark information may include the content provider identification information or the watermark server address. When the watermark information includes the content provider identification information, the video display device 100 displays the watermark server address corresponding to the content provider identification information extracted from the correspondence between the plurality of content provider identification information and the plurality of watermark server addresses . In this manner, when the video display apparatus 100 can not identify the main audiovisual contents by only the watermark information, it connects to the watermark server 21 corresponding to the acquired watermark server address and transmits the first query ( S109).

The watermark server 21 provides a first response to the first query (S111). The first response may include one or more of content information, supplementary service information, and available supplementary services.

If the watermark information and the first response do not include the additional service address, the video display device 100 can not obtain the additional service. However, the watermark information and the first response may include an additional service address providing server address. As described above, if the image display apparatus 100 fails to acquire the supplementary service address or the supplementary service through the watermark information and the first response and obtains the supplementary service address providing server address, the image display apparatus 100 displays And then transmits the second query including the content information to the additional service information providing server 40 corresponding to the service address providing server address (S119).

The additional service information providing server 40 searches for one or more available additional services associated with the content information of the second query. Thereafter, the supplementary service information providing server 40 provides the supplementary service information for the one or more available supplementary services to the image display device 100 in the second response to the second query (S121).

If the image display apparatus 100 acquires one or more available additional service addresses through the watermark information, the first response, or the second response, the image display apparatus 100 accesses the one or more available additional service addresses to request additional services (S123) , And obtains an additional service (S125).

FIG. 4 shows a watermark-based content recognition timing according to an embodiment of the present invention.

4, when the broadcast receiving apparatus 60 is turned on and the channel is tuned and the video display apparatus 100 receives the main audiovisual information of the channel tuned from the broadcast receiving apparatus 60 through the external input port 111, Upon receiving the content, the video display apparatus 100 can sense a content provider identifier (or broadcast station identifier) from the watermark of the main audiovisual content. Then, the image display apparatus 100 can detect the content information from the watermark of the main audiovisual content based on the detected content provider identifier.

At this time, as shown in FIG. 4, the detectable period of the content provider identifier and the detectable period of the content information may be different. In particular, the detectable period of the content provider identifier may be shorter than the detectable period of the content information. Accordingly, the image display apparatus 100 can have an efficient configuration for detecting only necessary information.

5 is a block diagram illustrating a fingerprint-based network topology in accordance with an embodiment of the present invention.

As shown in FIG. 5, the network topology according to an embodiment of the present invention further includes a fingerprint server 22.

The fingerprint server 22 as shown in FIG. 5 extracts and stores feature information from audio samples of some frames or portions of the main audiovisual content without modifying the main audiovisual content. When the fingerprint server 22 receives the feature information from the image display apparatus 100, the fingerprint server 22 provides the identifier and the time information of the audio-visual content corresponding to the received feature information.

6 is a ladder diagram illustrating data flow within a fingerprint-based network topology in accordance with an embodiment of the present invention.

First, the content providing server 10 transmits a broadcasting signal including main audio-visual contents and an additional service (S201).

The fingerprint server 22 receives a broadcast signal provided by the content providing server 10, extracts a plurality of feature information from a plurality of frame sections or a plurality of audio sections of the main audiovisual contents, And builds a database for a plurality of corresponding query results (S203). The query result may include one or more of content information, supplementary service information, and available supplementary services.

The MVPD 30 receives a broadcast signal including main audio-visual content and an additional service, generates a multiplex signal, and provides the multiplex signal to the broadcast receiver 60 (S205). At this time, the multiplexed signal may exclude the received additional service or may include a new additional service.

The broadcast receiving apparatus 60 tunes a channel selected by the user, receives a signal of a tuned channel, demodulates the received broadcast signal, channel-decodes the received broadcast signal, and performs AV decoding to obtain an uncompressed main audiovisual After generating the content, the generated uncompressed main audio-visual content is provided to the video display device 100 (S206).

Meanwhile, the content providing server 10 also broadcasts a broadcasting signal including main audiovisual contents through a wireless channel (S207).

In addition, the MVPD 30 may transmit the signal including the main audiovisual contents directly to the video display apparatus 100 without passing through the broadcast receiving apparatus 60 (S208).

The video display apparatus 100 can receive uncompressed main audio-visual contents through the set-top box 60. [ Alternatively, the video display apparatus 100 may receive a broadcast signal through a wireless channel, demodulate and decode the received broadcast signal, and obtain main audio-visual contents. Alternatively, the video display device 100 may receive a broadcast signal from the MVPD 30, demodulate and decode the received broadcast signal, and receive main audio-visual content. The image display apparatus 100 extracts the feature information from the audio samples of some frame or partial section of the acquired main audiovisual contents (S213).

The image display apparatus 100 accesses the fingerprint server 22 corresponding to the preset fingerprint server address and transmits the first query including the extracted feature information (S215).

The fingerprint server 22 provides the query result as the first response to the first query (S217). If the first response is a failure, the image display apparatus 100 may connect to the fingerprint server 22 corresponding to another fingerprint server address and transmit the first query including the extracted feature information.

The fingerprint server 22 may provide an Extensible Markup Language (XML) document as a query result. An example of an XML document containing the query result will be described with reference to FIG. 7 and Table 1. FIG.

FIG. 7 shows an XML schema diagram of an ACR-Resulttype that contains a query result according to an embodiment of the present invention.

As shown in FIG. 7, the ACR-Resulttype that contains the query result has the ResultCode attribute and the ContentID, NTPTimestamp, SignalingChannelInformation, and ServiceInformation elements.

For example, if the ResultCode attribute has a value of 200, this could mean that the query result is success. If the ResultCode attribute has a value of 404, this may mean that the query result is failure.

The SignalingChannelInformation element has a SignalingChannelURL element, and the SignalingChannelURL element has an UpdateMode and PollingCycle attribute. The UpdateMode property may have a Pull value or a Push value.

The ServiceInformation element has ServiceName, ServiceLogo, and ServiceDescription elements.

Table 1 shows the XML Schema of ACR-ResultType which contains the result of such query.

(Table 1)

As the ContentID element, an ATSC content identifier (ATSC content identifier) as shown in Table 2 below may be used.

(Table 2)

As shown in Table 2, the ATSC content identifier has a structure consisting of a TSID and a house number.

The 16-bit unsigned integer TSID carries a transport stream identifier.

The 5-bit unsigned integer end_of_day is set to the hour of the day when the broadcast ends and the content_id value can be reused.

The 9-bit unsigned integer unique_for is set to the number of days in which the content_id value can not be reused (the number of days).

The content_id indicates a content identifier. The image display apparatus 100 may reduce unique_for by 1 every time it corresponds to end_of_day every day, and may consider that content_id is unique if unique_for is not 0.

On the other hand, a Global Service Identifier for the ATSC-M / H service as described below may be used as the ContentID element.

The global service identifier has the following form:

- urn: oma: bcast: iauth: atsc: service: <region>: <xsid>: <serviceid>

Where <region> is the two-letter international country code as specified by ISO 639-2. The <xsid> for the local service is the decimal number of the TSID as defined in <region>, and the <xsid> for the regional service (major> 69) is "0". <serviceid> is defined as <major> or <minor>. <major> represents a major channel number, and <minor> represents a minor channel number.

Examples of global service identifiers are:

- urn: oma: bcast: iauth: atsc: service: us: 1234: 5.1

- urn: oma: bcast: iauth: atsc: service: us: 0: 100.200

On the other hand, as the ContentID element, an ATSC content identifier as described below can be used.

The ATSC content identifier has the following form.

urn: oma: bcast: iauth: atsc: content: <region>: <xsidz>: <contentid>: <unique_for>: <end_of_day>

Where <region> is the two-letter international country code as specified by ISO 639-2. <Xsid> for the local service is the decimal number of the TSID as defined in <region>, followed by "." <Serviceid>. The <xsid> for the regional service (major> 69) is <serviceid>. <content_id> is the base64 code of the content_id field defined in Table 2, <unique_for> is the decimal sign of the unique_for field defined in Table 2, and <end_of_day> is the decimal sign of the end_of_day field defined in Table 2 .

Hereinafter, Fig. 6 will be described again.

If the query result does not include the supplementary service address or the supplementary service but includes the supplementary service address providing server address, the image display apparatus 100 displays the supplementary service information providing server 40 corresponding to the obtained supplementary service address providing server address, And transmits a second query including content information (S219).

The additional service information providing server 40 searches for one or more available additional services associated with the content information of the second query. Then, the supplementary service information providing server 40 provides the supplementary service information for the one or more available supplementary services to the image display device 100 in the second response to the second query (S221).

If the image display apparatus 100 obtains one or more available additional service addresses through the first response or the second response, the image display apparatus 100 accesses the one or more available additional service addresses to request additional services (S223) (S225).

When the UpdateMode property has a Pull value, the image display apparatus 100 transmits an HTTP request to the supplementary service providing server 50 through the SignalingChannelURL, and transmits an HTTP response including the PSIP binary stream to the supplementary service providing server 50 50). In this case, the image display apparatus 100 can transmit an HTTP request according to the polling cycle designated by the PollingCycle attribute. In addition, the SignalingChannelURL element may have an update time attribute. In this case, the video display device 100 can transmit an HTTP request at the update time designated by the update time attribute.

When the UpdateMode property has a push value, the video display device 100 can receive the update from the server asynchronously using the XMLHTTPRequest API. When the video display apparatus 100 makes an asynchronous request to the server through the XMLHTTPRequest object, the server provides the signaling information as a response through the channel when there is a change in the signaling information. If there is a restriction on the waiting time of the session, the session timeout respond is generated, and the receiver immediately recognizes it and can re-request to keep the signaling channel between the receiver and the server at all times.

Figure 8 is a block diagram illustrating a watermark and fingerprint-based network topology in accordance with an embodiment of the present invention.

As shown in FIG. 8, the network topology according to an embodiment of the present invention further includes a watermark server 21 and a fingerprint server 22.

The watermark server 21 as shown in FIG. 8 inserts the content provider identification information into the main audiovisual contents. The watermark server 21 may embed the content provider identification information as a watermark that looks like a logo in the main audiovisual content or may insert the content provider identification information in the main audiovisual content as an invisible watermark.

The fingerprint server 22 extracts feature information from audio samples of some frames or portions of the main audiovisual content and does not modify the main audio-visual content. When the fingerprint server 22 receives the feature information from the image display apparatus 100, the fingerprint server 22 provides the identifier and the time information of the audio-visual content corresponding to the received feature information.

9 is a ladder diagram illustrating data flow within a watermark and fingerprint-based network topology in accordance with an embodiment of the present invention.

First, the content providing server 10 transmits a broadcast signal including main audio-visual contents and an additional service (S301).

The watermark server 21 receives a broadcast signal provided by the content providing server 10, inserts a visible watermark such as a logo on the main audiovisual contents by applying a transformation to the main audiovisual contents, Watermark information as an invisible watermark, and provides watermarked main audiovisual contents and an additional service to the MVPD 30 (S303). Watermark information inserted through an invisible watermark may include at least one of content information, supplementary service information, and available supplementary services. The content information and the supplementary service information are as described above.

The MVPD 30 receives the broadcast signal including watermarked main audiovisual contents and an additional service, generates a multiplex signal, and provides the multiplex signal to the broadcast receiver 60 (S305). At this time, the multiplexed signal may exclude the received additional service or may include a new additional service.

The broadcast receiving apparatus 60 tunes a channel selected by the user, receives a signal of a tuned channel, demodulates the received broadcast signal, channel-decodes the received broadcast signal, and performs AV decoding to obtain an uncompressed main audiovisual After generating the content, the generated uncompressed main audio-visual content is provided to the image display apparatus 100 (S306).

Meanwhile, the content providing server 10 also broadcasts a broadcasting signal including the main audiovisual contents through a wireless channel or the like (S307).

Also, the MVPD 30 may transmit the signal including the main audiovisual contents directly to the video display device 100 without going through the broadcast receiving device 60 (S308).

The video display apparatus 100 can receive uncompressed main audio-visual contents through the set-top box 60. [ Alternatively, the video display apparatus 100 may receive a broadcast signal through a wireless channel, demodulate and decode the received broadcast signal, and obtain main audio-visual contents. Alternatively, the video display device 100 may receive a broadcast signal from the MVPD 30, demodulate and decode the received broadcast signal, and receive main audio-visual content. The video display device 100 extracts watermark information from audio samples of some frames or partial intervals of the acquired main audiovisual contents. If the watermark information corresponds to the logo, the image display apparatus 100 confirms the watermark server address corresponding to the logo extracted from the correspondence between the plurality of logos and the plurality of watermark server addresses. When the watermark information corresponds to the logo, the video display device 100 can not identify the main audiovisual contents with only the logo. Also, even when the watermark information does not include the content information, the video display device 100 can not identify the main audiovisual content, but the watermark information may include the content provider identification information or the watermark server address. When the watermark information includes the content provider identification information, the video display device 100 displays the watermark server address corresponding to the content provider identification information extracted from the correspondence between the plurality of content provider identification information and the plurality of watermark server addresses . In this manner, when the video display apparatus 100 can not identify the main audiovisual contents by only the watermark information, it connects to the watermark server 21 corresponding to the acquired watermark server address and transmits the first query ( S309).

The watermark server 21 provides a first response to the first query (S311). The first response may include one or more of a fingerprint server address, content information, supplementary service information, and available supplementary services. The content information and the supplementary service information are as described above.

If the watermark information and the first response include the fingerprint server address, the image display apparatus 100 extracts the feature information from the audio samples of some frames or some intervals of the main audiovisual contents (S313).

The image display apparatus 100 accesses the fingerprint server 22 corresponding to the fingerprint server address in the first response and transmits a second query including the extracted feature information (S315).

The fingerprint server 22 provides a query result as a second response to the second query (S317).

If the query result does not include the supplementary service address or the supplementary service but includes the supplementary service address providing server address, the image display apparatus 100 displays the supplementary service information providing server 40 corresponding to the obtained supplementary service address providing server address, And transmits a third query including content information (S319).

The additional service information providing server 40 searches for one or more available additional services associated with the content information of the third query. Thereafter, the supplementary service information providing server 40 provides supplementary service information for the one or more available supplementary services to the image display device 100 in a third response to the third query (S321).

If the image display apparatus 100 obtains one or more available additional service addresses through the first response, the second response, or the third response, the image display apparatus 100 accesses the one or more available additional service addresses to request additional services (S323 ), And obtains additional services (S325).

Next, a video display device 100 according to an embodiment of the present invention will be described with reference to FIG.

10 is a block diagram of an image display apparatus according to an embodiment of the present invention.

10, an image display apparatus 100 according to an embodiment of the present invention includes a broadcast signal receiving unit 101, a demodulating unit 103, a channel decoding unit 105, a demultiplexing unit 107, And includes a decoding unit 109, an external input port 111, a reproduction control unit 113, a reproduction device 120, an additional service management unit 130, a data transmission / reception unit 141, and a memory 150.

The broadcast signal receiving unit 101 receives a broadcast signal from the content providing server 10 or the MVPD 30.

The demodulator 103 demodulates the received broadcast signal to generate a demodulated signal.

The channel decoding unit 105 performs channel decoding on the demodulated signal to generate channel decoded data.

The demultiplexer 107 demultiplexes the main audio-visual content and the supplementary service from the channel-decoded data. The separated supplementary service is stored in the supplementary service storage unit 152.

The audiovisual decoding unit 109 performs AV decoding on the separated main audio-visual contents to generate uncompressed main audio-visual contents.

The external input port 111 receives uncompressed main audiovisual contents from the broadcast receiving apparatus 60, a digital versatile disk (DVD) player, a Blu-ray disc player, and the like. The external input port 111 may include one or more of a DSUB port, a High Definition Multimedia Interface (HDMI) port, a DVI (Digital Visual Interface) port, a composite port, a component port, have.

The playback control unit 113 plays back at least one of the uncompressed main audio-visual content generated by the audio-visual decoding unit 109 or the uncompressed main audio-visual content received from the external input port 111 to the playback apparatus 120 do.

The playback apparatus 120 includes a display unit 121 and a speaker 123. The display unit 121 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) display, and a 3D display.

The supplementary service management unit 130 acquires the content information of the main audiovisual contents and acquires the available supplementary services based on the acquired content information. In particular, as described above, the supplementary service management unit 130 may acquire identification information of the main audiovisual contents based on audio samples of some frames or partial intervals of the uncompressed main audiovisual contents. In this specification, automatic contents recognition (ACR).

The data transmission / reception unit 141 may include an ATSC-M / H (Advanced Television Systems Committee-Mobile / Handheld) channel transmission / reception unit 141a and an IP transmission / reception unit 141b.

The memory 150 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The image display apparatus 100 may operate in association with a web storage that performs a storage function of the memory 150 on the Internet.

The memory 150 includes a content information storage unit 151, an additional service storage unit 152, a logo storage unit 153, a setting information storage unit 154, a bookmark storage unit 155, a user information storage unit 214, A usage information storage unit 156, and a usage information storage unit 157.

The content information storage unit 151 stores a plurality of pieces of content information corresponding to a plurality of pieces of feature information.

The additional service storage unit 152 may store a plurality of additional services corresponding to the plurality of pieces of feature information or may store a plurality of additional services corresponding to the plurality of pieces of content information.

The logo storage unit 153 stores a plurality of logos. The logo storage unit may further store a content provider identifier corresponding to the plurality of logos or a watermark server address corresponding to the plurality of logos.

The setting information storage unit 154 stores setting information for ACR.

The bookmark storage unit 155 stores a bookmark.

The user information storage units 214 and 156 store user information. The user information may include one or more of at least one of account information for one or more services, local information, family member information, preferred genre information, image display device information, and usage information providing range. The one or more account information may include account information for a usage information measurement server, account information of a social network service such as a twitter, and a facebook. Local information may include address information and zip code. The family member information may include the number of family members, the age of each member, the sex of each member, the religion of each member, and the occupation of each member. The preferred genre information can be set to one or more of sports, movies, dramas, education, news, entertainment, and other genres. The video display device information may include information on the type of video display device, manufacturer, firmware version, resolution, model name, OS, browser, presence of storage device, capacity of storage device, and network speed. When the use information providing range is set, the video display device 100 can collect and report the main audiovisual content viewing information and the additional service use information within the set range. The utilization information providing range can be set for each virtual channel. In addition, the usage information measurement permissible range may be set for the entire physical channel.

The usage information storage unit 157 stores main audiovisual content viewing information and supplementary service usage information collected by the video display device 100. [ Also, the video display device 100 may analyze the service usage pattern based on the collected main audio-visual content viewing information and the collected additional service usage information, and store the analyzed service usage pattern in the usage information storage unit 157 .

The supplementary service management unit 130 may acquire the content information of the main audiovisual contents from the fingerprint server 22 or the content information storage unit 151. [ If there is no content information corresponding to the extracted feature information in the content information storage unit 151 or there is not sufficient content information, the supplementary service management unit 130 can receive the additional content information through the data transmission / reception unit 141. [ Also, the supplementary service management unit 130 may continuously update the content information.

The supplementary service management unit 130 can acquire the supplementary services available from the supplementary service provision server 50 or the supplementary service storage unit 153. [ If there is no supplementary service or there is not enough supplementary service in the supplementary service storage unit 153, the supplementary service management unit 130 can update the supplementary service through the data transmission / In addition, the supplementary service management unit 130 may continuously update the supplementary service.

The supplementary service management unit 130 may extract the logo from the main audiovisual contents and acquire the content provider identifier or the watermark server address corresponding to the extracted logo by querying the logo storage unit 155. If there is no logo matching the logo extracted in the logo storing unit 155 or there is not enough logo, the supplementary service managing unit 130 can receive the additional logo through the data transmitting and receiving unit 141. In addition, the supplementary service management unit 130 may continuously update the logo.

The supplementary service management unit 130 may perform various methods for reducing the burden of the computation in performing the comparison between the logo extracted from the main audiovisual contents and the plurality of logos in the logo storage unit 155. [

For example, the supplementary service management unit 130 may perform comparison based on color characteristics. That is, the additional service management unit 130 may compare the color characteristics of the extracted logo and the color characteristics of the logo in the logo storage unit 155 to determine whether or not they match.

The supplementary service managing unit 130 can perform comparison based on character recognition. That is, the supplementary service management unit 130 can compare the characters recognized from the extracted logo with the characters recognized from the logo in the logo storage unit 155 to determine whether or not they match.

In addition, the supplementary service managing unit 130 can perform the comparison based on the shape of the outline of the logo. That is, the supplementary service management unit 130 may compare the outline shape of the extracted logo with the outline shape of the logo in the logo storage unit 155 to determine whether or not they match.

11 is a diagram illustrating a digital broadcasting system according to an embodiment of the present invention.

Specifically, a personalized broadcast system including a digital broadcast receiver for a personalized service is shown. According to an embodiment of the present invention, the user information includes, for example, a user profile, demographics and interests information. Hereinafter, each configuration of the personalized broadcasting system will be described.

User profiles, statistics and interests are expressed through responses to the questionnaire. The data structure summarizing the questionnaires and responses provided by a particular user is called the PDI Questionnaire or PDI Table. In the PDI table, the part corresponding to the question is called the PDI Question or PDI-Q, and the part corresponding to the response is called the PDI-A. The set of filter criteria is referred to as PDI-FC.

A receiver available in ATSC 2.0 / ATSC 3.0 includes the capability to generate a response (PDI-A instance) corresponding to the question in the questionnaire. The PDI-A generation function is used as an input to the PDI-Q instance. The receiver also provides a filter function that compares PDI-A to PDI-FC to determine content suitable for download and use.

As shown in FIG. 11, the personalized broadcasting system includes a content provider 300) and a receiver 200. According to an embodiment of the present invention, a receiver 200 includes a PDI engine 203, a filtering engine 201, a PDI storage 208, a content storage 207, a content module 206, interface module 209, as shown in FIG.

The content provider 300 according to an embodiment of the present invention may transmit the content, the PDI questionnaire 301 and the filtering criteria 202 to the receiver 200. [ A data structure summarizing the questionnaire and the response by a particular user may be referred to as a PDI Questionnaire (301). According to one embodiment of the present invention, the PDI questionnaire 301 may include information about the user's profile, statistics and interests.

The receiver 200 may process the content received from the content provider 300, the PDI questionnaire 301 and the filter criteria 202. Hereinafter, the operation of the modules included in the receiver 200 will be described with reference to the digital broadcasting system.

The PDI engine 203 may receive the PDI questionnaire 301 from the content provider 300 according to one embodiment of the present invention. The PDI engine 203 may send a PDI query including the received PDI questionnaire 301 to the UI module 209. [ When there is user input corresponding to the PDI query, the PDI engine 203 may receive the user response and other information. The PDI engine 203 may then process the PDI query and PDI response to support the personalization service through generation of the PDI data 204. The PDI data 204 may include the PDI query and PDI response described above. Thus, the PDI response to the PDI questionnaire (301) may indicate the user profile, statistics and interests.

Additionally, the PDI engine 203 may be updated with the PDI data 204 that received the PDI response. Specifically, the PDI engine 203 may delete, add, and modify the PDI data 204 using the ID of the PDI response. In addition, when another module requests the PDI engine 203 to transmit the PDI data 204, the PDI engine 203 may send the PDI data 204 to the module.

The filtering engine 201 according to an embodiment of the present invention may filter the content according to the PDI data 204 and the filtering criteria 202. [ Filtering criterion 202 refers to a set of filtering criteria for filtering only content appropriate to a user via PDI data 204. Specifically, the filtering engine 201 may receive the PDI data 204 from the PDI engine 203 and receive the content and filter criteria 202 from the content provider 300.

In addition, the content provider 300 may transmit the filtering criteria 202 associated with the content when transmitting the parameters related to the declarative content (hereafter, the content). The filtering engine 201 then compares the filtering criteria 202 with the PDI data 204 and filters and downloads the content based on the results of the comparison. The downloaded content may be stored in the content storage unit 207. [

The UI module 209 displays the PDI questionnaire received from the PDI engine 203 and the PDI response received in response to the PDI query from the user. The user may transmit a response to the PDI query displayed on the receiver 200 via the remote control device. The UI module 209 may send a PDI response to the PDI engine 203.

The content module 206 may access the PDI engine 203 to obtain the PDI data 204. As shown in Fig. 11, the content module 206 may receive the content provided from the content provider 300. [ The content may be content associated with an application executed by the receiver 200, and may include a declarative object, for example, a triggered declarative object (TDO).

Although not shown in FIG. 11, the content module 206 may access the PDI storage 208 to obtain a PDI query and a PDI response. In this case, the content module 206 may use an API (Application Programming Interface). Specifically, the content module 206 may retrieve the PDI storage 208 via the API to obtain at least one PDI query. And may transmit the PDI response to the PDI storage unit 208 through the UI module 209. [

The PDI storage unit 208 stores PDI data including a PDI query and a PDI response.

The content storage unit 207 stores the filtered content according to a filter criterion based on the PDI questionnaire.

12 is a block diagram showing the configuration of the PDI interface.

In one embodiment, the receiver 200 may be the configuration shown on the left side of FIG. The receiver 200 may include a function to create a response to a question in the questionnaire. PDI-generation function can produce PDI-Q as input and PDI-A as output. The PDI-Q and the PDI-A may be stored in the PDI storage unit 208 of the receiver 200. The receiver 200 may also compare PDI-A and PDI-FC to provide a filtering function to determine the content to use and download.

The content provider or the broadcast station 300 may include the function of distributing and holding the PDI table, as shown on the right side of Fig. Content metadata may also be produced along with the content. The metadata may include a PDI-FC based on a PDI query in the PDI table.

13 is a block diagram illustrating a configuration of a personalization service according to an embodiment of the present invention.

As shown in FIG. 13, the personalization service according to an embodiment of the present invention may include a content provider (or broadcaster) 300, a receiver 200, a second device 400, a PDI storage cloud 500 have.

The content provider 300 may transmit the content to the receiver 200 as described above and may provide the PDI questionnaire 301 for the personalization service. Also, the content provider 300 may transmit the metadata together with the main content to the receiver 200. The metadata may include additional services associated with the main content, subtitles, content titles, content thumbnails, and the like.

The receiver 200 may receive a broadcast signal from the content provider 300 and output audiovisual content. In one embodiment, the receiver 200 may output the uncompressed audio-visual content received from the set-top box.

The receiver 200 may include a filtering engine 201, a PDI engine 203, a content storage 207, a PDI storage 208, and an external device module 210. In the case of the filtering engine 201, the PDI engine 203, the content storage unit 207, and the PDI storage unit 208, the detailed description is omitted here.

The external device module 210 serves to connect the receiver 200 and the second device 400. More specifically, the receiver 200 transmits the PDI data received from the content provider 300 to the second device 400, receives the corresponding PDI data from the second device 400, PDI engine 203 or the like. The external device module 210 may be connected to the second device 400 in a wired or wireless manner.

The second device 400 may be an electronic device including a display function. The second device 400 may include at least one electronic device. May include a first external device 401, a second external device 402, and a third external device 403 as shown in FIG. The first external device 401, the second external device 402, and the third external device 403 may be separate devices, and may be separately connected to the receiver 200. The first external device 401, the second external device 402, and the third external device 403 may be different types of electronic devices. For example, the first external device 401 may be a smartphone, and the second and third external devices 402 and 403 may be a tablet PC.

The second device 400 may receive and output the PDI questionnaire from the receiver 200. [ The PDI questionnaire may be output to the receiver 200. However, if the PDI questionnaire is output to the receiver 200, the viewer may be disturbed by the viewing of the main content, Thereby outputting a PDI questionnaire so as to minimize interference with viewing of the main content.

In addition, the remote control device used with the conventional receiver 200 has a limitation on specific screen control, so that the user can respond to the PDI questionnaire through the second device 400 which is relatively easy to operate. For example, if a PDI questionnaire is displayed on a smartphone, a user can input an answer to a question item of the PDI questionnaire through a touch of a smart phone or a character input. Smartphones have a relatively easy-to-operate interface to run a variety of applications compared to video displays, which can be used for PDI data collection.

PDI data storage cloud 500 may store PDI data. Specifically, it may be a shared storage device other than the PDI storage unit 208 in the receiver 200. The receiver 200 may receive PDI data over the network via the PDI data storage cloud 500. Accordingly, the PDI data collected from the receiver other than the PDI data previously stored in the receiver 200 can also be accessed through the PDI data storage cloud 500.

FIG. 14 is a flowchart showing a process in which the image display apparatus 100 acquires PDI data from a user.

The broadcasting station or the content provider 300 delivers the PDI questionnaire to the receiver 200 together with the broadcast signal (S401). PDI questionnaires can be sent in XML schema form. A detailed PDI questionnaire table will be described with reference to FIG.

15 is a diagram showing an XML schema table of a PDI questionnaire. According to one embodiment of the present invention, a PDI questionnaire is a document that can be obtained by executing a PDI table of an XML schema. The question part in the PDI table can be informally referred to as a "PDI query" or "PDI-Q ". The answer to a given PDI question can be informally "PDI-A".

 PDI-Q and PDI-A can be obtained by executing a PDI table containing PDI-Q and PDI-A in the XML schema. The PDI-Q and PDI-A are designed to support type QIA, QBA, QSA, QTA, QAA, .

In order to distinguish characteristics and elements as shown in Fig. 15, "@" in the PDI table according to this embodiment may be displayed before the element name.

As shown in FIG. 15, the PDI user data table (hereinafter referred to as a PDI table) may include a protocolVersion element, a userDataId element, a userDataVersion element, a time element, and a user_id independently of the question type element. The PDI table may also include one or more query elements. Each of the query elements may be one of QIA, QBA, QSA, QTA and QAA. 1..N of <choice> means that each of the query elements can appear in any order.

The protocolVersion element of the PDI table can be configured in hexadecimal. The upper 4 bits indicate the major version of the protocol and the lower 4 bits indicate the minor version of the protocol. If the main version of the corresponding PDI table is set to 1, the receiver can determine that the PDI specified by the main version not supported by the receiver is to be discarded. If the minor version of the corresponding PDI table is set to 0, the receiver may determine that the PDI specified by the minor version not supported by the receiver will not be discarded. In this case, the receiver can determine that each element that the receiver does not support can be ignored.

The userDataId in the PDI table may be a globally unique identifier of the PDI user data.

The userDataVersion in the PDI table may be the version of the PDI user data. The value can have a value from 0 to 255. As userDataVersion increases by one, it can mean different users.

The time element of the PDI table represents the latest time of the change of the query in the PDI table.

The user_id element of the PDI table may be the identifier of the user of the table. The receiver 200 may identify PDI data obtained from a plurality of users via a user_id.

The QIA element represents an integer type of response to the question. The QIA element may contain content that selectively limits the minimum and maximum allowed values of the response.

The QBA element represents a Boolean response to the question.

The QSA element represents an optional response to the question. A QSA element can contain a QSA.minchoice element, and a QSA.minchoice element can specify a user-selectable minimum number. The QSA element may also contain a QSA.maxchoice element, which may specify the maximum number of user choices.

The QTA element represents a textual response to a question.

The QAA element represents an arbitrary (free form) response to the question.

Returning to FIG. 14 again.

The receiver 200 stores the PDI questionnaire 301 received from the content provider 300 (S403). The receiver 200 may store the PDI questionnaire 301 in the PDI storage unit 208. [

The receiver 200 transmits the PDI questionnaire 301 received from the content provider 300 to the second device 400 (S405). Specifically, the receiver 200 provides the PDI questionnaire 301 to the second device 400 in order to collect personal information (in particular, handicap information) of the user. As described above, when displaying on the receiver 200, interference with viewing of the main contents of the user can be minimized, and a questionnaire can be responded to via a smart phone that is relatively easy to operate.

The second device 400 displays the PDI questionnaire 301 received from the receiver 200 through the display unit or the like (S407). As described above, the second device 400 may be an electronic device including a display function, and may include various input means.

The second device 400 receives a response to the PDI questionnaire 301 displayed by the user (S409). Specifically, the second device 400 may collect the handicap information of the user through the PDI questionnaire 301.

The receiver 200 receives the PDI data including the response to the PDI questionnaire 301 collected by the second device 400 (S411). Specifically, PDI data can be received from the second device 400 through the external device module 210 in the receiver 200.

The receiver 200 searches the PDI storage unit 208 for the received PDI data (S413). The image display apparatus can search whether the same user ID is included in the PDI storage unit 208 by using the user ID (user_id) included in the PDI data. Specifically, PDI data can be retrieved through the PDI engine 203 in the receiver 200.

The receiver 200 stores or renews the PDI data based on the retrieved result (S415). In one embodiment, if the same user ID does not exist as a result of the PDI engine searching the PDI storage unit, PDI data having the corresponding user ID is newly stored in the PDI storage unit 208. Accordingly, the image display apparatus can collect handicap information of a new user and provide a personalization service based on the collected handicap information.

In another embodiment, if the same user ID exists as a result of the PDI engine 203 retrieving the PDI storage unit, the user version (userDataVersion) of the PDI data having the corresponding user ID is determined. As a result of the determination, if the newly received PDI data is more recent data, the PDI engine 203 updates the information of the corresponding user with the newly received PDI data.

Accordingly, the PDI storage unit 208 of the receiver 200 can store PDI data for a plurality of users, and can provide a personalized service using PDI data stored for each user.

16 is a block diagram illustrating a configuration of a content providing service for each user according to an embodiment of the present invention.

As shown in FIG. 16, a receiver 200 according to an exemplary embodiment of the present invention includes a conversion selector 211, a metadata parser 212, a conversion engine 213, (214) and a user management engine (215). On the other hand, the video display device of Fig. 16 includes all of the configurations shown in Fig. 13 together with the configuration shown in Fig.

The conversion selector 211 selects a format to be converted based on the PDI data received from the PDI engine 203. [ Specifically, the PDI data acquired based on the PDI questionnaire 301 is stored in the PDI storage unit 208, and the conversion selector 211 grasps the user's handicap information from the PDI data. Then, the conversion selector 211 determines the conversion type based on the handicap information of the identified user.

For example, if the current user is found to be deaf from the PDI data, the conversion selector 211 may choose to convert the audio data to be output with the main content into the form of sign language. As another example, if the current user is a hearing impaired who can not recognize the sign language from the PDI data, he can choose to convert the audio data to a subtitle.

The metadata parser 212 parses the broadcast signal received from the broadcast station or the content provider 300. Specifically, the main content and the text component can be extracted by parsing the broadcast signal received from the content provider 300. The main content may include video data and audio data, and the text component may include data describing the main content.

Conventionally, the content provider 300 has provided a broadcasting service by transmitting all the components supporting the main content to the broadcasting signal and selecting the components to be output from the receiver 200. However, according to the conventional method, the content provider 300 has a burden of preparing all the components in various cases, and has to pay a lot of money to transmit various kinds of components.

However, in the present invention, the content provider 300 transmits only one text component, and the receiver 200 converts and outputs the text component. Accordingly, the content provider 300 can transmit only one text component, thereby reducing the cost, and the user has an advantage that the user can receive the component corresponding to his / her specific specification.

The conversion engine 213 converts the text component parsed by the metadata parser 212 according to the format selected by the conversion selector 211. The type of conversion may include braille, hydration, voice and subtitles, and may include other forms of output.

The user information storage 214 stores login information of a user using the receiver 200. [ The login information is information for identifying a user, and may include a password, face information, voice information, a fingerprint, and the like.

The user management engine 215 identifies the user using the receiver 200. Specifically, the login information received from the user is matched with the data of the user information storage unit 214 to identify the user. If the user does not exist in the user information storage 214 as a result of the matching, a new user registration may be performed.

17 is a flowchart illustrating a personalization service providing process based on user-specific PDI data according to an embodiment of the present invention.

The receiver 200 receives the login input of the user currently viewing the content (S501). In order for the receiver 200 to provide a personalized service based on user-specific PDI data, the receiver 200 should be able to determine who is currently viewing the content through the receiver 200. [ Accordingly, the video display device outputs a user interface for login through the display device 121 for user identification.

In one embodiment, the receiver 200 may display a user interface for logging in after the receiver 200 is powered on. When the power of the receiver 200 is turned on, the user can determine that the user has the intention to view the content. Accordingly, the receiver 200 can output a user interface for identifying the user.

In yet another embodiment, the receiver 200 may display a user interface for logging in after receiving new PDI data. When the receiver 200 receives new PDI data through the PDI questionnaire, it can determine that the user who is the subject of the received PDI data has intention to view the content through the receiver 200 at present. Thus, the receiver 200 can output a user interface for user identification.

When the receiver 200 receives an input for login, the user ID (user_id) stored in the image display apparatus 100 is searched, and if the ID matches the search result, it is determined that the user is logged in. The user ID may be stored in the user information storage 214 in the receiver 200 and the user ID matching may be performed through the user management engine 215. [

Meanwhile, the receiver 200 receives an input for log-in and searches the user information storage 214, but there is no matching user ID. In this case, the receiver 200 may output a user interface for generating a new user ID. The receiver 200 receives user information through a user interface for creating a new user ID, and generates a user ID based on the received information.

The receiver 200 may output the user interface for user identification and the user interface for user ID generation to the second device 400. [ In this case, the user management engine 215 in the receiver 200 transmits the above-described user interfaces to the second device via the external device module 210. The second device outputs the received user interfaces and receives input for user identification or new ID generation. The user management engine 215 receives the user identification information or the new ID information collected through the external device module 210. When the information for user identification is received through the second device 400, the user can easily input the identification information because it is relatively easy to operate as compared with the remote control device of the receiver 200.

Also, in one embodiment of the present invention, the user management engine 215 may automatically identify the user, in addition to the method of identifying the user by inputting the user information to the receiver 200 or the second device 400. [ As a first example, a user identification method using face recognition can be used. As a second example, a user identification method through voice recognition can be used. As a third example, a user identification method through fingerprint recognition can be used. As a fourth example, a method of identifying a user through information of a second device 400 connected to the receiver 200 may be used.

Specifically, the face information of each user is stored in the user information storage 214, and the user management engine 215 can identify the user based on the face information collected through the camera attached to the receiver 200 . The user management engine 215 may collect face information through a camera attached to the second device 400 through the external device module 210. [ In the same way, each user's voice information can be used for user identification.

The third example will be described in more detail. The user fingerprint information is stored in the user information storage 214. The user management engine 215 may collect fingerprint information of the user through the second device 400 and identify the user based on the collected fingerprint information.

The fourth example will be described in more detail. In the case of the second device 400, most of the users have their own devices. The receiver 200 may receive the unique information of the second device 400 in the process of being connected to the second device 400. [ The user management engine 215 can search the user information storage 214 for a history of using the device based on the unique information received from the second device 400. [ The user management engine 215 may identify the user based on the retrieved result.

When the user login is completed according to the above-mentioned process, the PDI engine 203 searches PDI data of the logged-in user (S503). The PDI engine 203 searches the PDI storage unit 208 for the PDI data of the user based on the login information.

The PDI engine 203 determines whether the PDI data of the corresponding user exists based on the search result (S505). The determination of the presence of the PDI data can be made based on the user_id of the PDI data.

In one embodiment, if the PDI engine 203 determines that there is PDI data of the user logged in the PDI storage unit 208, the corresponding PDI data is loaded from the PDI storage unit 208 (S509).

In another embodiment, if the PDI engine 203 determines that there is no user PDI data logged in the PDI storage unit 208, the PDI data of the logged-in user is collected (S507). The process of collecting PDI data has been described with reference to FIG. 14, and a description thereof will be omitted here.

The receiver 200 converts the metadata based on the acquired PDI data (S511). Specifically, the conversion selector 211 acquires the retrieved PDI data or newly collected PDI data. The conversion selector 211 selects a format for converting the metadata supporting the main content based on the obtained PDI data. For example, if the PDI data indicates that the currently logged-in user is deaf, the conversion selector 211 may select braille, subtitle, or sign language to convert the metadata. The result selected by the conversion selector 211 is transferred to the conversion engine 213. The conversion engine 213 converts the component describing the main content received from the meta data parser 212 according to the selection result received from the conversion selector 211.

In one embodiment, the transform selector 211 may select one or more transform types. For example, if the PDI data indicates that the currently logged-in user is blind, the conversion selector 211 may select both voice and Braille in a format to convert the metadata.

In another embodiment, the transform selector 211 may select the transform type via the ACR function in the receiver 200. [ More specifically, the receiver 200 can recognize the currently broadcasted content through the ACR function. For example, if the user's handicap information based on the PDI data is a hearing impaired person, the conversion selector 211 can select the sign language and the caption in a conversion format. If the content recognized by the ACR function is news, the conversion selector 211 can select sign language in a conversion format due to the characteristic that the scene change is small and the speaker is fixed. However, if the content recognized through the ACR function is a drama, there may be a lot of scenes and a large number of speakers. Therefore, in this case, the conversion selector 211 can select the subtitle as a conversion type that can stably assist the currently reproduced content as compared with the sign language.

The receiver 200 transmits the converted metadata from the conversion engine 213 to the second device 400 (S513). Transmission of the converted metadata may be performed through the external device module 210. In one embodiment, the video display device may transmit the converted metadata to a plurality of second devices 400 at the same time. In another embodiment, the video display device may output the converted metadata through the display device 121 when there is no connected second device or when the metadata can not be transmitted to the second device.

The filtering engine 201, the PDI engine 203, the conversion selector 211, the conversion engine 213, and the metadata parser 212 may be included in one control unit have. In addition, the content storage unit 207, the PDI storage unit 208, and the user information storage unit 214 may all be included in one storage unit.

18 is a block diagram illustrating a structure of a fingerprint-based image display apparatus according to another embodiment of the present invention.

In FIG. 18, Tuner 501 extracts a symbol from an 8-VSB RF signal transmitted over an air channel.

The 8-VSB demodulator 503 demodulates the 8-VSB symbol extracted by the tuner 501 to recover meaningful digital data.

The VSB decoder 505 decodes the digital data restored by the 8-VSB demodulator 503 and restores the ATSC main service and the ATSC M / H service.

The MPEG-2 TP Demux 507 filters the Transport Packet to be processed by the image display apparatus 100 from the MPEG-2 Transport Packet stored in the MPEG-2 Transport Packet or PVR Storage transmitted through the 8-VSB signal, Relay module.

The PES decoder 539 buffers and restores the packetized elementary stream transmitted through the MPEG-2 Transport Stream.

The PSI / PSIP decoder 541 buffers and analyzes the PSI / PSIP Section Data transmitted through the MPEG-2 Transport Stream. The analyzed PSI / PSIP data is collected by the Service Manager (not shown) and stored in the DB in the form of Service Map and Guide data.

The DSMCC Section Buffer / Handler 511 buffers and processes the DSMCC Section Data for file transmission and IP Datagram encapsulation transmitted through the MPEG-2 TP.

The IP / UDP Datagram Buffer / Header Parser 513 encodes the IP datagram transmitted through the MPEG-2 TP encapsulated through the DSMCC Addressable section and analyzes the header of each datagram by buffering and restoring it. The IP / UDP Datagram Buffer / Header Parser 513 buffers and restores the UDP Datagram transmitted through the IP Datagram, and analyzes and processes the restored UDP Header.

The stream component handler 557 may include an ES Buffer / Handler, a PCR Handler, an STC module, a Descrambler, a CA Stream Buffer / Handler, and a Service Signaling Section Buffer / Handler.

The ES Buffer / Handler buffers and restores elementary streams such as video and audio data transmitted in PES format, and transfers them to the appropriate A / V decoder.

PCR Handler processes PCR (Program Clock Reference) data used for time synchronization of Audio and Video Stream.

The STC module performs time synchronization by correcting the clock value of the A / V decoders using the reference clock value transmitted through the PCR Handler.

If Scrambling is applied to the Payload of the received IP Datagram, the Descrambler restores the payload data using the Encryption key received from the CA Stream Handler.

The CA Stream Buffer / Handler buffers and processes data such as key values for Descrambling of EMM and ECM transmitted for Conditional Access function transmitted through MPEG-2 TS or IP Stream. The output of the CA Stream Buffer / Handler is delivered to the Descrambler, and the descrambler performs the decryption of the MPEG-2 TP or IP Datagram transmitting the A / V Data and File Data.

Service Signaling Section Buffer / Handler buffers, restores and analyzes NRT Service Signaling Channel Section Data transmitted in the form of IP Datagram. The Service Manager (not shown) collects the analyzed NRT Service Signaling Channel Section data and stores it in the DB in the form of Service Map and Guide data.

The A / V decoder 561 decodes the compressed audio / video data received through the ES handler and presents it to the user.

The MPEG-2 Service Demux (not shown) may include an MPEG-2 TP Buffer / Parser, a Descrambler, and a PVR Storage module.

The MPEG-2 TP Buffer / Parser (not shown) buffers and restores the MPEG-2 Transport Packet transmitted through the 8-VSB signal, and detects and processes the Transport Packet Header.

Descrambler restores the payload data using the Encryption key received from the CA Stream Handler for the packet payload to which Scramble is applied in the MPEG-2 TP.

The PVR Storage module stores the received MPEG-2 TP using the 8-VSB signal according to the user's request and outputs the MPEG-2 TP according to the user's request. The PVR Storage module can be controlled by the PVR Manager (not shown).

The file handler 551 may include an ALC / LCT Buffer / Parser, an FDT Handler, an XML parser, a File Reconstruction Buffer, a Decompressor, a File Decoder, and a File Storage.

ALC / LCT Buffer / Parser buffers and restores ALC / LCT data transmitted in UDP / IP Stream, and analyzes Header and Header extension of ALC / LCT. The ALC / LCT Buffer / Parser can be controlled by the NRT Service Manager (not shown).

The FDT Handler analyzes and processes the File Description Table of the FLUTE protocol transmitted through the ALC / LCT session. The FDT Handler can be controlled by the NRT Service Manager (not shown).

The XML parser analyzes the XML document transmitted through the ALC / LCT session and delivers the analyzed data to the appropriate modules such as FDT Handler and SG Handler.

The File Reconstruction Buffer restores files that are transferred to ALC / LCT and FLUTE sessions.

Decompressor performs decompression process when a file transmitted in ALC / LCT or FLUTE session is compressed.

The file decoder decodes a file decompressed from a file reconstruction buffer or decompressed from a decompressor, or a file extracted from a file storage.

File Storage saves or extracts the restored files as needed.

The M / W Engine (not shown) processes data such as files that are not A / V streams transmitted through the DSMCC Section and IP Datagram. , The M / W Engine transfers the processed data to the Presentation Manager module.

The SG Handler (not shown) collects and analyzes the Service Guide data transmitted in the form of an XML document, and transmits the collected Service Guide data to the EPG Manager.

Service Manager (not shown) collects and analyzes Service Signaling Section Data transmitted through PSI / PSIP Data and IP Stream transmitted through MPEG-2 Transport Stream, and creates Service Map. The Service Manager (not shown) stores the created service map in the Service Map & Guide Database, and controls access to the desired service by the user. (Not shown), and controls the tuner 501, the MPEG-2 TP Demux 507, the IP Datagram Buffer / Handler 513, and the like.

The NRT Service Manager (not shown) performs overall management of the NRT service transmitted in object / file form via the FLUTE session on the IP layer. The NRT Service Manager (not shown) can control FDT Handler, File Storage, and the like.

The Application Manager (not shown) performs overall management of the processing of application data transmitted in the form of an object, a file, and the like.

The UI Manager (not shown) transmits the user's input to the operation controller through the user interface and starts the operation of the process for the service requested by the user.

The Operation Controller (not shown) processes the command of the user received through the UI Manager and allows the Manager of the required module to execute the corresponding action.

Fingerprint Extractor 565 extracts fingerprint feature information from the Audio / Video stream.

The Fingerprint Comparator (567) compares the feature information extracted by the Fingerprint Extractor with the reference fingerprint to find the matching content. The Fingerprint Comparator 567 may use a reference fingerprint DB stored in a local area, or may inquire a Fingerprint query server on the Internet to receive a result. The result data that is matched by the comparison result can be transferred to the application and used.

The application 569 is an application module that provides an enhanced service based on the ACR function or an ACR function based on the ACR. The application 569 identifies the broadcast content being viewed and provides extended services associated with the broadcast content.

19 is a block diagram illustrating a structure of a watermark-based image display apparatus according to another embodiment of the present invention.

The watermark-based image display apparatus shown in FIG. 19 is similar to the fingerprint-based image display apparatus shown in FIG. 18, but includes a fingerprint extractor 565 and a fingerprint comparator 567 of a fingerprint-based image display apparatus But instead includes a Watermark Extractor (566).

The Watermark Extractor 566 extracts watermark embedded data from the Audio / Video stream. The extracted data can be transferred to the application and used.

According to an embodiment of the present invention, the above-described method can be implemented as a code readable by a processor on a medium on which a program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described image display apparatuses are not limited to the configurations and methods of the embodiments described above, but the embodiments may be modified such that all or some of the embodiments are selectively combined .

Claims (16)

The method comprising: receiving a broadcast signal from a content provider, the broadcast signal including metadata that aids main content and main content;
Receiving login information and identifying a user;
Retrieving data for a personalization service corresponding to the identified user;
Converting an output format of the metadata based on the search result; And
And outputting the converted metadata
A method of operating a video display device. The method according to claim 1,
The metadata received from the content provider has only one form,
The step of converting the output format of the metadata
Determining user's handicap information from the data for the personalization service and selecting the metadata conversion type based on the determined handicap information
A method of operating a video display device. 3. The method of claim 2,
The method of claim 1,
Recognizing the received main content, and selecting a conversion type based on the recognized genre information of the main content
A method of operating a video display device. The method according to claim 1,
The step of retrieving data for the personalization service
Collecting data for the personalization service if data matching the identified user information does not exist in the storage,
A method of operating a video display device. 5. The method of claim 4,
The step of collecting data for the personalization service
Sending a questionnaire received from a content provider to an external device including a display function and receiving data for a personalization service including a response to the questionnaire from the external device
A method of operating a video display device. 6. The method of claim 5,
The questionnaire received from the content provider includes a protocolVersion element, a userDataId element, a userDataVersion element, a time element, a user_id element and a question type element,
Wherein the question type element includes an integer responsive question, a Boolean answer question, a choice answer question, a text answer question, a random answer question
A method of operating a video display device. 6. The method of claim 5,
Retrieving from the storage unit an ID identical to a user ID included in data for the personalization service received from the external device,
Storing the data for the received personalization service if the search result does not exist,
And updating the pre-stored data with the received personalization service if the search result is present
A method of operating a video display device. The method according to claim 1,
The login information includes a user ID in a text form, user face information, user voice information, user fingerprint information, and identification information of an external device connected to the video display device
A method of operating a video display device. The method according to claim 1,
And transmitting the converted metadata to an external device
A method of operating a video display device. Receiving a broadcast signal including metadata that assists main content and main content from a content provider, receiving login information, identifying the user by matching the stored user information, and providing a personalization service corresponding to the identified user A control unit for searching for data and converting an output form of the metadata based on a search result; And
And a storage unit for storing data for the personalization service, the main content and metadata, and user information for the user identification
Video display device. 11. The method of claim 10,
The metadata received from the content provider includes only one type,
Wherein,
Determines the handicap information of the user from the data for the personalization service, and selects the conversion type of the metadata based on the determination result
Video display device. 12. The method of claim 11,
Wherein,
Recognizes the received main content, and selects a conversion type based on the recognized genre information of the content
Video display device. 11. The method of claim 10,
Wherein,
Transmitting the questionnaire received from the content provider to an external device including a display function, and collecting data for a personalization service including a response to the transmitted questionnaire
Video display device. 14. The method of claim 13,
Wherein,
Searches the storage unit for an ID identical to a user ID included in the collected personalization service data, and stores or updates data for the personalization service based on the search result
Video display device. 11. The method of claim 10,
Wherein the control unit transmits the converted metadata to an external device
Video display device. 16. The method of claim 15,
The control unit transmits the converted metadata to one or more external devices
Video display device.

Images