KR101452629B1 - Method and system for content searching service - Google Patents

Abstract

There is provided a method and system capable of finding information of a content based on the sound of the content and providing additional service information of the content while the user is viewing the content.

Description

TECHNICAL FIELD [0001] The present invention relates to a content search service,

TECHNICAL FIELD The present invention relates to a content information providing technique based on acoustic data, and more particularly, to a method and system for providing information on contents of a TV program.

Interactive TV services have been proposed in various ways by the broadcasting and communication convergence technology. Also, with the proliferation of smart devices, viewers' TV viewing styles are changing with respect to smart devices.

Due to the increased use of smart devices, users can participate in the transmission and viewing process of the images directly via the TV. In addition, new forms of media services are emerging that enable various forms of interaction in the process of participation.

Korean Patent Publication No. 2005-0121641 (published on December 27, 2005) discloses various public events such as a home, a terminal waiting room, a running road, a train, a bus, a theater, a baseball field, a soccer field, a basketball court, A wireless Internet communication service is provided to viewers who are exposed to programs such as various advertisements, surveys, news, education, drama, sitcom, documentary, sports broadcast, education broadcast, comedy, contest, Discloses a broadcasting response system (BRS) which enables a user to communicate with a viewer by using a technology.

One embodiment can provide a method and system for providing recognition information of contents and reproduction time information of contents based on the sound of the contents.

One embodiment may provide a method and system for providing additional services based on perceived content information based on sound.

One embodiment may provide a method and system for providing content information for a content duplication portion that is a part of a plurality of content portions.

There is provided a content search service method comprising a step of receiving sound data from a terminal, searching for a content corresponding to the sound data, and transmitting content information about the searched content to the terminal.

The content information may include content organization information of the searched content.

The content information may include a reproduction time at which a circular sound corresponding to the sound data of the contents is outputted.

The reproduction time may be a time corrected according to the time used for the search.

The content may be a TV broadcast program.

The content search service method may further include receiving original sound signals of the contents and program information of the contents, and storing the original sound signals and the program information of the contents in association with each other.

The searching step may search the contents based on a result of comparing the sound data and the original sound signal.

The storing step may store information of a part of the original sound signal.

The portion of the information of the original sound signal may include at least one of a fingerprint and a watermark within the original sound signal.

The retrieving may retrieve the content by comparing the fingerprint in the audio data and the fingerprint of the content.

The retrieving may retrieve the content by comparing the watermark in the audio data and the watermark of the content.

The content search service method includes the steps of mapping supplementary service information for a supplementary service related to a reproduction section of a part of a content to a reproduction section of a part of the content, receiving an additional service request for the content from the terminal, Searching for a playback section corresponding to the supplementary service request among the contents, and transmitting the supplementary service information mapped with the searched playback section to the terminal.

The content search service method includes: mapping supplementary service information for a supplementary service related to a reproduction section of a part of the content to a reproduction section of a part of the content; searching for a reproduction section corresponding to the sound data among the contents As shown in FIG.

The step of transmitting to the terminal may transmit the supplementary service information mapped in the retrieved playback section to the terminal.

According to another aspect of the present invention, there is provided a recognition service providing system, wherein the recognition service providing system receives sound data from a terminal, searches contents matching the sound data, and transmits content information about the searched contents to the terminal A content search service system is provided.

The recognition service providing system may correct the time at which the original sound is output according to the time used for the search.

The content search service system may further include a recognition information generation system.

The recognition information generation system may receive the original sound signal of the content and the combination information of the content, and may store the original sound signal and the combination information of the content in association with each other.

The recognition service providing system may search the contents based on a result of comparing the sound data and the original sound signal.

The recognition information generation system may store a part of the original sound signal.

The recognition service providing system may search the content by comparing the fingerprint in the sound data and the fingerprint of the content.

The recognition service providing system can search for the content by comparing the watermark in the sound data and the watermark of the content.

The content search service system may further include a service manager and a service web server.

The service manager may map supplementary service information for the supplementary service related to the reproduction period of the part of the content to the reproduction period of the part.

The service web server may receive a supplementary service request for the content from the terminal, retrieve a reproduction interval corresponding to the supplementary service request among the contents, and transmit the supplementary service information mapped with the retrieved reproduction interval to the terminal Lt; / RTI >

According to another aspect of the present invention, there is provided a method of reproducing audio data, comprising the steps of: receiving sound data from a terminal, performed by a server; searching for a content duplication part corresponding to the sound data; and transmitting content information on the searched content duplication part to the terminal Wherein the content duplication portion is a part common to a plurality of contents managed by the server.

The content duplication portion may be at least one of an OST of the content, an advertisement, a full-content title, and a content-after-title.

The plurality of contents may be a series of TV broadcast programs.

According to another aspect of the present invention, there is provided a recognition service providing system, wherein the recognition service providing system receives sound data from a terminal, searches for a content duplication part corresponding to the sound data, To the terminal, and the content duplication portion is a part common to a plurality of contents managed by the recognition service providing system.

There is provided a content search service method and a content search service system for searching for content corresponding to the sound data received from the terminal and transmitting the content information about the searched content to the terminal.

And a recognition service providing system, wherein the recognition service providing system receives the sound data from the terminal, searches the content corresponding to the sound data, and transmits the content information about the searched content to the terminal do.

There is provided a method and system for providing content information for a content duplication portion that is a part common to a plurality of contents.

1 is a configuration diagram of a content search service system according to an embodiment.
2 is a block diagram of a content search service system for transmitting content information and supplementary service information results according to an embodiment.
FIG. 3 is a configuration diagram of a content search service system for transmitting an additional service information result according to an embodiment.
4 is a configuration diagram of a smart link indexing system according to an example.
5 is a configuration diagram of a smart link matching system according to an example.
6 is a configuration diagram of a smart link service manager according to an example.
7 is a configuration diagram of a mapping between contents and additional service information according to an example.
8 is a configuration diagram of a service page template management according to an example.
FIG. 9 is a connection diagram of a smart link service DB, a smart link web server, and a smart link service manager according to an example.
FIG. 10 is a block diagram of a smart link service web server, a computing device, and a smart link service DB according to an example.
11 is a functional block diagram of a smart link service web server according to an example.
FIG. 12 is a configuration diagram of a session management of a smart link service web server according to an example.

In the following, embodiments will be described in detail with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

Various modifications may be made to the embodiments and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the description. It is to be understood, however, that the intention is not to limit the embodiments to the embodiments, but to include all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms used in the examples are used only to illustrate specific embodiments and are not intended to limit the embodiments. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this embodiment belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the embodiments, a detailed description of related arts will be omitted if it is determined that the gist of the embodiments may be unnecessarily blurred.

1 is a configuration diagram of a content search service system according to an embodiment.

In the following, the content may be a TV broadcast program. Hereinafter, the TV broadcast program will be outlined as "program ". Alternatively, the content may be a radio program, a satellite broadcast program, a video and music, and the like.

The content search service system 100 includes a computing device 110 for receiving sound signals of a program and transmitting the sound data to the recognition service providing system 120, a search device for searching the information of the corresponding content based on the transmitted sound data, A recognition service providing system 120 for transmitting the content information to the computing device 110 and a recognition information generating system 130 for storing the content information in association with the original sound signals of the content.

The content information may include information related to the content, e.g., content metadata of the content. Further, the content information may include content formation information.

The content organization information may be information related to the combination of broadcasting of the content. For example, the content organization information may include an identifier (ID) of a content, a provider of the content, a content name, a start time of the content, and an end time of the content.

The recognition service providing system 120 may receive the sound data from the computing device 110 and may search the recognition information generation system 130 for the content information of the content corresponding to the transmitted sound data, To the device (110).

The recognition information generation system 130 may receive the original sound signal of the program and the information on the content formation. The recognition information generation system 130 may store the original sound signal as digital data in association with the corresponding content information. The recognition information generation system 130 may store information of a part of the original sound signal when storing the associated information. The information of a part of the original acoustic signal may include at least one of a finger print and a water mark in the original acoustic signal. The recognition information generation system 130 can process and store information inputted in real time without delay.

In step 140, the recognition service providing system 120 may receive acoustic data from the terminal. The terminal may be a device capable of receiving sound. For example, the terminal may include a computing device 110, a smart device 210 described below with reference to FIG. 2, a cellular phone, and the like. The computing device 110 may be coupled to the recognition service providing system 120 via the Internet.

In step 145, the recognition service providing system 120 can search for content that matches the acoustic data received from the recognition information generation system 130. [

In the process of outputting the sound of the content, the original sound may be deformed and output due to noise, loss in the compression process, loss in the decompression process, loss in the analog output, and the like. In addition, the original sound can be deformed in the course of inputting the outputted sound to the terminal. Thus, the sound represented by the sound data may not coincide with the original sound of the content. Means that the sound represented by the sound data is identified as having the same content as the original sound of the content. Or "match" may mean the extent to which the sound of the sound data is not the same as the original sound of the content, but can uniquely correspond to the original sound. Or "match" may mean the extent to which the viewer can perceive the sound of the sound data and the original sound of the content as identical. Or "match" may mean that the sound of the sound data may be considered to be the output and re-input of the original sound of the content.

Using the searched content, the recognition service providing system 120 can search the contents information of the searched contents in the recognition information generation system 130. [

The content information may include content organization information of the searched content. The content information may include a reproduction time at which a circular sound corresponding to the sound data out of the searched contents is output.

The recognition service providing system 120 can correct the reproduction time according to the time used for searching when transmitting the current reproduction time of the contents as part of the contents information. That is to say, the reproduction time may be the time corrected according to the time used for the search.

In step 150, the recognition service providing system 120 may transmit content information about the searched content to the terminal.

2 is a block diagram of a content search service system for transmitting content information and supplementary service information results according to an embodiment.

The content search service system 100 of the content may include a smart link matching system 220, a smart link indexing system 230, a smart link service manager 240 and a smart link service web server 250.

In step 261, the smart link indexing system 230 may obtain the original acoustic signal from the broadcast image and sound 260.

At step 262, the smart link indexing system 230 may receive content programming information from the content programming information and transmission operating system 270.

The smart link indexing system 230 may correspond to the recognition information generation system 130 described above in FIG. The smart link indexing system 230 may store the original sound signal received at step 261 and the content programming information received at step 262 in association with each other.

 The smart link indexing system 230 may create a separate database for content redundancy if the original sound signal is a content overlapping portion. Here, the content duplication portion may be a part common to a plurality of contents managed by the contents search service system 100 of FIG. The common part may also be content.

For example, the plurality of contents may include a series of TV broadcast programs. In other words, the broadcast sequences constituting a series of TV broadcast programs may be a plurality of contents. The plurality of contents may be contents related to each other in a name, a subject, a cast, a producer, a broadcasting company, etc. The content duplication part may be a plurality of contents having different broadcast times, such as an OST of the contents, an advertisement, Or a plurality of contents irrelevant to the content may have the same sound. One content may include a content full title, one or more full advertisements, a video that is the content of the content, one or more post-ads, and a post-content title. When a user views a content duplication part, it may be difficult for the user to watch the content of one or more contents including the content duplication part. Therefore, when the user watches the content duplication portion, the content duplication portion is specified, and information on the content duplication portion may need to be provided to the user.

In indexing the content including the content duplication portion, the smart link indexing system 230 can index the content information of the content and index the content information of the content duplication portion. That is, the content duplication part can be indexed separately from the content. Here, the content information of the content duplicated portion may include a content duplicate ID, an advertisement ID, an OST ID, and the like.

The indexed content information of the content can be stored for each content. The indexed content information of the content duplication portion can be separately stored according to the type of the content duplication portion. For example, the indexed content information of the content duplication portion can be classified and stored as OST, advertisement, full-content title, and content post-title.

The smart link indexing system 230 can extract unique features for each point in time of the original sound of the content being broadcast and can configure a fingerprint based on the extracted unique features. The smart link indexing system 230 may store fingerprint and content information together.

As described above, the content duplication portion can be indexed separately from the content. The smart link indexing system 230 may extract the fingerprint of the content from the original sound signal received in step 261 and index the content using the extracted fingerprint. In addition, the smart link indexing system 230 can extract the fingerprint in the content redundancy portion and index the content redundancy portion using the extracted fingerprint.

The steps 261 and 262 performed by the smart link indexing system 230 may be performed in any order. The smart link indexing system 230 may associate the original sound signal and the content information with each other after steps 261 and 262 are performed.

At step 271, the smart link service manager 240 may receive the content programming information from the content programming information and the transmission operating system 270. [

At step 272, the smart link service manager 240 may receive content metadata from a content management system (CMS). Here, the content metadata may mean a playback time of a content, a channel, a broadcast type, and the like. The type of broadcast may be either broadcast or rebroadcast.

The smart link service manager 240 may map the supplementary service information for the supplementary service related to the reproduction period of the part of the content to the reproduction period of the part of the content. Here, some of the playback sections may be included in the content information acquired based on the sound data. The starting point of a part of the reproduction section may be the reproduction time at which the original sound signal corresponding to the sound data in the contents is outputted.

For example, the supplementary service information may include information of a character and a position where the content is produced.

The service link service manager 240 can organize and manage the contents of the additional service provided to the user through the smart link service web server 250.

Steps 271 and 272 performed by the smart link service manager 240 may be performed in any order. The smart link service manager 240 may map supplementary service information related to a reproduction interval of a part of the content and a reproduction interval of a part of the content after the step 271 and the step 272 are performed.

In step 281, the smart device 210 may receive the sound of the content. The sound of the content may be TV sound.

At step 282, the smart link matching system 220 receives the sound data from the smart device 210. The smart link matching system 220 may correspond to the recognition service providing system 120 described above in FIG.

In step 283, the smart link matching system 220 may search for content based on the comparison of the received sound data and the original sound signals. The smart link matching system 220 can search for content that matches the sound data. The smart link matching system 220 may retrieve content by comparing the fingerprint in the audio data and the fingerprint of the content. In addition, the smart link matching system 220 can search for content by comparing the watermark in the audio data and the watermark of the content. Here, the fingerprint of the content may be a fingerprint of the original sound signal of the content. The watermark of the content may be a watermark of the original sound signal of the content.

In step 283, when the result of the comparison of the sound data and the original sound signal is a content duplication part, the smart link indexing system 220 can search contents and content duplication parts corresponding to the sound data.

As described above, the content duplication portion can be indexed separately from the content. For example, when the content is a program, the overlapping portion of the contents of the plurality of programs may correspond to the sound data. When the program ID indicates the program, and the content duplicate part ID indicates a content duplication part, the smart link indexing system 220 can retrieve a plurality of program IDs and a content duplicate part ID. Here, the plurality of program IDs may be the IDs of programs including the content duplication part corresponding to the sound data.

The smart link indexing system 220 can provide only a duplicate part ID of the retrieved plurality of program IDs and the content duplicated part ID as a result of the search. Therefore, at a later step, only the content duplication portion excluding the plurality of programs can be provided as a result of the search.

For example, when a user watches an OST of a program, the sound data of the OST may correspond to one or more programs including the OST and a content overlap portion of the one or more programs. The smart link indexing system 220 can detect the program IDs of one or more programs and the content ID of the OST, but can provide clear information by using only the content ID of the OST as a result of the search.

When the smart link matching system 220 sends the retrieved content to the smart link indexing system 230, the smart link indexing system 230 may retrieve the content information.

At step 283, the smart link matching system 220 may use a hybrid scheme that uses both immutable and variable data for content retrieval.

For example, the immutable data may be one or more of the ID of the content and the name of the content. Constant data can be embedded in the content as a watermark. The variable data may be metadata of the contents. The metadata may include a playback point of time, a channel, a main frame, or a return status of the content. The variable data can be inserted into the content as a fingerprint.

For example, the smart link matching system 220 can detect a watermark, which is constant data in the original sound of the content being broadcast, and can acquire the content information by using the detected watermark. The content information included in the watermark may include the ID of the content and the name of the content. The smart link matching system 220 can retrieve the playback time of the content and additional service information for the content by using the obtained content information and the fingerprint. By the hybrid scheme, the time used for retrieving the content information can be reduced more than the time used by the method using only the fingerprint.

At step 284, the smart link indexing system 230 may send the retrieved content information to the smart link matching system 220.

When the content overlapping portion is received, the smart link indexing system 230 transmits to the smart matching system 220 the content information of the overlapping portion of the plurality of content information searched at Step 283 and the content information of the content overlap portion .

At step 285, the smart link matching system 220 may send the retrieved content information to the smart device 210. The smart link matching system 220 may transmit content information for the retrieved content redundancy portion to the smart device 210. [

At step 286, the smart link service web server 250 may receive an additional service request for content from the terminal. The supplementary service request may be based on the content information transmitted to the terminal by the smart link service web server 250. The supplementary service request may include the playback time of the content.

At step 287, the smart link service web server 250 may receive an additional service request. Upon receiving the supplementary service request, the smart link service web server 250 can retrieve the supplementary service information from the smart link service manager 240. The smart link service web server 250 can retrieve the playback interval corresponding to the supplementary service request among the contents.

The additional service information may be related to the reproduction time of the content which is the content information. In other words, the smart link service web server 250 can retrieve the supplementary service information mapped with the playback interval corresponding to the playback time of the content included in the supplementary service request.

In step 288, the smart link service manager 240 may transmit the retrieved playback interval and the mapped supplementary service information to the smart link service web server.

In step 289, the smart link service web server 250 may transmit the supplementary service information mapped with the searched playback interval to the mobile station.

The smart link service manager 240 can be named and outlined as a service manager.

The smart link service web server 250 may be named and outlined as a service web server.

The smart link matching system 220, the smart link indexing system 230, the smart link service manager 240, and the smart link service web server 250 may be physically configured as one system. For example, the smart link matching system 220 and the smart link indexing system 230 may be implemented in a physically single first server. In addition, the smart link service manager 240 and the smart link service web server 250 may be implemented in a physically single second server. Alternatively, a single server of the system may perform all the functions of smart link matching system 220, smart link indexing system 230, smart link manager 240 and smart link service web server 250.

FIG. 3 is a configuration diagram of a content search service system for transmitting an additional service information result according to an embodiment.

3, the contents search service system 100 of FIG. 1 receives the sound data from the smart device 210 of FIG. 2 and transmits the additional service information to the smart device 210 based on the received sound data Fig.

The content search service system 100 includes a smart link matching system 220 of FIG. 2, a smart link indexing system 230 of FIG. 2, a smart link service manager 240 of FIG. 2, a smart link service web server 250 and the Smart Link Service Database (DB) 252 of FIG.

At step 310, the smart device 210 may receive input of a content sound.

At step 320, the smart link service web server 250 may receive the sound data from the smart device 210. [

In step 330, the smart link service web server 250 may request the smart link matching system 220 to search for the sound data 140.

At step 340, the smart link matching system 220 may retrieve content information from the smart link indexing system 230 using a fingerprint or watermark of the acoustic data. When a watermark and a fingerprint are used together, a hybrid method can be used.

The smart link indexing system 230 may include a registration system, a memory DB, and a fingerprint main DB. The registration system, the memory DB and the fingerprint main DB are described in more detail in the following embodiments.

In steps 310 to 330, the smart link indexing system 230 associates the original sound signal and the content schedule information of the broadcast image and sound 260 with the content schedule information transmitted from the transmission operation system 270, As shown in Fig. Steps 261 and 262 described above with reference to FIG. 2 may be performed between steps 310-330.

At step 350, the smart link indexing system 230 may send the retrieved content information to the smart link matching system 220.

At step 360, the smart link matching system 220 may send the retrieved content information to the smart link service web server 250.

In step 370, the smart link service web server 250 can search for supplementary service information in the smart link service DB 252 based on the content information transmitted from the smart link matching system 220. In addition, the smart link service web server 250 can search for a playback interval corresponding to the sound data in the content. The retrieval of the supplementary service information may be a retrieval of the supplementary service information mapped with some reproduction section including the reproduction time of the transmitted contents information.

In step 380, the smart link service DB may transmit the supplementary service information mapped in the retrieved playback section to the smart link service web server 250. [

In step 390, the smart link service web server 250 may transmit the supplementary service information mapped in the retrieved playback section to the smart device 210. [

Between steps 310-360), the smart link service manager 240 receives the content programming information from the content delivery information and delivery operating system 270 of FIG. 2 and receives the content metadata from the CMS 280 can do. The smart link service manager 240 may map the supplementary service information for the supplementary service related to the reproduction period of part of the content to a part of the reproduction period.

The smart link service manager 240 may store the mapped data in the smart link service DB 252. [ The smart link service DB 252 may be located inside or outside the smart link service manager 240. In other words, the smart link service manager 240 and the smart link service DB 252 may be physically one device or a separate device.

4 is a configuration diagram of a smart link indexing system according to an example.

The smart link index system 230 may include a registration system 410 and a fingerprint DB 420.

The registration system 410 may include a schedule synchronization manager 411, a real time fingerprint extractor 412, a file unit fingerprint extractor 413, a fingerprint register 414 and an interface device 415.

At step 452, the interface device 415 may obtain the original sound from the broadcast video and sound 260 of FIG.

In step 454, the interface device 415 may send the received original sound to the real-time fingerprint extractor 412 as an audio signal. If the sound for which the fingerprint is to be extracted is generated in the sound source file 416, the file may be transmitted to the file-based fingerprint extractor.

In step 456, the real-time fingerprint extractor 412 may extract the fingerprint data from the original sound audio signal obtained via the interface device 415. [ The fingerprint data may be digitized data. The real-time fingerprint extractor 412 may use a parallel scheme, which is a way to reduce the impact on the operation of the broadcast system when extracting the fingerprint data from the original sound. The parallel scheme may mean that the original sound output of the broadcast system and the output to the real-time fingerprint extractor 412 do not affect each other. The real-time fingerprint extractor 412 may send the fingerprint data to the fingerprint register 414.

In the case where sound is generated in the sound source file 416, the file-based fingerprint extractor 413 can extract the fingerprint data from the file. The file-based fingerprint extractor 413 can extract fingerprints using an algorithm applied to the original sound. The file-by-file fingerprint extractor 413 may send the fingerprint data to the fingerprint register 414.

At step 458, when fingerprint data is sent from the real-time fingerprint extractor 412 to the fingerprint register 414, the fingerprint register 414 sends the detailed information of the content for the entered fingerprint to the schedule table synchronization manager 414 411). For example, the detailed information of the content may include a broadcast date of the content, a start time and an end time of the content, a content ID, a content name, a broadcast channel of the content,

At step 460, the schedule synchronization manager 411 may request and receive the content scheduling information and the content scheduling information from the transmission operating system 270. For example, the content organization information may include daily broadcast schedule management information, weekly broadcast schedule management information, updated broadcast schedule information, and the like.

At step 462, the schedule synchronization manager 411 may process the scheduling information of the content into the content detail information. The schedule synchronization manager 411 may transmit fingerprint data and detailed information requiring indexing among the detailed content information to the fingerprint register 414. [ For example, the detailed information may include a content ID and a current playback time of the content. Among the detailed contents information, information used for indexing may be content information.

At step 464, the fingerprint register 414 may receive content detail information from the schedule synchronization manager 411. [ The fingerprint register 414 may generate the fingerprint ID, the reproduction time, by utilizing the fingerprint data and the content detail information transmitted from the real-time fingerprint extractor 412 or the file-based fingerprint 413. The fingerprint register 414 may store the playback time corresponding to the fingerprint, the fingerprint ID, and the fingerprint in the memory DB 422 and the main DB 424. The fingerprint register 414 can quickly store the fingerprints generated in the original sound in the database or respond quickly to queries of the user viewing the original sound content and some data stored in the memory DB 422 . The fingerprint, the fingerprint ID, and the playback time may be stored in advance in the memory DB 422. [ In addition, the fingerprint register 414 may store the fingerprint extracted from the sound source file 416 and the fingerprint of a large capacity in the disk-based fingerprint main DB 424 for stable management. The fingerprint register 414 may store only the fingerprint extracted from the sound source file 416 in the fingerprint main DB 424 instead of the memory DB 422. [

5 is a configuration diagram of a smart link matching system according to an example.

The smart link matching system 220 may include a fingerprint interface manager 510 and a fingerprint distribution processor 520.

At step 552, the fingerprint interface manager 510 may receive a fingerprint search request from the smart link service web server 250. [

At step 554, the fingerprint interface manager 510 may send a search request to the fingerprint distribution processor 520. [

In step 556, the fingerprint distribution processor 520 may retrieve content information that matches the fingerprint search request received from the smart link indexing system 230. [ The smart link indexing system 230 can be divided into a memory DB and a main DB to correspond to a plurality of queries. Diversification of system load and database stability can be ensured by dualization. The memory DB can recognize the fingerprint more quickly than the main DB.

The smart link index system 230 can use a memory-based database in addition to a disk-based database in order to process a large-scale plurality of queries of users viewing original sound content. The fingerprint distribution processor 520 may store in the memory DB 501 fingerprint data of a predetermined amount of time during the fingerprint of the original sound content. The fingerprint distribution processor 520 can improve the response speed of the user's request by primarily performing the fingerprint search in the memory DB 230. [

The fingerprint distribution processor 520 may secondarily search the main DBs 501 to 505 for fingerprints if a matching fingerprint is not found in the memory DB 501. [ The fingerprint distribution processor 520 may simultaneously send a search request to the physically distributed main DBs 501 to 505, respectively, at which the fingerprint received in step 554 is transmitted. As the amount of fingerprints to be processed increases, the concurrent collective distributed search method can prevent concentration of fingerprint search requests to some main DBs. The above-described distributed processing can also be applied to other searches of the embodiment.

In step 558, the smart link matching system 220 may send content information matching the fingerprint search request to the fingerprint distribution processor 520.

At step 560, the fingerprint distribution processor 520 may send the content information to the fingerprint interface manager 510.

At step 562, the fingerprint interface manager 510 may send the content information to the smart link service web server 250. [

6 is a configuration diagram of a smart link service manager according to an example.

The smart link service manager 240 of FIG. 2 includes a smart link service DB 252, an interface manager 610, a content service organizer 620, automatic and manual transmission manager 630, service template page management 640) and a service template page creation (650).

The interface manager 610 receives the content programming information and the content programming information from the transmission operating system 270 and receives the content meta data from the CMS 280. [

The content service creator 620 may enable the viewer to edit the content based on the supplementary service information corresponding to the viewpoint when the user knows the current viewing point.

For example, the supplementary service information may include text, images, moving images, and Internet links present on-line. The content service creator 620 may perform mapping of additional service information based on a time line of the content video and the content video. The mapped supplementary service information may be stored in the smart link service DB 252. The content service creator 620 can designate a time when the mapped supplementary service information is exposed to the user when the content scene is searched according to the time line.

The automatic and manual transmission manager 630 transmits supplementary service information corresponding to the viewing point automatically changed through the smart link service web server 250 to the user as the viewing point of time of the user changes. The manager can transmit the additional service information immediately to the user who watches the content selected by the administrator in addition to the transmission of the automatic additional service information through the manual broadcast transmission. The specific additional service information selected by the administrator can be stored in the DB 252 for the smart link service.

The service template page management unit 640 can perform management for displaying additional service information. Here, the management may be a service page management for displaying page formats previously created for each category in order to quickly create a service page of additional service information. The service template page management 640 may provide a function for an administrator to register and delete template categories of service pages. For example, a template of a service page may include a person, a place, a commodity, an event, and an Internet link.

The service template page creation (650) may provide a function by which an administrator can create a template of a service page.

7 is a configuration diagram of a mapping between contents and additional service information according to an example.

In FIG. 7, additional service information for the timeline of the content 710 is mapped to the supplementary service information table 720. When the content is reproduced, when the supplementary service information reaches the mapped playback period, information indicated in the supplementary service information table 720 can be delivered to the user.

8 is a configuration diagram of a service page template management according to an example.

The template 820 of the service page 810 may be managed through the service template page management 640. [ The categories in the template 820 can be registered and deleted.

FIG. 9 is a connection diagram of a smart link service DB, a smart link web server, and a smart link service manager according to an example.

The smart link service DB 252 can be used to stably manage user session information and content formation information. The smart link service DB 252 may provide stored content metadata management, service page management registered by the content service scheduler, automatic and manual transmission schedule management, service page template management, and session management.

FIG. 10 is a block diagram of a smart link service web server, a computing device, and a smart link service DB according to an example.

The computing device 110 may further include an external interface 1011 and a fingerprint extraction module 1012 to recognize broadcast sound as the computing device 110 of FIG. 1 and obtain additional service information.

At step 1052, the fingerprint module 1010 of the computing device 110 may receive sound. The fingerprint extraction module 1012 can extract a fingerprint from the input sound.

At step 1054, the computing device 110 may send the extracted fingerprint to the smart link service web server 250 of the content search service system 100. The smart link service web server 250 may receive a fingerprint from the computing device 110 and then create a user's session

At step 1056, the smart link service web server 250 may retrieve additional service information mapped to the fingerprint transmitted in the smart link service DB 252 of FIG. 2 based on the received fingerprint.

At step 1058, the smart link service web server 250 may send the address of the service page of the supplementary service information to the HTML5 SSE processing module 1013 of the computing device 110. The HTML5 SSE processing module 1013 can continuously receive a resource locator (URL) of a service page for the content received from the smart link service web server 250. [ The interface manager 1021 of the smart link web server 250 can inquire the service information of the automatic sending part of the smart link service DB 252. [ The smart link service web server 250 may continuously transmit the automatic transmission service information to the HTML5 SSE processing module 1013. [

In step 1060, the HTML5 SSE processing module 1013 sends the received automatic transmission unit service information URL to the web view control module 1014 so that the web module accesses the web server corresponding to the additional service information URL, . In step 1070, when the service information of the manual transmission broadcasting section is input by the administrator, the smart link service manager 240 can register the supplementary service information in the smart link service DB 252. [

In step 1072, the smart link web server 250 may transmit the registered manual broadcasting service supplementary service information to the HTML5 SSE processing module 1013 of the computing device 110 of users viewing the corresponding content. A search for users viewing the content may be provided by the session manager 1023. [

Steps 1070 and 1072 may be performed between steps 1054 to 1060. [

The computing device 110 may query the supplementary service page from an external web server 1090 other than the smart link web server 250. [

11 is a functional block diagram of a smart link service web server according to an example.

The smart link service web server 250 of FIG. 2 may include an interface manager 1120, a session manager 1130, and a fingerprint interface manager 1140.

At step 1152, the interface manager 1120 may receive a request and fingerprint of the supplementary service information from the computing device 110.

In step 1154, the interface manager 1120 may create the user's session. The generated session can be managed by the session manager 1130. [ The managed session can be automatically destroyed after a predetermined time in a state in which no user is connected.

In step 1156, the interface manager 1120 may request the fingerprint interface manager 1140 to recognize the fingerprint transmitted from the terminal.

At step 1158, the fingerprint interface manager 1140 may retrieve the content information via the smart matching system 220. [

In step 1160, the fingerprint interface manager 1130 may send the retrieved content information to the interface manager 1120. [

In step 1162, the interface manager 1120 may retrieve supplementary service information from the smart link service DB 252 based on the received content information.

In step 1164, the interface manager 1120 may transmit the retrieved supplementary service information to the terminal.

FIG. 12 is a configuration diagram of a session management of a smart link service web server according to an example.

12 may represent session management of the smart link service web server 250. [

When an additional service information request is sent to a plurality of users, the interface manager 1120 can send a session creation request to the session manager 1130. [ The session manager 1130 can create a session and store the generated session data in the smart link service DB 252.

The development of SmartLink TV service system can be based on the already developed hybrid broadcasting service platform. By combining the convenience of watching video, which is the advantage of TV, and online information accessibility, which is the advantage of smart devices, a service related to the content of a broadcast program or video on demand (VOD) being watched can be provided through a smart device.

The smart link TV service may be a service that can recognize the sound of the broadcast content being watched on the TV as a smart device and provide detailed information on the content. Here, the detailed information on the content may include the content provider, the content name, and the viewing / reproduction time. The smart link TV service may include a service for providing additional service information.

The user can obtain information on the current playback time of the content using the smart device that is being carried at any time during TV viewing. In addition, the user can obtain supplementary service information related to the content being viewed. That is to say, the SmartLink TV service may mean linking the smart device with the TV content and using the linked smart device to provide the supplementary service information associated with the content and playback time.

100: Content search service system
220: Smart Link Matching System
230: Smart link indexing system
240: Smart Link Service Manager
250: Smart Link Service Web Server

Claims (13)

Receiving acoustic data from a terminal;
Retrieving a content corresponding to the sound data; And
Transmitting content information about the searched content to the terminal
Lt; / RTI >
Wherein the content information includes a reproduction time at which an original sound corresponding to the sound data of the contents is outputted,
Wherein the playback time is a time corrected according to a time used for the search. The method according to claim 1,
Wherein the content information includes content organization information of the searched content. delete delete The method according to claim 1,
Receiving original sound signals of the contents and program information of the contents; And
Storing the original sound signal and the combination information of the contents in association with each other
Further comprising:
Wherein the searching step searches the contents based on a result of comparing the sound data and the original sound signal. 6. The method of claim 5,
Wherein the storing step stores information of a part of the original sound signal. The method according to claim 6,
Wherein the information of the part of the original sound signal includes at least one of a fingerprint and a watermark within the original sound signal. The method according to claim 1,
Mapping supplementary service information for a supplementary service related to a reproduction section of the content to a reproduction section of a part of the content;
Receiving an additional service request for the content from the terminal;
Retrieving a playback interval corresponding to the supplementary service request among the contents; And
Transmitting the supplementary service information mapped to the searched playback interval to the terminal
The content search service method further comprising: The method according to claim 1,
Mapping supplementary service information for a supplementary service related to a reproduction section of the content to a reproduction section of a part of the content; And
Searching for a reproduction section that matches the sound data among the contents
Further comprising:
Wherein the step of transmitting to the terminal transmits the supplementary service information mapped to the retrieved playback section to the terminal. Recognition service providing system
/ RTI >
Wherein the recognition service providing system receives the sound data from the terminal, searches the content corresponding to the sound data, transmits the content information about the searched content to the terminal,
Wherein the content information includes a reproduction time at which an original sound corresponding to the sound data of the contents is outputted,
Wherein the playback time is a time corrected according to a time used for the search. Performed by the server,
Receiving acoustic data from a terminal;
Retrieving a content duplication portion corresponding to the sound data; And
Transmitting content information on the searched content duplication part to the terminal
Lt; / RTI >
Wherein the content duplication portion is a part common to a plurality of contents managed by the server,
Wherein the content information includes a reproduction time at which an original sound corresponding to the sound data of the contents is outputted,
Wherein the playback time is a time corrected according to a time used for the search. Recognition service providing system
/ RTI >
Wherein the recognition service providing system receives the sound data from the terminal, searches for a content duplication part corresponding to the sound data, transmits the content information about the searched content duplication part to the terminal,
Wherein the content duplication portion is a part common to a plurality of contents managed by the recognition service providing system,
Wherein the content information includes a reproduction time at which an original sound corresponding to the sound data of the contents is outputted,
Wherein the playback time is a time corrected according to a time used for the search. 13. The method of claim 12,
Wherein the content duplication portion is at least one of an OST of the content, an advertisement, a pre-content title, and a content post-title.

Images