WO2015137621A1

WO2015137621A1 - System and method for providing related content at low power, and computer readable recording medium having program recorded therein

Info

Publication number: WO2015137621A1
Application number: PCT/KR2015/000890
Authority: WO
Inventors: 김태현
Original assignee: 주식회사 사운들리
Priority date: 2014-03-11
Filing date: 2015-01-28
Publication date: 2015-09-17

Abstract

Provided is a system for providing related content comprising the steps of: sensing a sound output from a content reproducing device for reproducing content (referred to as "main content"); converting the sensed sound into digital data; extracting an acoustic wave ID from the digital data; selecting related content corresponding to the acoustic wave ID; and displaying the related content in an image and/or speech by a mobile receiver.

Description

Computer-readable recording media recording low power associated content providing systems, methods, and programs

The present invention relates to a low-power related content providing system, a method, and a computer-readable recording medium having recorded therein low-power, and providing related content.

The present invention is a national research and development project of the Small and Medium Business Administration (host: SME Technology Information Agency, Assignment No .: S2145032, research project name: Health Diagnosis Establishment Growth Technology Development Project, research title: It is supported by sound wave multiple access technology development.

[National R & D project supporting this invention]

[Project unique number] S2145032

[Department name] Small and Medium Business Administration

[Research and Management Agency] Small and Medium Business Technology Information Agency

[Name of Research Project] Technology Development Project for Entrepreneurship Growth Linked to Health Diagnosis

[Study title] Development of sound wave multiple access technology for precise indoor location detection with smartphone

Conventional merchandise advertising applies products that are intended to advertise not only directly through newspapers, radio and television broadcasts, but also to props that appear in content such as movies, dramas, and radio broadcasts, or products that are naturally referred to as part of the content. Advertisement techniques (hereinafter referred to as indirect advertisement or indirect advertisement) are widely used. In particular, the indirect advertisement of the product is increasing because it provides a strong advertising effect to viewers and listeners. However, in the case of the conventional indirect advertisements, the products are simply exposed to the screen or referred to as sound. For this reason, even if a viewer or listener needs to purchase a product, there is a problem of separately searching for related information about the product and a retailer, and then purchasing the product based on the searched information. It is becoming. For example, Korean Patent Registration No. 10-1310943 (September 11, 2013) (System and method for providing content related information related to broadcast content), Korean Patent Registration No. 10-1313293 (September 24, 2013) (System and Method of Providing Additional Information of Broadcast Content), and Korean Patent Registration No. 10-0893671 (April 9, 2009) (Creation and Matching Hash of Multimedia Content) uses sound characteristics of the content to be broadcast or reproduced. In addition, the present invention discloses a method of providing related content for product indirect advertisement included in content that is being viewed or listened to by a viewer or listener's mobile receiver. As another example, Korean Patent Registration No. 10-1363454 (February 10, 2014) (Recording the user-customized advertisement providing system based on the sound signal output from the TV, a method for providing the customized advertisement, and MIM service program recorded The computer-readable recording medium) provides a method of providing related content for indirect advertisement of a product included in content that is watched or listened to by a viewer or listener's mobile receiver by inserting non-audible sound waves into the broadcast or played content. It is starting.

However, there is still a problem in the above-described techniques that a viewer or listener must actively request related content using a mobile receiver. That is, in order to receive the related content at the time of playing the content, the application of the mobile receiver must be executed or the content recognition should be instructed to the application of the mobile receiver providing the related content. As such, a separate effort is required for receiving related content because it is opposed to a behavior pattern of a viewer or listener who passively enjoys the content, or because a separate effort interferes with the immersion of the viewer or listener. The utility of the content may be diminished.

On the other hand, such a problem can be solved by the mobile receiver continually extracting acoustic features or inaudible sound waves, but this causes a power shortage problem for a mobile battery operated with limited battery.

According to an embodiment of the present invention, there can be provided a low-power related content providing system that can automatically provide the associated content for the main content being watched or listened while operating at a constant low power even without the viewer or listener actively looking for it. have.

According to an embodiment of the present invention, the sound wave ID included in the main content broadcast or reproduced through the content player is extracted at low power, and the main content of the user is not performed by the user or through the user's mobile receiver. A low power related content providing system may be provided that provides related content for indirect advertising products included in the main content without disturbing immersion in.

According to an embodiment of the present invention, there is a need for a method for providing low power related content that can be automatically provided without viewers or listeners actively seeking related content for main content being watched or listened while operating at low power.

According to an embodiment of the present invention, the sound wave ID included in the main content broadcast or reproduced through the content player is extracted at low power, and the main content of the user is not performed by the user or through the user's mobile receiver. A method may be provided for providing associated content for an indirect advertisement product included in content without disturbing immersion in.

According to an embodiment of the present invention, a method for providing a low power related content may be executed by a computer which may automatically provide the related content of the main content being watched or listened to while being operated at low power at all times without actively seeking the related content. A recording medium on which the program is recorded may be provided.

According to an embodiment of the present invention, the sound wave ID included in the main content broadcast or reproduced through the content player is extracted at low power, and the main content of the user is not performed by the user or through the user's mobile receiver. A recording medium may be provided in which a program for executing a computer for executing a method of providing related content for an indirect advertising product included in main content without interfering with the immersion into the main content.

According to an embodiment of the present invention, a computer-readable recording medium recording a low power-related content providing system, method, or program may be provided.

According to one embodiment of the invention, the step of detecting the sound output from the content player for playing the content (referred to as 'main content'); Converting the sensed sound into digital data; Extracting a sound wave ID from the digital data; Selecting related content corresponding to the sound wave ID; And displaying, by the mobile receiver, the related content in an image and / or voice. The related content providing method may be provided.

According to an embodiment of the present invention, a mobile receiver comprising a memory and a processor, the mobile receiver comprising: a microphone for detecting sound output from a content player for reproducing content (called 'main content'); An ADC for converting the sound sensed by the microphone into digital data; A sound wave ID extracting unit for extracting a sound wave ID from the digital data; A mobile receiver capable of providing related contents may include a display unit for displaying related contents selected in correspondence with the sound wave ID.

According to an embodiment of the present invention, a microphone for sensing a sound output from a memory, a processor, a content player for reproducing content (called 'main content'), and an ADC for converting the sound detected by the microphone into digital data. A computer readable medium having recorded thereon a program for executing a related content providing method in a computer comprising: extracting a sound wave ID from the digital data; And displaying the related content selected corresponding to the sound wave ID through a display unit provided in the computer. A computer readable medium may be provided that records a program for executing a related content providing method comprising a.

According to one or more embodiments of the present invention, the sound wave ID included in the main content broadcast or reproduced through the content player is extracted at low power, and the mobile receiver of the user, which is the viewer or the listener, may be used without the user's own efforts or by the user. It is possible to provide related content for indirect advertising products included in the main content without disturbing the immersion into the main content.

1 is a view for explaining a related content providing system for providing related content using a sound wave ID according to an embodiment of the present invention.

2 is a view for explaining an embodiment of a mobile receiver.

3 is a view for explaining another embodiment of a mobile receiver.

4 is a view for explaining a method of providing related content according to an embodiment of the present invention.

5 is a view for explaining a method for providing related content according to an embodiment of the present invention.

6 is a view for explaining a method for providing related content according to an embodiment of the present invention.

7 is a view for explaining a method for providing related content according to an embodiment of the present invention.

8 is a diagram for explaining main content.

9 is a view for explaining that the acoustic recognition module is activated at a predetermined cycle to operate at low power.

10 is a view for explaining that the sound recognition module is activated in accordance with the time that the sound wave ID can be extracted during the broadcast and playback time of the main content.

FIG. 11 is a diagram for describing an activation cycle of an acoustic recognition module when the existence of the acoustic wave ID is estimated to be greater than or equal to a preset probability through a process of extracting the acoustic wave ID.

FIG. 12 is a diagram for explaining that a period of an acoustic recognition module is adjusted according to an event in which another application using a microphone, which is a shared resource inside a mobile receiver, is activated.

FIG. 13 is a view for explaining that the period of the acoustic recognition module is adjusted according to an event in which another application using a microphone, which is a shared resource, is executed in the mobile receiver.

FIG. 14 is a diagram for describing methods of receiving an event of a mobile receiver to operate an acoustic recognition module at low power.

15 is a view for explaining a method for providing related content according to an embodiment of the present invention.

[Signature Signature]

100: content player 201, 401: microphone

203, 403: ADC 205: Sound Wave ID Extraction Unit

207, 407: Processors 209, 409: Memory

211 and 411: Storage devices 213 and 413: Display unit

215, 415: communication unit

221, 421: DMB application or video flare or codec

223, 423: related content providing applications

200, 400: Mobile receiver 300: Server

402: low power acoustic intensity measurement unit 404: sound wave pattern recognition unit

Objects, other objects, features and advantages of the present invention will be readily understood through the following preferred embodiments associated with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosure may be made thorough and complete, and to fully convey the spirit of the present invention to those skilled in the art.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, the words 'comprises' and / or 'comprising' do not exclude the presence or addition of one or more other components.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In describing the specific embodiments below, various specific details are set forth in order to explain the invention more specifically and to help understand. However, those skilled in the art can understand that the present invention can be used without these various specific details. In some cases, it is mentioned in advance that parts of the invention which are commonly known in the description of the invention and which are not highly related to the invention are not described in order to prevent confusion in explaining the invention without cause.

■ Definition of terms

-Content: Refers to information that provides educational, cultural, informational, and / or entertainment value with structured digital data in the form of video, sound or sound and image.

-Content player: A device that plays content, such as a TV, radio, desktop computer, or mobile device (e.g. laptop computer, tablet computer, smartphone, wearable computer, etc.) to produce sound or speakers The terminal may further include a monitor or an image output terminal for displaying an image.

-Content: i) means main content or related content, or ii) collectively refers to both main content and related content.

-Main content: Content that is output (broadcast or played) through the content player.

Related Content: Content output through a mobile receiver, which refers to information and / or information such as sound, video, documents, internet addresses, etc., associated with the content of the main content output through the content player or the background for delivering the content. Digital identifier value. For example, related content may include content that is difficult to include in the main content due to content organization restrictions or other laws and regulations, content that induces the viewer or listener to participate, or content related to indirect advertising that does not attract the viewer or listener's attention. Or information about a product that comes out of an indirect advertisement or content about a purchase of the product.

-Mobile receiver: A device capable of outputting related content, which includes a microphone and an analog-to-digital converter (ADC) to analyze the sound included in the main content, and analyzes the sound of the main content to identify the main content or After identifying the relative time position of the main content, it is a mobile device that provides the content viewers or listeners in a form that can visually or auditoryly recognize the related content associated with all or part of the main content. In this case, the viewer or listener may be a person who owns the mobile receiver. For example, the mobile receiver may be a mobile device such as a laptop computer, tablet computer, smartphone, wearable computer, or the like.

-Indirect advertisement or indirect advertisement: An advertisement that is output as part of a main content image or as part of sound as an advertisement such as a product, a service, a brand logo, a tagline, etc., which are included as a part or all of the main content. Product indirect advertising or indirect advertising includes the concept of sponsored advertising (advertising that is provided as part of the content by offering a spot at a free or discounted price in the production of the content). The content associated with.

-Indirect advertising products: Products or services that are output as all or part of the main content by indirect advertising or indirect advertising, and are used to include not only products having a specific form but also services, brands, logos, and taglines. do.

Sound ID: The sound wave ID is derived from (1) digital data represented by an audible sound artificially inserted into the sound of the main content (non-audible sound wave ID) or (2) extracted from sound contained as part of the main content. It is abbreviated digital data (audible fingerprint ID) that indicates the characteristics of the sound. The artificially inserted non-audible sound wave ID has a feature that is modulated into non-audible sound waves that are inaudible to humans using psychosychoacoustics. An audible fingerprint sonic ID that characterizes the sound does not require artificial insertion, but can only be extracted when the main content contains sound. Both kinds of sonic IDs can either (1) identify the main content, (2) identify the relative time position within the main content, (3) identify the associated content associated with indirect advertising within the main content, or (4) It may be used to identify associated content associated with an indirect advertisement at a relative time position within the content. Non-audible sound wave IDs may include serial numbers, meaningful words, or Internet links. Audible fingerprint sound wave IDs include Singular / Eigen Value Decomposition (SVD / EVD), Principal Component Analysis (PCA), Peak Landmarks Detection, Zero-crossing counting, and Mel Frequency Cepstral Coefficient (MFCC). , PLPC (Perceptual Linear Prediction Coefficient), LPC (Linear Prediction Coefficient) can be extracted using a hash function (Hash Function), and the extracted value can be used as a hash key (Hash Key). For convenience of description of the non-audible sound wave ID, the non-audible sound wave ID may refer to one of an analog form of an audible sound wave in which digital data is modulated, digital data before modulation, and digital data after demodulation. In addition, for the convenience of technology, the audible fingerprint sound wave ID obtained by extracting a feature of the sound may include extracted feature digital data, a part of main content sound from which it can be extracted, and a feature of extracting feature digital data detected by a mobile receiver. A portion of the content sound may refer to one of the characteristic digital data extracted by the mobile receiver.

Acoustic recognition module: used to mean at least one component used to detect and extract sound wave ID, for example, at least one of a microphone, an ADC, a sound wave ID extractor, and an associated content providing application. Can be named collectively.

■ Related Content Provision System

1 is a view for explaining a system for providing related content using a sound wave ID according to an embodiment of the present invention (hereinafter, ?? associated content providing system ??).

Referring to FIG. 1, the related content providing system is a system for providing related content related to all or a part of main content output from the content player 100 through the mobile receiver 200.

The mobile receiver 200 may provide, for example, related content including an advertisement related to at least one of an image and a sound output as main content. As an example, the mobile receiver 200 may provide related content for an indirect advertising product, which is represented by an image or output by sound, output from main content. According to an embodiment of the present disclosure, the mobile receiver 200 may provide related contents associated with each of a plurality of indirect advertisement products, which are displayed as images output from the main content or referred to as sound.

The associated content providing system according to the present embodiment may include a content player 100, a mobile receiver 200, and a server 300.

The content player 100 may output an image and / or sound. The mobile device 200 may extract at least one sound wave ID from the sound output by the content player 100. For example, the sound wave ID may correspond to the related content through the related content database (DB) —the DB in which the sound wave ID and the related content correspond to each other.

For example, the sonic ID identifies the main content, identifies a relative time position within the main content, identifies an associated content associated with (corresponded to) an indirect ad within the main content, or indirect ad at a relative time position within the main content. Can be used to identify associated content associated with (corresponding).

The content player 100 may include, for example, a speaker or a sound output terminal for reproducing sound such as a TV, a radio, a desktop computer, and a mobile device (eg, a laptop computer, a tablet computer, a smartphone, a wearable computer, etc.). In addition, it may further include a monitor or an image output terminal for displaying an image.

The mobile receiver 200 may extract a sound wave ID included in the sound of the main content output from the content reproducing apparatus 100, and may recognize the related content corresponding to the sound wave ID in a visual or auditory manner. A mobile device that provides content viewers or listeners. In this case, the viewer or listener may be a person who owns the mobile receiver. For example, the mobile receiver 200 may include a mobile device such as a laptop computer, a tablet computer, a smartphone, a wearable computer, and the like. In addition, the mobile receiver 200 may include a microphone (not shown) and an ADC (not shown) for receiving a sound wave ID.

In the present embodiment, the mobile receiver 200 extracts the sound wave ID, transmits the sound wave ID to the server 300, receives the related content corresponding to the sound wave ID from the server 300, and displays the sound content as a sound or image. have. For example, the mobile receiver 200 may display the related content corresponding to the sound wave ID to the user in the form of a notification message, an image, and / or a voice.

In the present exemplary embodiment, the mobile receiver 200 is a mobile device such as a laptop computer, a tablet computer, a wearable computer, a smartphone, and the like, and has a microphone (not shown) for recognizing sound and communication with a server. And a communication unit (not shown), a computer processor (not shown), a memory (not shown), and a program (not shown). Here, the program is loaded into the memory under the control of the computer processor to perform the operation (e.g., recognize the sound wave ID, transmit the recognized sound wave ID to the server 300, and notify the related content received from the server 300). A message, an image, and / or voice).

The following describes how sound wave IDs can be included in content.

The technique of inserting and extracting the sound wave ID into the main content may use the technique disclosed in the following patent applications.

For example, a technique of inserting an ID into a sound and a technique of extracting a content ID inserted into a sound include, for example, a patent application No. 10 filed by the inventor with the Korean Patent Office on April 12, 2012. The technique disclosed in -2012-0038120 (position estimation method and position estimation system for estimating the position of a mobile terminal using an acoustic system and an acoustic system used therein) can be used.

As another example, the inventor (patent application ID No. 10-2012-0078410 filed to the Korean Intellectual Property Office on July 18, 2012 (includes the ID in the sound signal), It is also possible to use a technique for recognizing and extracting ID included in the sound signal.

For another example, the technology disclosed in the patent application No. 10-2012-0053286 (the system for identification of the speaker and the position estimation system using the same) filed by the inventor with the Korean Intellectual Property Office on May 18, 2012 It is also possible to use a technique of including the ID in the signal and recognizing and extracting the ID included in the acoustic signal.

For another example, the technique disclosed in Patent Application No. 10-2012-0078446 (Method and Apparatus for Intimate User Calculation Using Proximity Information), filed with the Korean Patent Office on July 18, 2012, in the sound signal It is also possible to use a technique of including an ID and recognizing and extracting the ID included in the acoustic signal.

For another example, the technology disclosed in the patent application No. 10-2013-0107604 (Sound wave transmission and reception method using a time-varying frequency-based symbol and the device using the same) filed by the inventor of the Korean Patent Office on September 6, 2013 It is also possible to use (a technique for including digital information in a sound signal and recognizing and extracting digital information included in a sound signal).

Meanwhile, the technologies disclosed in each of the above-mentioned patent application numbers 10-2012-0038120, 10-2012-0078410, 10-2012-0053286, 10-2012-0078446, and 10-2013-0107604 are all It is combined with and treated as part of the present specification.

The technique of extracting the audible fingerprint sound wave ID corresponding to the characteristic of the sound included in the main content may use the technique disclosed in the following patent applications.

For example, Korean Patent Registration No. 10-1310943 (September 11, 2013) (System and method for providing content related information related to broadcast content), Korean Patent Registration No. 10-1313293 (September 24, 2013) It is possible to use the technique disclosed in (Additional Information Providing System and Method of Broadcasting Content), Korean Patent Registration No. 10-0893671 (April 9, 2009) (generation and matching of hashes of multimedia content).

On the other hand, the techniques disclosed in each of the above-mentioned specifications of Korean Patent Registration Nos. 10-1310943, 10-1313293, and 10-0893671 are all combined herein and treated as part of the present specification.

The server 300 may receive a sound wave ID from the mobile receiver 200, select the related content corresponding to the sound wave ID by referring to a related content DB (not shown) corresponding to the corresponding content for each sound wave ID, and select the selected sound content. The related content is transmitted to the mobile receiver 200.

In this embodiment, the server 200 may be, for example, an advertisement server that stores and manages advertisements, or may be a multimedia instant messenger (MIM) service server that transmits text and / or multimedia data to friends belonging to the same group. have.

When the server 200 is implemented as a MIM service server, the server 200 transmits related content to a client program that interacts with the MIM service server.

■ Mobile receiver to extract sound wave ID

2 is a view for explaining an embodiment of a mobile receiver.

Referring to FIG. 2, the mobile receiver 200 includes a microphone 201, an ADC 203, a processor 207, a memory 209, a storage device 211, a display unit 213, and a communication unit 215. can do.

According to an embodiment, the microphone 201 may detect a sound output from the content player 100. That is, the received sound may be converted into an electrical signal.

According to an embodiment, the ADC 203 converts sound, which is an analog sound wave signal detected by the microphone 101, into digital data. In one embodiment, the ADC 203 may convert a sound wave signal within a preset frequency range detected by a microphone or a maximum frequency range allowed by hardware into digital data.

According to an alternative embodiment, the ADC 203 may only convert the sound into digital data when the sound has an intensity above a predetermined predetermined threshold. In addition, the ADC 203 may convert the digital data only when the sound within the preset frequency range has a strength greater than or equal to a preset threshold.

The threshold may be, for example, a relative value such as signal-to-noise ratio (SNR) or an absolute value such as sound intensity.

The processor 207 is a device for interpreting and executing computer instructions, which may be a central processing unit (CPU) of a general-purpose general purpose computer. For a mobile device such as a smartphone, for example, it may mean an application processor (AP) and memory. The program loaded at 209 can be executed.

The memory 209 may be implemented as a volatile storage device such as random access memory (RAM), and an operating system (OS), an application, data, and the like required for driving the mobile device 200 are loaded into the memory 209. Can be. The storage device 211 is a nonvolatile storage device for storing data, and may be implemented as a device such as a hard disk drive (HDD), a flash memory, or a secure digital (SD) memory card.

The display unit 213 may display content by voice, video, and / or text to the user. For example, the display unit 213 may be implemented as a device such as a display unit, a sound output terminal, or a speaker connected thereto.

The communication unit 215 may be implemented as an apparatus for communicating with the outside. For example, the communication unit 215 may communicate with the server 100.

According to an embodiment, the associated content providing application 223 may be loaded and operated in the memory 209 under the control of the processor 207. The associated content providing application 223 receives the sound using the microphone 201 and the ADC 203, and when the sound wave ID is successfully extracted from the sound, the extracted sound wave ID is transmitted to the server 300 through the communication unit 215. To).

According to one embodiment, the associated content providing application 223, as an additional function, determines whether the sound sensed by the ADC 203 has an intensity above a threshold, and only when the sound has an intensity above a preset threshold. It may have a function for extracting the sound wave ID.

According to another exemplary embodiment, the related content providing application 223 may, as an additional function, determine whether the sound sensed by the ADC 203 has a strength greater than or equal to a threshold within a preset frequency range, and the sound may have a preset frequency. A sound wave ID may be extracted from the sensed sound only when the intensity is greater than or equal to a preset threshold within the range.

According to another embodiment, the operation of detecting and extracting the sound wave ID may be implemented as hardware other than the related content providing application 223. That is, the sound wave ID extractor 205, which is indicated by a dotted line in FIG. 2, is implemented in hardware, and the mobile receiver 200 may further include a sound wave ID extractor 205. In this case, the sound wave ID extracting unit 205 extracts the sound wave ID from the digital data converted by the ADC 203, and the associated content providing application 223 communicates the sound wave ID extracted by the sound wave ID extracting unit 205 to the communication unit. It transmits to the server 300 through the (215).

According to an embodiment of the present invention, at least one of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the associated content providing application 223 is deactivated. When the schedule or the predetermined event occurs, it is activated to perform its own operation.

For example, the microphone 201 and the ADC 203 may be activated when a schedule or a predetermined event occurs to perform their operation. In this example, the microphone 201 and ADC 203 are deactivated and are components that are activated when a schedule or predetermined event occurs.

For example, when the related content providing application 223 is implemented to have a function of determining whether the sound sensed by the microphone 201 has a strength above a threshold within a preset frequency range, the microphone 201 When the ADC 203 is activated, the associated content providing application 223 is also activated to perform an operation.

For example, if the related content providing application 223 is implemented to have a function of extracting a sound wave ID, when the microphone 201 and the ADC 203 are activated, the related content providing application 223 may also have the same function. Is activated to perform the first operation.

For example, in the embodiment in which the sound wave ID extracting function is performed by the sound wave ID extracting unit 205, the associated content providing application 223 may be activated only when the sound wave ID is extracted successfully. The activated related content providing application 223 transmits the extracted sound wave ID to the server 300, receives related content corresponding to the sound wave ID from the server 300, and transmits the related sound content to the voice and / or video through the display unit 213. I can display it. Alternatively, the activated related content providing application 223 may search for and select the related content corresponding to the extracted sound wave ID, and display the related content by voice and / or image through the display unit 213. In this case, the related content database may be loaded in the memory 209, and the related content providing application 223 may search the related content database to select related content corresponding to the sound wave ID.

For example, the microphone 201 is deactivated and is activated when a schedule or event occurs. If the microphone 201 detects a sound, for example, when the microphone 201 detects a sound belonging to a preset band, Other components (eg, ADC 203, associated content providing application 223, sound wave ID extractor 205, etc.) may be activated simultaneously or sequentially.

For example, when sound is detected by the microphone 201, a component (eg, an associated content providing application) that determines whether the sensed sound has an intensity above a preset threshold may be activated. When it is determined that the detected sound has an intensity equal to or greater than a preset threshold, a component (for example, a sound wave ID extracting unit) for extracting a sound wave ID is activated.

For a more detailed description of the schedule, please refer to the embodiments described with reference to FIG. 4. For operations that are activated at regular or irregular periods according to a schedule, refer to the embodiments described with reference to FIGS. 9 to 11, and to activate operations where a predetermined event is generated with reference to 12 to 14. See the examples.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be deactivated. Then, depending on the schedule created with reference to the main content schedule, it may be activated at regular intervals or at irregular intervals. For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may have a main content. Only during the playback time, it can be activated at a predetermined cycle. Alternatively, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the associated content providing application 223 may play the main content. During the period of time for which the main content is not played, the period may be longer than the period above.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may include a sound wave ID. Depending on the schedule created with reference to the schedule, it may be activated at regular intervals or at irregular intervals. For example, it can be activated at a time when sound wave ID can be extracted.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be preset. If the sound wave ID is failed to be extracted, but it is determined that the sound wave ID exists, the sound wave ID may be activated at a shorter period than the preset period. After that, if the ID is extracted, it can be performed in the original period.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be preset. Cycles, the following formula

Threshold> intensity of detected sound> (α × threshold), where α is less than 1

If it is satisfied, may be performed in a shorter period than the period set above. That is, in one embodiment, the inventor of the present invention, if the sensed intensity of the sound is smaller than the threshold, but close to the threshold, it is determined that there is a possibility that there is a sound with a sound wave ID, satisfies the above formula In this case, the components are configured to be activated in a shorter period than the original period.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be predetermined. When an event occurs, it can be activated. Here, the event may mean, for example, when a specific application is executed, when a display unit of the mobile receiver 200 is activated or deactivated, or when the mobile receiver 200 receives a push message from the outside.

Here, the specific application may be an application that uses a microphone only for itself, but this may be another application as an example.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be preset. While being performed at intervals, when a predetermined event occurs, it may be configured not to be executed. Then, when the predetermined event ends, it is activated according to the original cycle.

For example, at least one component of the microphone 201, the ADC 203, the sound wave ID extracting unit 205, the display unit 213, the communication unit 215, and the related content providing application 223 may be predetermined. When an event occurs, processor 207 communicates the event to an operating system (OS) (not shown) that may be loaded in memory 209, which the mobile receiver 200 has, and the operating system may The occurrence is transmitted to the filters (not shown), which may be loaded in the memory 209, respectively, corresponding to the applications operating in the mobile receiver 200, and each of these filters is preset. Therefore, the occurrence of the event is not communicated to the applications corresponding to it or not. Please refer to the embodiments described with reference to FIG. 14 for a more detailed description of these operations.

3 is a view for explaining another embodiment of a mobile receiver.

Referring to FIG. 3, the mobile receiver 400 includes a microphone 401, a low power acoustic strength measurement unit 402, an ADC 403, a sound wave pattern recognition unit 404, a processor 407, and a memory 409. The storage device 411 may include a display unit 413 and a communication unit 415.

Comparing the embodiment of FIG. 3 with the embodiment of FIG. 2, it can be seen that the sound wave ID extractor 205 of FIG. 2 is divided into a low power acoustic intensity measurement unit 402 and a sound wave pattern recognition unit 404. have.

According to an embodiment, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be arranged in a schedule. It may be activated according to, or activated when a predetermined event occurs. The schedule may be written to be activated at regular or irregular intervals. For a more detailed description of the schedule, please refer to the embodiments described with reference to FIG. 4. For operations that are activated at regular or irregular intervals according to a schedule, refer to the embodiments described with reference to FIGS. 9 to 11, and to the operations for which a predetermined event is generated with reference to FIGS. 12 to 14. See examples.

For example, the microphone 401, the low power acoustic strength measurement unit 402, and the ADC 403 may be activated when a schedule or a predetermined event occurs to perform their operation. In this example, the microphone 401, the low power acoustic strength measurement unit 402, and the ADC 403 are inactive components that are activated when a schedule or predetermined event occurs.

For example, the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, and the sound wave pattern recognition unit 404 are activated when a schedule or a predetermined event occurs to perform their operation. can do.

For example, the associated content providing application 423 may be activated only when extraction of the sound wave ID is successful. The activated related content providing application 423 transmits the extracted sound wave ID to the server 300, receives related content corresponding to the sound wave ID from the server 300, and transmits the related content to the voice and / or video through the display unit 413. I can display it. Alternatively, the activated related content providing application 423 may search for and select related content corresponding to the sound wave ID, and display the related content by voice and / or video through the display unit 413. In this case, a related content database may be loaded in the memory 409, and the related content providing application 423 may search the related content database and select related content corresponding to the sound wave ID.

For example, the microphone 401 is deactivated and is activated when a schedule or event occurs. If the microphone 401 detects a sound, for example, when a sound belonging to a preset band is detected. Other components (e.g., ADC 403, associated content providing application 423, low power acoustic strength measurement unit 402, or sound wave pattern recognition unit 404) may be activated simultaneously or sequentially. have.

For example, when sound is detected by the microphone 401, the low power sound intensity measuring unit may be activated to determine whether the sensed sound has an intensity above a preset threshold. In addition, when it is determined that the detected sound has an intensity greater than or equal to a preset threshold, the sound wave pattern recognition unit 404 extracting the sound wave ID may be activated.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 is inactive. According to the schedule created with reference to the main content schedule, it may be activated at regular intervals or at irregular intervals. For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may include a main content. Only during the playback time, it can be activated at a predetermined cycle. Alternatively, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be configured to play the main content. During the period of time for which the main content is not played, the period may be longer than the original period.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may include a sound wave ID. Depending on the schedule created with reference to the schedule, it may be activated at regular intervals or at irregular intervals. For example, it can be activated at a time when sound wave ID can be extracted.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be preset. If the sound wave ID is failed to be extracted but it is determined that the sound wave ID exists, it may be activated in a shorter period than the original period. After that, if the ID is extracted, it can be performed in the original period.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be preset. Cycles, the following formula

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be predetermined. When an event occurs, it can be activated. Here, the event may mean, for example, when a specific application is executed, when a display unit of the mobile receiver 200 is activated or deactivated, or when the mobile receiver 200 receives a push message from the outside.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be preset. While being performed at intervals, when a predetermined event occurs, it may be configured not to be executed. Then, when the predetermined event ends, it is activated according to the original cycle.

For example, at least one component of the microphone 401, the low power acoustic strength measurement unit 402, the ADC 403, the sound wave pattern recognition unit 404, and the associated content providing application 423 may be predetermined. When an event occurs, the processor 407 informs the fact that the event occurred to an operating system (OS) (not shown) that may be loaded in the memory 409 of the mobile receiver 400, and the operating system may be configured to send the event. The occurrence is transmitted to the filters (not shown), which may be loaded in the memory 409, respectively, corresponding to the applications operating in the mobile receiver 400, and each of these filters is preset. Therefore, the occurrence of the event is not communicated to the applications corresponding to it or not. Please refer to the embodiments described with reference to FIG. 14 for a more detailed description of these operations.

According to one embodiment, the low power acoustic strength measurement unit 402 consumes low power but is continuously activated, and the acoustic wave pattern recognition unit 404 is more power than the low power acoustic intensity measurement unit 402. It is usually consumed but inactive and activated according to a schedule or when a predetermined event occurs (ie intermittently). The low power acoustic intensity measuring unit 402 may detect a sound within a specific frequency range of the sound, and may continuously determine whether the detected sound intensity has an intensity above a preset threshold.

In the present embodiment, the low power acoustic intensity measuring unit 402 determines whether the sound input through the microphone 401 has an intensity greater than or equal to a preset threshold within a preset frequency range. If the intensity has an intensity greater than or equal to the threshold value, the low power acoustic intensity measurement unit 402 may activate the ADC 403 and the sound wave pattern recognition unit 404 that have been deactivated for low power operation. In this case, the activated ADC 403 may convert the sound into digital data and transmit the sound to the activated sound wave pattern recognition unit 404.

According to an embodiment, the sound wave pattern recognition unit 404 receives the sound in the form of digital data from the ADC 403, and extracts the sound wave ID if the sound wave ID is included in the received sound. At this time, when the processor 407 is inactivated to operate at low power, the sound wave pattern recognition unit 404 activates the processor 407. In the present embodiment, the sound wave pattern recognition unit 404 provides the extracted sound wave ID to the related content providing application 423.

According to another embodiment, the sound wave pattern recognition unit 404 receives the sound in the form of digital data from the ADC 403, compares the received sound with at least one stored sound wave ID, and transmits the sound from the ADC 403. After determining whether the sound wave ID can be extracted from the received sound, if it is determined that the sound wave ID can be extracted, the sound wave ID is directly extracted. According to an alternative embodiment, the sound wave pattern recognition unit 404 determines whether a sound wave ID can be extracted from the sound received from the ADC 403, and if it is determined that the sound wave ID can be extracted, the ADC 403. ) Provides the sound received from the related content providing application 423, the related content providing application 423 may extract the sound wave ID.

On the other hand, a technique for implementing the operation of detecting and extracting the sound wave ID in a separate hardware may use the technology disclosed in the following patent applications.

For example, a technique of extracting sound waves modulated into non-audible sound at low power is disclosed, for example, in Korean Patent Application No. 10-2013-0141504 (Low, Nov. 20, 2013). Power sound wave receiving method and a mobile device using the same) can be used.

On the other hand, the techniques disclosed in the specification of the above-mentioned patent application No. 10-2013-0141504 are all incorporated herein and treated as part of the present specification.

Meanwhile, the memory 409 may be loaded with a DMB application or a video player operating under the control of the processor 407, or a codec 221 essential for playing contents through the same. The DMB application, the video player, or the codec 221 may be stored in the storage 411 and loaded and operated in the memory 409 by a user's command. Although not shown, when a message arrives from the server 300, a message processing program (not shown) may be loaded into the memory 409 to display the message through the display unit 413.

Meanwhile, components that are not described in the embodiment of FIG. 3 are the same as or similar to the operations of the elements having similar reference numerals in the embodiment of FIG. 2, and thus, the description of the embodiment of FIG.

■ Low Power Sound Wave ID Extraction

In the present specification, for the purpose of describing the present invention, the term ?? acoustic recognition module ?? is a microphone, an ADC, a sound wave ID extractor, an associated content providing application, a low power acoustic strength measurer, and a sound wave pattern included in a mobile receiver. Means at least one component of the recognition unit, these at least one component detects or extracts a sound wave ID, or activates a component that detects or extracts a sound wave ID.

According to an embodiment of the present invention, in order to operate at a low power, the acoustic recognition module is activated according to a schedule or when a predetermined event occurs (that is, intermittently), and when activated, detects sound and detects sound wave ID. To perform the extraction operation. In this case, activation means that the acoustic recognition module performs only the minimum functions to minimize the power consumption, and executes the necessary hardware module, runs the necessary software, or inputs necessary information in order to be able to detect and extract the sound wave ID. It is a state that can perform various activities such as requesting from OS.

Intermittent activation for operating at low power is shown in the specific embodiment in FIGS. 9-14.

Intermittently activated to operate at low power, for example, (a) the acoustic recognition module may be activated and operated in a 10-minute period, which is shown in FIG. For another example, (b) the acoustic recognition module is activated and operated at irregular intervals by utilizing the temporal feature of extracting the sound wave ID from the main content or by considering the broadcast time of the main content including the sound wave ID to be extracted. To be shown in FIG. 10. As another example, (c) in consideration of the result of the previous attempt to receive the sound wave ID, the acoustic recognition module may be activated and operated at irregular intervals, which is shown in FIG. As another example, (D) the acoustic recognition module may be activated and operate at irregular intervals according to an event issued in the mobile receiver, which is illustrated in FIGS. 12 to 14. 14 shows how an event occurring inside the mobile receiver is delivered to the acoustic recognition module.

(A) Periodic activation

9 a) shows that the acoustic recognition module is activated at the same time in preparation for the time when the main content is output. 9B shows that the acoustic recognition module is activated at the second half or the completion time of the main content in preparation for the time when the main content is output. 9C shows that the acoustic recognition module is activated a plurality of times (for example, twice) with a predetermined period in preparation for the time when the main content is output.

(B) Activation according to the broadcast time of the contents and the available time of sound wave ID extraction

According to an embodiment, when the acoustic recognition module is periodically activated, the activation cycle may be performed by considering the temporal position of the sound wave ID extracted from the main content output by the content player 100 and the degree of extraction of the sound wave ID. Can be determined. For example, the sound wave ID in the content may be repeatedly extracted for two consecutive minutes, after which the entire four-minute pattern that is difficult to extract may be continuously repeated throughout the content. At this time, if the sound recognition module detects whether the sound wave ID can be extracted at a maximum of 2 minutes, the sound wave ID can be intermittently attempted to extract the sound wave ID while extracting the sound wave ID that can be extracted from the content in a short time. Reduce consumption

According to another embodiment, when the broadcast date and time of the content output from the content reproducing apparatus 100 can be known in advance, the acoustic recognition module may be activated at a shorter period at the corresponding time when the content is broadcast. For example, assume that the period of the acoustic recognition module is set to 10 minutes. At this time, if the specific content of the TV is aired for 1 hour every Saturday at 6 pm, and the sound wave ID can be extracted from the content, the acoustic recognition module every five minutes every Saturday from 6 pm to 7 pm instead of 10 minutes It can work. Alternatively, the acoustic recognition module may be continuously deactivated and may be operated at a cycle of only one hour from 6 pm to 7 pm every Saturday.

Now, the two embodiments described above will be described with reference to FIG. 10. The

mobile receiver

200 or 400 knows the time at which the sound wave ID can be extracted, that is, the reproduction time of the sound wave ID1, ID2, and ID3 of FIG. This time may be received by the acoustic recognition module of the

mobile receiver

200 or 400 through the server. When the playing time of the sound wave ID1 is reached, the acoustic recognition module is activated to extract the sound wave ID1. If the sound wave ID1 extraction fails, the acoustic recognition module may determine that the user of the

mobile receiver

200 or 400 is not watching or listening to the main content 1. If the extraction of the sound wave ID1 is successful, the

mobile receiver

200 or 400 may receive and display the related content corresponding to the sound wave ID1 from the server 300. Similarly, if the acoustic recognition module is activated in accordance with the playback time of sound wave ID2 and ID3 and the sound wave ID is extracted and succeeded, the

mobile receiver

200 or 400 receives related content corresponding to the sound wave ID from the server 300. I can display it. In this manner, the

mobile receiver

200 or 400 may activate the acoustic recognition module at irregular intervals only at a time when it is predicted that the acoustic ID may be extracted, and display the acoustic ID and display related contents at low power.

(C) Activation according to the previous sound wave ID extraction result

Referring to FIG. 11, the

mobile receiver

200 or 400 activates an acoustic recognition module in accordance with the originally planned period. At this time, the acoustic recognition module determines that the sound wave ID may be extracted (hereinafter, described as existence determination), but the extraction may fail. That is, for example, when the extracted sound wave ID has a hamming distance from the specific sound wave ID existing on the related content DB or less than a preset value, the sound wave ID may fail to be extracted, but it may be assumed that there exists. Alternatively, the presence of the sound wave ID itself may be checked and the CRC (Cyclic Redundant Check) information that may check the extraction failure may be included and used. Alternatively, to determine whether a sound wave ID exists, a signal-to-noise ratio (SNR) may be measured to determine whether the sound wave ID is greater than or equal to a preset threshold value, and thus, the sound wave ID may be estimated. Suppose that the existence of sound wave ID is confirmed in the above manner, but it is determined that extraction has failed. At this time, the acoustic recognition module cancels the activation of the planned acoustic recognition module after period 3, which is a period to be originally executed, in order not to miss a time slot for extracting sound wave ID of the main content, and activates with a period 4 that is shorter than the period 3 The period after period 4 can be adjusted to the original period. Therefore, by adjusting the period with the result according to the sound wave ID extraction attempt, it is possible to operate at low power while maintaining the existing frequency of activation while increasing the probability of sound wave ID extraction.

(D) activation according to mobile receiver internal events;

According to an embodiment, the acoustic recognition module may be activated and operate at irregular intervals according to an event issued in the

mobile receiver

200 or 400. This will be described with reference to FIGS. 12 and 13.

12 is a view for explaining that the period of the acoustic recognition module is adjusted according to an event in which another application using the

microphone

201 or 401 that is a shared resource inside the

mobile receiver

200 or 400 is activated.

Multiple applications may run simultaneously on the

mobile receiver

200 or 400. In this case, the

microphone

201 or 401, which is part of the

mobile receiver

200 or 400, is a resource shared by several applications, and used by another application until the first-come first-served (FCFS) method, that is, the first application is stopped. Can be shared in an impossible way. In such a case, the intermittently activated acoustic recognition module may cause the use of the

microphone

201 or 401 to fail when another application attempts to use the

microphone

201 or 401. Therefore, the acoustic recognition module delays activation until an application is terminated if an application capable of using the

microphone

201 or 401 is detected or if an attempt to use the

microphone

201 or 401 fails. The total number of activations can be reduced and operate at low power without interrupting the operation of the application using 201 or 401.

For example, as shown in FIG. 12, the application 1 included in the acoustic recognition module may be loaded in the

memory

209 or 409 of the

mobile receiver

200 or 400 at time t1. Although the acoustic recognition module was scheduled to be activated according to the preset period 5, the acoustic recognition module was notified of the execution of the application 2 having a high probability of using the

microphone

201 or 401 before t2 hours, or the acoustic recognition module at the time t2. Before executing sound wave ID extraction, the currently running application information may be obtained from the operating system to detect application 2 (the method of detecting or being notified at this time will be described in detail later with reference to FIG. 14). When the application 2 is running, the acoustic recognition module may cancel the scheduled activation at time t2 and newly set the next period to be activated after t2. The newly set period may be a period for activating the acoustic recognition module at a time doubled from t1 to period 5. In addition, as shown in FIG. 12, the newly set period may be modified at a time when the application 2 is notified as a period in which the time t3 that is notified of the termination of the application 2 is adjusted to be a time when the acoustic recognition module is activated. .

As another specific application, for example, when the camera application is activated, the application may exclusively use a part of the acoustic recognition module for video recording. At this time, the acoustic recognition module may stop the periodic activation so that the camera application can exclusively use a part of the acoustic recognition module, and may be activated when the camera application is terminated.

FIG. 13 is a diagram illustrating that the period of the acoustic recognition module is adjusted according to an event in which another application using the

microphone

201 or 401 that is a shared resource is activated in the

mobile receiver

200 or 400.

Assume that the acoustic recognition module of the

mobile receiver

200 or 400 is activated at time t4 to attempt sound wave ID extraction. Thereafter, the originally planned period may be the period 7 and the acoustic recognition module will be activated again at time t6. However, in this case, the operating system may notify the acoustic recognition module that the application 4 using the

microphone

201 or 401 is executed at time t5 (the method of detecting or receiving the notification may be described in detail later with reference to FIG. 14). do). If the operating system provides sharing of the use of the

microphone

201 or 401, the acoustic recognition module that is notified of the execution of the application 4 is the

microphone

201 or 401, the

ADC

203 or 403, the

processor

207 or 407, or the like. Application 4 can use and share the hardware and software resources needed for sound wave ID extraction. For example, the operating system may simultaneously provide data, which the sound coming into the

microphone

201 or 401, is converted into digital data through the

ADC

203 or 403 to the acoustic recognition module as well as the application 4, thereby providing data on the sound from both sides. I can receive it. Through the support of such an operating system, the acoustic recognition module is separately activated to extract the sound wave ID with lower power than the total power consumption of the

mobile receiver

200 or 400 as compared to attempting to extract the sound wave ID. In addition, when such activation is activated, the acoustic recognition module may operate at low power by canceling the schedule of activation at the originally scheduled time t6.

As a specific application, for example, when a telephony application is activated, this application basically utilizes a

microphone

201 or 401 and an

ADC

203 or 403 and activates the

processor

207 or 407, thereby simultaneously acknowledging the acoustic recognition module. When enabled, sound wave ID extraction is possible with low power. As another example, when a video player, a DMB application, or a

codec

221 or 421 operates, the

mobile receiver

200 or 400 may assume that the content player also serves as a content player. The recognition module can be activated.

12 and 13, it has been described that the acoustic recognition module can estimate whether a particular application uses the

microphone

201 or 401. Such estimation can be performed by viewing the hardware usage permission information of each application from the operating system. You can check it. At this time, the list of running applications can also be obtained by requesting from the operating system.

12 and 13 illustrate that the acoustic recognition module has an irregular period according to an event inside a mobile receiver in which a specific application is executed or terminated, the type of event is not limited thereto. For example, FIGS. 12 and 13 may not only execute or terminate a specific application, but also may switch an application from the background mode to the foreground mode or from the foreground mode to the background mode. For another example, FIG. 12 may be an event in which one or more of internal components of the

mobile receiver

200 or 400 are activated, but not in a specific application, and FIG. 13 may be an event inactive. In more detail, this may be an event in which the display of the display unit is activated or deactivated. As another example, the event may be that the

mobile receiver

200 or 400 receives a push message from a push server.

FIG. 14 is a diagram for describing methods of receiving an event of a

mobile receiver

200 or 400 in order to operate the acoustic recognition module at low power as shown in FIGS. 12 and 13.

FIG. 14A illustrates a method of being notified when a specific event occurs to applications running by an operating system. 14 b) illustrates a method of notifying that an application occurs to an application corresponding to a sound recognition module by a specific application. 14 c) illustrates how the acoustic recognition module is notified that a push server sends a message that allows a message to be received from the server at low power.

14a) will be described as an example.

The

mobile receiver

200 or 400 may include the function of receiving a call. In this case, when a call is received, the operating system operates an application that receives a call, and transmits a call reception event to a filter of each application so that another application may respond to the incoming call. For example, application 5 may be an application that handles incoming calls, and application 6 may be an application that is playing music. When a call comes in, the operating system forwards the incoming call to each filter in the application. Since application 6 must stop playing music when a call is received, the filter of application 6 may be set to pass a call reception event. The filter of the application 5 may be set to pass a call reception event since the user interface must be prepared and a phone call can be made when a call is received. Assume that the digital data input through the

microphone

201 or 401 in the operating system can be delivered simultaneously to multiple applications through the

ADC

203 or 403. In this case, the acoustic recognition module may receive sound digital data and perform sound wave ID extraction. Therefore, the application corresponding to the application of the acoustic recognition module can be configured to pass a call reception event to the filter. In other words, the event required to operate at low power among the events sent by the operating system is set to pass through the filter, so that the acoustic recognition module can extract the sound wave ID at low power.

14 b) will be described as an example.

When the application 2 of FIG. 12 and the application 4 of FIG. 13 notify the acoustic recognition module of activation and termination of the application, the acoustic recognition module may operate at low power. Assume that application 7 of FIG. 14B) is application 2 of FIG. 12 or application 4 of FIG. In this case, the application 7 may deliver an event to another operating system to the operating system. This event may be an activation event of application 2 of FIG. 12 or an end event of application 4 of FIG. 13. The event is passed to the operating system, which can then pass the event to all application filters. At this time, the filter of the application included in the acoustic recognition module may be set to pass through the activation or termination event. In this way, the acoustic recognition module can be notified of activation or termination of a particular application. In another example, a method of receiving an event indicating the use or termination of the hardware or software that the acoustic recognition module intends to use rather than the activation or termination of an application may be used.

14b) will be described as another example.

The broadcast or playback schedule of the main content 1 and the main content 2 of FIG. 10 may be known in advance. In this case, the application 7 of FIG. 14B may be the same application as the sound wave ID extraction app. The sound wave ID extraction app may receive a broadcast or playback schedule of main content 1 and 2 from a server and register the corresponding time as an alarm of the operating system. When the registered alarm reaches the corresponding time, the registered alarm is delivered to the filter of each application. Application 5 functions independently of this alarm event, so Application 5's filter does not forward this event to Application 5. However, the filter of the application included in the acoustic recognition module (filter of the sound wave ID extraction app of FIG. 14B) may receive an alarm event and start sound wave ID extraction. In the same way, by registering the time when the sound wave ID of the main content 1 and 2 as an alarm of the operating system, the application included in the acoustic recognition module can end the sound wave ID extraction.

14c) will be described as an example.

14 c), a

mobile receiver

200 or 400 receives a message from a push server and delivers the message to an application through an operating system. Assume that the application receiving the push message is Application8. Specifically, assume application 8 is a MIM service. At this time, the

mobile receiver

200 or 400 receives a push message from the server and delivers it to the operating system, which delivers it to the application filters. The filter in Application 8 may be set to pass through a push message to perform the MIM function. Application 9 operates independently of the push message, and the filter of Application 9 may be set not to pass through the push message. The

mobile receiver

200 or 400 may receive the push message while activating only the minimum hardware and software for receiving the push message in order to reduce power consumption. In this case, the MIM service that receives the push message from the operating system takes an appropriate action for the push message. The filter in the application portion of the acoustic recognition module may be set to pass this push message. In such a case, the acoustic recognition module activates the acoustic recognition module when various components of the

mobile receiver

200 or 400 such as an AP, a display, etc. are activated for the processing of a push message, and thus, among the components of the

mobile receiver

200 or 400. The hardware and software required for the acoustic recognition module can be shared with Application8. By sharing such hardware and software, the acoustic recognition module can operate at low power in terms of the entire

mobile receiver

200 or 400.

■ How to provide related content that has a related content DB on the server

Assuming that the related content providing method according to the present embodiment is applied to the system of FIG. 1, the embodiment of FIG. 4 will be described.

In the related content providing method according to the present embodiment, the step of outputting the main content by the content reproducing apparatus 100 (S101), the step of activating the acoustic recognition module of the mobile receiver (S103), and extracting a sound wave ID (S105). ), The server 300 receives the sound wave ID extracted in step S105, retrieves and selects the related content corresponding to the sound wave ID (S107), and the mobile receiver 200 displays the related content selected in the step S107. It may include the step (S109).

In step S103, the acoustic recognition module is activated according to a schedule or activated when a predetermined event occurs. The schedule may be written such that the acoustic recognition module is activated at regular and / or irregular intervals. The schedule may be, for example, created with reference to a main content schedule—a schedule indicating the times when the main content is played—or a sound wave ID schedule—a schedule indicating the times from which the sound ID may be extracted, but this is merely illustrative. The contents schedule or sound wave ID schedule does not necessarily have to be created with reference.

According to an embodiment of the present disclosure, the schedule may be written to activate the acoustic recognition module at regular intervals or to be activated at irregular intervals. The example in which the acoustic recognition module is activated according to the schedule may be the embodiments described with reference to FIGS. 9 to 11, and it will be easily understood by those skilled in the art.

An example in which the acoustic recognition module is activated when there is an event may be the embodiments described with reference to FIGS. 12 to 14, but it will be easily understood by those skilled in the art.

When the related content providing application is used as part of the acoustic recognition module, the related content providing application operating in the memory 209 of the mobile receiver 200 transmits the extracted sound wave ID to the server 300. The server 300 searches for the related content corresponding to the received sound wave ID (S107). Then, the server 300 transmits the selected related content to the mobile receiver 200, and the mobile receiver 200 transmits the related content. Display to the user (S109).

■ How to provide related content that has related content DB in mobile receiver

Assuming that the related content providing method according to the present embodiment is applied to the system of FIG. 1, the embodiment of FIG. 5 will be described based on differences from the embodiment of FIG. 3.

In the related content providing method according to the present embodiment, a step of outputting main content by the content reproducing apparatus 100 (S201), a step of activating an acoustic recognition module of the mobile receiver (S203), and extracting a sound wave ID (S207) ), The mobile receiver searching for and selecting the related content corresponding to the sound wave ID extracted in step S205 (S209), and the mobile receiver 200 displaying the related content selected in step S209 (S211). Can be.

Meanwhile, the related content providing method according to the present embodiment may further include a step (S205) in which the server transmits update data for the related content DB to the mobile receiver, and the mobile receiver transmits the update data from the server. You can update the related content DB that you receive and store.

For example, the related content providing application operating in the memory 209 of the mobile receiver stores the related content DB received from the server 300 in the storage device 211 or updates the previously stored DB. For example. The related content providing application operating in the memory 209 may search for and select the related content DB corresponding to the sound wave ID (S209), and display the selected related content through the display unit 213 (S211).

The step S203 of activating the acoustic recognition module of the mobile receiver may be activated according to a schedule, or may be activated when a predetermined event occurs, similarly to the activation step described with reference to FIG. 4.

Meanwhile, in the embodiment of FIG. 5, unlike the embodiment of FIG. 4, the mobile receiver 200 directly selects the related content corresponding to the sound wave ID. To this end, the server 300 may send some or all of the related content DB to the mobile receiver 200 to update some or all of the related content previously stored in the mobile receiver 200.

The time when the server 300 transmits the related content DB is not limited to the time shown in FIG. 5. When the server 300 sends the related content DB to update the related content DB previously stored in the mobile receiver 200, the server 300 is not limited to a specific time.

■ How the mobile receiver acts as a content player

In the system of FIG. 1, it is assumed that the mobile receiver 200, not the content player 300, outputs main content, and the mobile receiver 200 is implemented to recognize sound. Assuming that the content providing method according to the present embodiment is applied to the system of FIG. 1, the embodiment of FIG. 6 will be described based on differences from the embodiment of FIG. 4.

In the related content providing method according to the present embodiment, the mobile receiver 200 outputs main content (S301), the acoustic recognition module of the mobile receiver is activated (S305), and the sound wave ID is extracted (S307). The server may include searching for and selecting related content corresponding to the sound wave ID extracted in step S307 (S309), and displaying, by the mobile receiver 200, the related content selected in step S309 (S311). .

Step S307 can be implemented, for example, in the following manner.

First, a method in which the microphone 201 detects a sound output through a speaker (not shown) that is part of the display unit 213 provided by the mobile receiver 200, and the sound recognition module extracts an ID from the detected sound.

Second, a method of extracting a sound wave ID by receiving the acoustic recognition module received from the internal operating system digital data before being output through a speaker (not shown) which is part of the display unit 213 provided in the mobile receiver 200.

Third, the input terminal of the microphone 201 of the headset (not shown) connected to the mobile receiver 200 is connected to the small sound output terminal of the headset (not shown), and is transferred to the small speaker of the headset (not shown). How the acoustic recognition module extracts the ID from the signal.

The acoustic recognition module transmits the sound wave ID extracted in step S307 to the server 300 (S307). The server 300 searches for and selects the related content corresponding to the sound wave ID by referring to the related content DB-server 300 or stored in a separate storage device (not shown) (S309). The server 300 transmits the selection result to the mobile receiver 200, and the mobile receiver 200 displays the received selection result.

In the system of FIG. 1, it is assumed that the mobile receiver 200, not the content player 100, outputs main content, and the mobile receiver 200 is implemented to recognize sound. It is assumed that the related content providing method according to the present invention is applied, and the embodiment of FIG. 7 will be described based on differences from the embodiment of FIG. 4.

In the related content providing method according to the present embodiment, the content reproducing apparatus 100 outputs main content (S401), a step of activating an acoustic recognition module of the mobile receiver (S403), and extracting a sound wave ID (S407). ), The mobile receiver searching for and selecting the related content corresponding to the sound wave ID extracted in step S407 (S409), and the mobile receiver 200 displaying the related content selected in step S409 (S411). Can be.

Meanwhile, the related content providing method according to the present embodiment may further include a step (S405) in which the server transmits update data for the related content DB to the mobile receiver, and the mobile receiver transmits the update data from the server. You can update the related content DB that you receive and store.

The above-described step S407 can be implemented by the same method as the above-described step S307.

The step S403 of activating the acoustic recognition module of the mobile receiver may be activated according to a schedule, or may be activated when a predetermined event occurs, similarly to the activation step described with reference to FIG. 4.

In the embodiment of FIG. 7, unlike the embodiment of FIG. 4, the mobile receiver 200 directly selects the related content corresponding to the sound wave ID. To this end, the server 300 may send some or all of the related content DB to the mobile receiver 200 to update some or all of the related content previously stored in the mobile receiver 200.

The time when the server 300 transmits the related content DB is not limited to the time shown in FIG. 7. That is, when the server 300 sends the related content DB to update the pre-stored related content DB in the mobile receiver 200, the time when the server 300 transmits the related content DB is not limited to a specific time. .

■ Location of sound wave ID that can be extracted from content

8 is a diagram for explaining main content.

8A illustrates an example of main contents, each of which is composed of a combination of an image and a sound, and a sound wave ID may be extracted at a corresponding time of the main content.

For example, assuming that main content such as a) of FIG. 8 is output in the embodiment of FIG. 4, the operation thereof will be described.

The content player 100 outputs main contents (main contents 1, 2, and 3) as time passes, and extracts sound wave IDs from the contents. Sound wave ID1 can be extracted from sound when main content 1 is output. Sound wave ID2 can be extracted from sound when main content 2 is output. Sound wave ID3 can be extracted from sound when main content 3 is output.

The mobile receiver 200 sequentially extracts each sound wave ID, transmits the sound wave IDs to the server 300, and receives and displays related contents corresponding to the sound wave IDs from the server 300.

Meanwhile, the mobile receiver 200 may perform extraction of the sound wave ID according to a schedule or when a predetermined event occurs (that is, intermittently). As it is the same as the above-described embodiments will not be described in detail.

FIG. 8B illustrates another example of main contents output through the content reproducing apparatus. Each of the main contents includes a combination of an image and a sound, and a sound wave ID may be extracted from the sound.

In case of b) of FIG. 8, the present invention differs from a) of FIG. 8 in that there are contents capable of extracting two different sound wave IDs.

For example, assuming that main content such as b) of FIG. 8 is output in the embodiment of FIG. 6, the operation thereof will be described.

The mobile receiver 200 outputs main contents (main contents 4, 5, 6) over time, and may extract sound wave IDs from these main contents, respectively. Sound wave ID1 can be extracted from sound when main content 4 is outputted. Sound waves ID2 and ID3 can be extracted from sound when main content 5 is outputted. Sound wave ID4 can be extracted from sound when main content 6 is outputted.

The mobile receiver 200 sequentially extracts each sound wave ID, transmits these IDs to the server 300, and receives and displays related contents corresponding to the sound wave IDs from the server 300.

■ How to output related content without immersion

According to an embodiment of the present invention, the time for displaying the related content through the mobile receiver may be a time that does not require the immersion of the viewer or the listener in the main content such as an advertisement or an ending credit in the main content. For example, when the main content is a drama and the related content is indirect advertising product information from the drama, the mobile receiver displays the related content on the mobile receiver, even if the sonic ID from which the related content can be obtained is located in the middle of the drama. Doing can be postponed to the point when the drama ends and the ending credits are displayed.

Meanwhile, a computer-readable recording medium having recorded thereon a program for executing the steps constituting the method described above with reference to FIGS. 4 to 7 may be provided.

■ How to provide related content

Assuming that the related content providing method according to an embodiment of the present invention is applied to the system of FIG. 1, the related content providing method according to an embodiment of the present invention will be described.

Referring to FIG. 15, the related content providing method according to an exemplary embodiment of the present disclosure may include: detecting, by the mobile receiver 200, a sound output from a content player for playing content (S501); Converting the sensed sound into digital data by the mobile receiver 200 (S503); Extracting, by the mobile receiver 200, sound wave ID from the digital data (S505); Searching for and selecting related content corresponding to the extracted sound wave ID (S507); And displaying, by the mobile receiver 200, the selected related content as an image and / or a voice (S509).

In addition, the method for providing a related content according to an embodiment of the present invention further includes the step of determining whether the sound sensed in step S501 has a predetermined threshold value or more; the step of determining the step S501 and It may be performed between steps S505.

For example, the determining may be performed between steps S501 and S503. In this case, the step S503 of converting the sound sensed in step S501 into digital data may be performed when it is determined that the sound sensed in step S501 has an intensity greater than or equal to a preset threshold.

In addition, the related content providing method according to an embodiment of the present invention, the mobile terminal 200 may be implemented to perform the step (S507) of selecting the related content, or the server 300 may be implemented to perform. . When the server 300 is configured to perform the step of searching and selecting the related content (S507), the related content providing method according to an embodiment of the present invention, the sound wave extracted by the mobile receiver 200 in step S505 The method may further include transmitting an ID to the server 300, and transmitting the related content selected by the server 300 to the mobile receiver 200, wherein transmitting the sound wave ID to the server 300. After the operation S505 is performed, the transmitting of the related content selected by the server 300 to the mobile receiver 200 may be performed between the operations S507 and S509.

In addition, the related content providing method according to an embodiment of the present invention, the step of determining whether the sound detected in step S501 has a strength or more than a predetermined threshold ('S502' for the purpose of explanation); And activating at least one component of the above-described component that performs S501, the component that performs S502, the component that performs S503, the component that performs S505, and the component that performs S507. For the purpose of 'S504'); may further include.

For example, the activating step S504 may be a step of activating a component that performs S501, a component that performs S502, and a component that performs S503. In this example, the component that performs S501, the component that performs S502, and the component that performs S503 are deactivated and then activated components, the component that performs S505, and the component that performs S507. May be already activated.

For example, activating (S504) may include activating a component that performs S501, a component that performs S502; And activating a component that performs S503 and a component that performs S507 if the execution result of step S502 succeeds.

For example, the step of activating (S504), the step of activating the component performing the step S501; and when the sound is detected as a result of performing the step S501-for example, by detecting the sound belonging to the preset band Case-in, activating the component performing the step S502 and the component performing the step S503; may be configured to include. Here, the component performing the step S507 and the component performing the step S509 may be configured to be activated, for example, already activated or activated when the step S503 succeeds, or when the step S503 is activated. have.

For example, activating step S504 may include: activating a component that performs step S502 when sound is detected as a result of performing step S501; If it is determined that the sound sensed in step S501 has a strength greater than or equal to a preset threshold, the operation of step S502 may include activating a component that performs the step S503 and a component that performs the step S505. have. Here, when the sound wave ID is extracted as a result of performing the step S505, the component performing the step S507 and the component performing the step S509 may be activated simultaneously or sequentially.

For example, the activating step S504 may be performed according to a schedule or may be performed when a predetermined event occurs. For a detailed description of the schedule, please refer to the embodiments described with reference to FIG. 4. For example, the activating step S504 may be performed according to a schedule as in the embodiments described with reference to FIGS. 9 to 11, or there may be a predetermined event as in the embodiments described with reference to FIGS. 12 to 14. In this case, it may be performed.

For example, the activating step S504 may be performed at regular intervals and / or at irregular intervals according to a schedule created by referring to the main content schedule. For example, the activating step S504 may be performed at a predetermined period only during the time when the main content is played. Alternatively, the activating step (S504) may be performed at a first predetermined period during the time when the main content is played, and at a second period longer than the first period during the time when the main content is not played.

For example, the activating step S504 may be performed at regular intervals and / or at irregular intervals according to a schedule created with reference to the sound wave ID schedule. For example, the activating step S504 may be performed at a time when the sound wave ID may be extracted.

For example, the activating step (S504) is performed in a preset third period, but when the extraction of the sound wave ID has failed, but it is determined that the sound wave ID is present, activating at a fourth period shorter than the third period. Step S504 may be performed. After that, if the ID is extracted, the activating step (S504) may be performed in a third cycle.

For example, the step of activating (S504) is performed in the fifth predetermined period, the result of the execution of S502 the following equation

If satisfies, may be performed in a sixth period shorter than the fifth period. That is, in one embodiment, the inventor of the present invention, if the sensed intensity of the sound is smaller than the threshold, but close to the threshold, it is determined that there is a possibility that there is a sound with a sound wave ID, satisfies the above formula In this case, the activation step (S504) is performed in a sixth period shorter than the fifth period.

For example, activating step S504 may be executed when a predetermined event occurs. Here, the event may mean, for example, when a specific application is executed, when a display unit of the mobile receiver 200 is activated or deactivated, or when the mobile receiver 200 receives a push message from the outside.

For example, the activating step (S504) may be performed to be performed at a preset seventh cycle and not executed when a predetermined event occurs. When the predetermined event ends, the activating step (S504) is performed again according to a preset cycle (for example, the seventh cycle).

For example, in the step of activating (S504), when a predetermined event occurs, the fact that the event occurs is delivered to the operating system (OS) (not shown) of the mobile receiver 200, the operating system generates the event Transmitting the facts to filters provided corresponding to respective applications operating in the mobile receiver 200; And each filter may be configured to deliver or not deliver the fact that an event has occurred to applications corresponding to it, as set in advance. For a more detailed description, please refer to the embodiments described with reference to FIG. 14.

In the associated content providing method described with reference to FIG. 15, the component performing the step of detecting sound may be, for example, a microphone, and the component performing the step of converting the sound into digital data may be an ADC. The component performing the step of extracting the sound wave ID may be a sound wave ID extractor or an associated content providing application, and the component performing the step of searching for and selecting the related content may be a mobile receiver (for example, a related content providing application). Or a server, and a component for performing the step of displaying the related content by video and / or audio may be a display unit.

The related content providing method described with reference to FIG. 15 has been described as a mobile receiver detecting a sound output from a content player, but this is merely an example, and the sound output from the mobile receiver as in the embodiment of FIG. 6 or 7 is described. It can also be implemented as a configuration that the mobile receiver detects.

When implementing the related content providing method in a configuration in which the mobile receiver receives the sound output from the mobile receiver, the mobile receiver senses the sound output from the mobile receiver; Converting the sensed sound into digital data by the mobile receiver; Extracting, by the mobile receiver, the acoustic wave ID from the digital data; Selecting the associated content corresponding to the extracted sound wave ID; And displaying, by the mobile receiver, the selected related content as an image and / or an audio. The method may further include determining whether the sensed sound has an intensity greater than or equal to a preset threshold; And selecting, by the mobile receiver, a component for converting the detected sound into digital data, a component for extracting a sound wave ID from the digital data, and a related content corresponding to the extracted sound wave ID. And activating at least one of the components for performing the operation, and the mobile terminal displaying the selected related content in video and / or audio. Here, for a detailed example of the activating step, please refer to the above-described embodiments.

■ A computer-readable medium that records a program for executing the associated content providing method.

A related content providing method according to an embodiment of the present invention described above is a program executed in a mobile receiver having a memory, a processor, a microphone, and an ADC (for the purpose of explanation, a program for executing the related content providing method). It can be implemented as). That is, a mobile receiver having a memory, a processor, a microphone, and an ADC has a function as a computer, and thus, the related content providing method can be implemented in the form of a program executed in such a mobile receiver.

A program for executing a related content providing method according to an embodiment of the present invention may include an associated content providing application.

According to an embodiment, the program for executing the related content providing method may be executed after being loaded into a memory under the control of a processor.

The program for executing the related content providing method may be, for example, executing one of the embodiments described with reference to FIG. 15.

For example, the program for executing the related content providing method may execute, in each of the steps described with reference to FIG. 15, steps that can be implemented as a program in the mobile receiver.

For example, the program for executing the related content providing method may execute steps S502, S503, S504, S505, S507, and / or S509 to the mobile receiver. That is, steps S502, 503, S504, S505, S507, and S509 are all implemented as a program, or some of the steps S502, S503, S504, S505, S505, S507, and S509 are hardware. The remaining steps can be implemented in a program. For details of the steps S502, S503, S504, S505, S507, and S509, refer to the embodiments described with reference to FIG. 15.

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications from these descriptions. And variations are possible.

The scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims below but also by the equivalents of the claims.

Claims

Detecting sound output from a content player that plays content (called 'main content');

Converting the sensed sound into digital data;

Extracting a sound wave ID from the digital data;

Selecting related content corresponding to the sound wave ID; And

And displaying, by the mobile receiver, the related content in a video and / or audio format.
The method of claim 1,

Determining whether the sensed sound has an intensity greater than or equal to a preset threshold;

The converting of the digital data is performed only when it is determined that the sound has an intensity greater than or equal to a preset threshold.
The method of claim 1,

The sound wave ID is,

Digital data representing an audible sound artificially inserted into a sound output from the content reproducing apparatus (referred to as a 'non-audible sound wave ID'), or

And digital data (referred to as an "audible fingerprint ID") indicating characteristics of a sound which can be extracted from a sound included as a part of the sound output from the content reproducing apparatus.
The method of claim 3,

The related content,

A method of providing related content, characterized in that it refers to information such as sound, video, document, internet address, etc. and / or a digital identifier value indicating the content of the main content or background information for delivering the content.
The method of claim 4, wherein

The sound wave ID is,

Used to identify the associated content, and

1) identifying the main content, 2) identifying a relative time position within the main content, or 3) identifying an associated content associated with an advertisement at the relative time position within the main content. How to provide related content.
The method of claim 4, wherein

Transmitting the sound wave ID to a server; And

Transmitting, by the server, the related content corresponding to the sound wave ID to the mobile receiver;

Extracting the sound wave ID and transmitting the sound wave ID to a server are performed by the mobile receiver;

And selecting the associated content is performed by the server.
The method of claim 4, wherein

And the content player and the mobile receiver are the same device.
The method of claim 2,

If it is determined that the sound has a strength equal to or greater than a preset threshold, activating a component that performs the step of converting the digital data and a component that performs the step of extracting the sound wave ID to operate. Related content providing method comprising the.
The method of claim 1,

On a schedule or when a predetermined event occurs,

Activating at least one of the components for performing the steps.
The method of claim 9,

The component activated by the activating step,

And a component for detecting the sound output from the content player, a component for converting the sound into digital data, and a component for extracting the sound wave ID. Related content providing method.
The method of claim 9,

The activating is performed in a first period, and the detected intensity A of the sound is

Threshold> intensity of detected sound> α × threshold, where α is less than 1

If satisfied, characterized in that performed in a second period shorter than the first period characterized in that the content providing method.
The method of claim 9,

The activating step is performed periodically, but if a predetermined event occurs when it is time to perform, the associated content providing method, characterized in that not performed.
The method of claim 1,

Activating at least one of the components for performing the steps; And

Storing a main content schedule indicating the times at which the main content is played;

The activating step is performed by referring to the main content schedule, wherein the main content is played at a time when the main content is played.
The method of claim 13,

The activating step is related content providing method, characterized in that performed only in a predetermined period during the time that the main content is played.
The method of claim 13,

The activating step may be performed in a first period during the time when the main content is played, and in a second period longer than the first period during the time when the main content is not played.
The method of claim 1,

Activating at least one of the components for performing the steps; And

Storing a sound wave ID schedule indicating time at which the sound wave ID can be extracted;

And the activating step is performed at a time when the sound wave ID can be extracted with reference to the sound wave ID schedule.
The method of claim 16,

The activating step is performed in a first period, and when the extraction of the sound wave ID is unsuccessful but it is determined that the sound wave ID is present as a result of performing the step of extracting the sound wave ID, the activation is performed in a second period shorter than the first period. Related content providing method, characterized in that.
The method of claim 9,

The event,

Run a specific application,

The display unit provided by the mobile receiver is activated or deactivated, or

And means for receiving the push message by the mobile receiver.
The method of claim 12,

The predetermined event,

The application that uses only the microphone is executed, the related content providing method.
The method of claim 9,

Transmitting an event occurrence to an operating system when the event occurs;

Transmitting, by the operating system, the occurrence of the event to each filter corresponding to applications running in the mobile receiver; And

Each filter may or may not communicate the occurrence of the event to applications corresponding to it according to a preset setting.
The method of claim 1,

The component performing the step of converting the sound into digital data and the component performing the step of extracting the sound wave ID include a band in which a component performing the step of detecting the sound output from the content player is preset. When detecting a sound belonging to the associated content providing method characterized in that it is activated simultaneously or sequentially.
The method of claim 2,

The component performing the step of determining whether the sound has an intensity greater than or equal to a preset threshold may have detected a sound belonging to a preset band by the component performing the step of detecting the sound output from the content player. When it is activated,

The component for converting the sound into digital data and the component for extracting the sound wave ID are activated simultaneously or sequentially when it is determined that the sound has an intensity above a preset threshold. Related content providing method characterized in that the.
The method of claim 21 or 22,

The component performing the step of selecting the relevant content and the component performing the step of displaying the related content by video and / or audio by the mobile receiver are simultaneously or when the sound wave ID is extracted from the digital data. Related content providing method characterized in that the activation sequentially.
In a mobile receiver comprising a memory and a processor,

A microphone that senses sound output from a content player that plays content (called 'main content');

An ADC for converting the sound sensed by the microphone into digital data;

A sound wave ID extracting unit for extracting a sound wave ID from the digital data;

And a display unit for displaying the related content selected in correspondence with the sound wave ID.
The method of claim 24,

And a low power acoustic strength measurement unit for determining whether the sound sensed by the microphone has a strength greater than or equal to a preset threshold.

And the ADC converts the sound detected by the microphone into digital data only when it is determined that the sound has an intensity greater than or equal to a preset threshold.
The method of claim 24,

And a communication unit which transmits the sound wave ID extracted by the sound wave ID extracting unit to a server and receives related content corresponding to the sound wave ID from the server.

And the display unit displays the related content received by the communication unit.
The method of claim 24,

And the content player and the mobile receiver are the same device.
The method of claim 25,

When the low power acoustic strength measurement unit determines that the sound detected by the microphone has a strength greater than or equal to a preset threshold,

And the ADC and the sound wave ID extracting unit to be activated to operate.
The method of claim 24,

On a schedule or when a predetermined event occurs,

And at least one of the components-a microphone, an ADC, a sound wave ID extractor, and a display.
The method of claim 28,

At least one component that is activated is

And the microphone, the ADC, and the sound wave ID extracting unit.
The method of claim 28,

At least one of the components is activated at a first period, and the detected intensity A of the sound is

Threshold> intensity of detected sound> α × threshold, where α is less than 1

If it satisfies, the mobile receiver, characterized in that activated in a second period shorter than the first period.
The method of claim 28,

At least one of the components are activated periodically, but when the predetermined time occurs when the activation time, the mobile receiver that can provide the associated content, characterized in that not activated.
The method of claim 24,

The memory stores a main content schedule indicating the times when the main content is played,

And at least one of the components is activated in accordance with the main content schedule.
The method of claim 33, wherein

At least one of the components is a mobile receiver capable of providing associated content, characterized in that performed only at a predetermined period during the time the main content is played.
The method of claim 33, wherein

At least one of the components may be performed in a first period during the time when the main content is played, and in a second period shorter than the first period during the time when the main content is not played. Mobile receiver that can provide.
The method of claim 24,

The memory stores a sound wave ID schedule indicating times at which sound wave IDs can be extracted,

And at least one of the components is activated at a time when the sound wave ID can be extracted according to the sound wave ID schedule.
The method of claim 36,

At least one of the components is activated in a first period,

The sound wave ID extraction unit fails to extract the sound wave ID, but when it is determined that the sound wave ID exists, the mobile receiver according to the second period shorter than the first period is activated.
The method of claim 28,

The event,

Run a specific application

The display unit provided by the mobile receiver is activated or deactivated, or

And the mobile receiver receives a push message.
33. The method of claim 32,

The predetermined event,

A mobile receiver capable of providing associated content, wherein an application using only the microphone is running.
The method of claim 29,

An operating system loaded and executed in the memory under the control of the processor; And

Further comprising a filter,

The operating system receives an event occurrence when the event occurs, the operating system transmits the occurrence of the event to the filter,

And the filter transmits the fact of occurrence of the event to a component comparable to it according to a preset setting.
The method of claim 24,

The ADC and the sound wave ID extracting unit, when the microphone detects a sound belonging to a predetermined band, the mobile receiver capable of providing the associated content, characterized in that activated simultaneously or sequentially.
The method of claim 25,

The low power acoustic strength measurement unit is activated when the microphone detects sound belonging to a preset band,

And the ADC and the sound wave ID extracting unit are activated simultaneously or sequentially when the sound is determined to have an intensity greater than or equal to a preset threshold.
43. The method of claim 41 or 42,

A display unit for displaying the related content in video and / or audio, and at least some functions of the processor, including a function for controlling the display unit, simultaneously or sequentially when a sound wave ID is extracted from the digital data. A mobile receiver capable of providing associated content that is activated.
A method for providing related content to a computer including a memory, a processor, a microphone for detecting sound output from a content player for reproducing content (called 'main content'), and an ADC for converting the sound detected by the microphone into digital data. A computer-readable medium having recorded thereon a program for executing the above,

The related content providing method,

Extracting a sound wave ID from the digital data; And

Displaying the related content selected corresponding to the sound wave ID through a display unit provided in the computer; A computer-readable medium having recorded thereon a program for executing the related content providing method, comprising: a.
The method of claim 44,

The related content providing method,

And determining whether the sound sensed by the microphone has an intensity greater than or equal to a preset threshold. The computer-readable medium having recorded thereon a program for executing the related content providing method.
The method of claim 44,

The related content providing method,

And transmitting the sound wave ID to a server through a communication unit provided in the computer, and receiving the related content corresponding to the sound wave ID from the server. The computer-readable medium that recorded the data.
The method of claim 44,

And a computer for recording a program for executing the related content providing method, wherein the content player and the computer are the same device.
The method of claim 45,

The related content providing method,

And if it is determined that the sound detected by the microphone has a strength equal to or greater than a preset threshold, activating the ADC and activating the sound wave ID. A computer-readable medium that records a program for execution.
The method of claim 45,

The related content providing method,

Activating at least one of the microphone, the ADC, the sound wave ID, and the displaying, when a predetermined event occurs according to a schedule or when a predetermined event occurs. A computer-readable medium having recorded a program for executing the program.
The method of claim 49,

The activating step

And extracting the microphone, the ADC, and the sound wave ID. The computer-readable medium having recorded thereon a program for executing the related content providing method.
The method of claim 49,

The activating is performed in a first period, and the detected intensity A of the sound is

Threshold> intensity of detected sound> α × threshold, where α is less than 1

If it is satisfied, characterized in that performed in a second period shorter than the first period, the computer-readable medium having recorded a program for executing the associated content providing method.
The method of claim 49,

The activating step is performed periodically, but if a predetermined event occurs when it is time to perform, the computer-readable medium having recorded a program for executing the method for providing a related content, characterized in that not performed.
The method of claim 44,

The related content providing method,

Activating at least one of the microphone, the ADC, the sound wave ID, and the displaying; And

Storing a main content schedule indicating the times at which the main content is played;

The activating step is performed by referring to the main content schedule so that the main content is played at a time when the main content is played. A computer readable medium having recorded thereon a program for executing the related content providing method.
The method of claim 53,

The activating step is a computer-readable medium having recorded thereon a program for executing a method for providing a related content, wherein the main content is performed only at a time during which the main content is played.
The method of claim 53,

The activating may be performed in a first period during the time when the main content is played, and in a second period longer than the first period during the time when the main content is not played. A computer-readable medium that records a program to make it work.
The method of claim 44,

The related content providing method,

Activating at least one of the microphone, the ADC, the sound wave ID, and the displaying; And

Storing a sound wave ID schedule indicating time at which the sound wave ID can be extracted;

The activating step is performed by referring to the sound wave ID schedule at a time when the sound wave ID can be extracted. The computer-readable medium having recorded thereon a program for executing the related content providing method.
The method of claim 56, wherein

The activating step is performed in a first period, and when the extraction of the sound wave ID is unsuccessful but it is determined that the sound wave ID is present as a result of performing the step of extracting the sound wave ID, A computer-readable medium having recorded thereon a program for executing the related content providing method.
The method of claim 49,

The event,

Run a specific application,

The display unit provided by the computer is activated or deactivated, or

A computer readable medium having recorded thereon a program for executing the related content providing method, characterized in that the computer receives a push message.
The method of claim 52, wherein

The predetermined event,

A computer-readable medium in which a program for executing an associated content providing method is executed, in which an application using only a microphone is executed.
The method of claim 49,

The related content providing method,

Transmitting an event occurrence to an operating system when the event occurs;

Transmitting, by the operating system, the occurrence of the event to each filter corresponding to applications running on the computer; And

Each filter further comprises the step of delivering or not forwarding the occurrence of the event to applications corresponding to the filter according to a preset setting. The filter may further include a program for executing the related content providing method. Readable media.
The method of claim 44,

The component for performing the step of extracting the ADC and the sound wave ID, the program for executing the associated content providing method, characterized in that when the microphone detects a sound belonging to a predetermined band, simultaneously or sequentially. The computer-readable medium that recorded the data.
The method of claim 45,

A component for determining whether the sound has an intensity greater than or equal to a preset threshold is activated when the microphone detects a sound belonging to a preset band,

The component for performing the step of extracting the ADC and the sound wave ID, if it is determined that the sound has a strength above a predetermined threshold, the associated content providing method, characterized in that to be activated simultaneously or sequentially. Computer-readable media that records the program for the program.
63. The method of claim 61 or 62,

A component that performs the step of displaying the associated content in video and / or audio and at least some functions of the processor, the function of controlling the component that performs the displaying step; A computer-readable medium having recorded thereon a program for executing the related content providing method, wherein the ID is extracted, and activated simultaneously or sequentially.