KR20190081160A - Method for providing advertisement using stereoscopic content authoring tool and application thereof - Google Patents

Abstract

Disclosed are a method for providing advertisement using a stereophonic sound content authoring tool and an application therefor. According to an embodiment of the present invention, the method for providing advertisement provides a visual interface for applying a binaural effect to audio synchronized with a video played through a mobile terminal, and provides advertisement content along with rendering status feedback while rendering stereophonic sound content applied with the binaural effect based on the visual interface. Therefore, the present invention can generate revenue for a service provider.

Description

TECHNICAL FIELD [0001] The present invention relates to an advertisement providing method using stereophonic content authoring tool,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to an advertisement providing technology, and more particularly, to a technique for providing advertisement contents while rendering stereophonic contents using a stereophonic content authoring tool in a mobile terminal.

Generally, in order to apply a three-dimensional sound effect to an image, a multi-channel method of inputting a sound output by using a plurality of speakers is used. In addition, a system that realizes a stereo sound effect using a two-channel system using two speakers or headphones has been developed, but it is mainly a technique based on hardware.

Recently, there have been introduced technologies for implementing a stereo sound effect using software, but a technology for implementing a stereo sound effect using a three-dimensional position is merely to locate an image at a position corresponding to a two-dimensional position based on a listener It is difficult to see.

Korean Registered Patent No. 10-1599554, registered on February 25, 2016 (name: 3D binaural filtering system and method using SAC side information) Korean Registered Patent No. 10-0971700, July 15, 2010 (Name: Space Cue-Based Binaural Stereo Synthesizer and Method, and Binaural Stereo Decoder Using the Same)

It is an object of the present invention to generate revenue for a service provider by providing advertisement contents based on a stereophonic content authoring application including an intuitive interface.

It is also an object of the present invention to induce a user to use applications more actively by paying user compensation based on advertisement contents.

Another object of the present invention is to provide an advertisement content considering a user's application usage history to improve an advertisement viewing rate.

It is also an object of the present invention to author stereophonic contents including audio to which a binary effect is applied through a mobile terminal.

It is also an object of the present invention to provide stereophonic content capable of providing a sense of direction, distance, and space.

According to another aspect of the present invention, there is provided a method of providing an advertisement using a stereophonic content authoring tool, the method comprising: providing a visual interface for applying a binaural effect to audio synchronized with an image reproduced through a mobile terminal; ; And providing the advertisement contents together with the rendering status feedback while rendering the stereophonic contents to which the binary effect is applied based on the visual interface.

At this time, the rendering may be performed based on the touch input to the visual interface and based on the metadata synchronized with the image.

At this time, the advertisement content may be selected based on at least one of the rendering time factor and the content profile factor from the advertisement content database.

In this case, the rendering time factor may be calculated in consideration of at least one of a reproduction time corresponding to the image and a file size corresponding to the image.

At this time, the content profile parameter may be calculated based on the history of the user rendering the stereophonic content.

At this time, the advertisement contents can be selected in consideration of the sound source position control propensity factor.

At this time, the sound source position control tendency factor may be calculated based on the position change value of the sound image corresponding to the metadata.

At this time, the step of providing advertisement contents may provide user compensation corresponding to the provision of the advertisement contents, and the user compensation may be available to skip other advertisement contents.

At this time, the advertisement contents may include the audio to which the binary effect is applied.

At this time, the advertisement contents may include audio that varies depending on the peripheral device connection state to the mobile terminal.

At this time, the audio to which the binary effect is applied may correspond to the two channel audio corresponding to the left channel and the right channel.

In addition, an application stored in a computer-readable recording medium according to an exemplary embodiment of the present invention includes a step of providing a visual interface for applying a binaural effect to audio synchronized with an image reproduced through a mobile terminal ; And providing the advertisement content together with the rendering status feedback while rendering the binaural effect-applied stereophonic content based on the visual interface.

At this time, the rendering may be performed based on the touch input to the visual interface and based on the metadata synchronized with the image.

At this time, the advertisement content may be selected based on at least one of the rendering time factor and the content profile factor from the advertisement content database.

In this case, the rendering time factor may be calculated in consideration of at least one of a reproduction time corresponding to the image and a file size corresponding to the image.

At this time, the content profile parameter may be calculated based on the history of the user rendering the stereophonic content.

At this time, the advertisement contents can be selected in consideration of the sound source position control propensity factor.

At this time, the sound source position control tendency factor may be calculated based on the position change value of the sound image corresponding to the metadata.

At this time, the advertisement contents may include the audio to which the binary effect is applied.

At this time, the advertisement contents may include audio that varies depending on the peripheral device connection state to the mobile terminal.

According to the present invention, revenue can be generated for a service provider by providing advertisement contents based on a stereophonic content authoring application including an intuitive interface.

In addition, the present invention can induce a user to more actively use an application by paying user compensation based on advertisement contents.

In addition, the present invention can improve advertisement viewing rate by providing advertisement contents considering a user's application usage history.

In addition, the present invention can author stereoscopic sound contents including audio to which a binary effect is applied through a mobile terminal.

In addition, the present invention can provide stereophonic content capable of providing a sense of direction, a sense of distance, and a sense of space.

1 is a diagram illustrating an advertisement providing system using a stereophonic content authoring tool according to an embodiment of the present invention.
2 is a diagram illustrating an example of an application screen according to the present invention.
3 is a diagram illustrating an example of an advertisement content providing screen according to the present invention.
4 is a view showing another example of an advertisement content providing screen according to the present invention.
5 is a diagram illustrating an example of a visual interface according to the present invention.
6 is a diagram illustrating an example of metadata synchronized with an image according to the present invention.
7 is a block diagram illustrating an example of a rendering process according to the present invention.
8 is a diagram illustrating an example of an advertisement selection related parameter and an advertisement content database according to the present invention.
9 is a view illustrating an example of a process of selecting advertisement contents in a mobile terminal according to the present invention.
10 is a diagram illustrating an example of a process of selecting advertisement contents in a server according to the present invention.
11 is a view showing an example of an application execution screen for providing user compensation according to the present invention.
12 is a view showing an example of an application execution screen for skipping advertisement contents using user compensation according to the present invention.
13 is a view showing an example of an application execution screen for providing contents usable with user compensation according to the present invention.
14 is a flowchart illustrating an advertisement providing method using a stereophonic content authoring tool according to an embodiment of the present invention.
15 is a flowchart illustrating an advertisement providing method according to an exemplary embodiment of the present invention.
16 is a block diagram showing an example of the mobile terminal shown in FIG.

The present invention will now be described in detail with reference to the accompanying drawings. Hereinafter, a repeated description, a known function that may obscure the gist of the present invention, and a detailed description of the configuration will be omitted. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an advertisement providing system using a stereophonic content authoring tool according to an embodiment of the present invention.

1, an advertisement providing system using a stereophonic content authoring tool according to an exemplary embodiment of the present invention includes a mobile terminal 110, an application server 120, an advertisement server 130, and a network 140 .

The mobile terminal 110 corresponds to a terminal equipped with an application for providing an advertisement according to an embodiment of the present invention and includes a mobile phone, a tablet PC, a smart phone, a PDA (Personal Digital Assistants) , ≪ / RTI > In addition, the application for providing an advertisement according to an embodiment of the present invention can be executed in a terminal such as a PC (Personal Computer) or a notebook computer.

16, a mobile terminal 110 according to an embodiment of the present invention may include a communication unit 1610, a processor 1620, and a memory 1630. [

At this time, the mobile terminal 110 may download an application according to an embodiment of the present invention for authoring stereoscopic sound content and providing advertisement from the application server 120 based on the communication unit 1610 and store the downloaded application in the memory 1630 And may execute the application based on the processor 1620.

For example, the communication unit 1610 can send and receive information necessary for authoring and providing stereoscopic sound content to the application server 120 or the advertisement server 130 through a communication network such as the network 140. [

At this time, the processor 1620 of the mobile terminal 110 corresponds to the central processing unit and can execute and control the application stored in the memory 1630.

At this time, the memory 1630 of the mobile terminal 110 may store various applications including an application according to an embodiment of the present invention together with an operating system (OS). Accordingly, an application according to an embodiment of the present invention may correspond to a computer program installed and executed in a mobile terminal.

In addition, the memory 1630 may support a function for providing an advertisement according to an embodiment of the present invention. At this time, the memory 1630 may operate as a separate mass storage and may include a control function for performing operations. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In various different implementations, the memory may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

At this time, the mobile terminal 110 can select and provide the advertisement contents while rendering the stereophonic contents through the application. At this time, the advertisement contents may be selected from any one of the mobile terminal 110 and the application server 120.

In this case, although not shown in FIG. 1, the advertisement contents according to an embodiment of the present invention may be provided through a separate advertisement contents database. At this time, the advertisement content database may be included in or interworked with any one of the mobile terminal 110, the application server 120, and the advertisement server 130 to operate to provide advertisement contents.

The advertisement server 130 manages advertisement contents to be provided through the mobile terminal 110. The advertisement server 130 provides advertisement contents to the mobile terminal 110 or information related to the advertisement contents stored in the mobile terminal 110 Can be updated.

1 shows the application server 120 and the advertisement server 130, respectively, the application server 120 and the advertisement server 130 may be the same server.

Network 140 is a concept that encompasses both existing and future developable networks. For example, the network 140 may be a third-generation mobile communication network including an i-bimetal, a wired network, a Wibro (Wireless Broadband) network, a WCDMA, and a High Speed Downlink Packet Access (3G) mobile communication network including LTE network, a fourth generation mobile communication network including LTE advanced, a satellite communication network, and a Wi-Fi network.

2 is a diagram illustrating an example of an application screen according to the present invention.

2, an application stored in a computer-readable recording medium according to an exemplary embodiment of the present invention includes a visual interface for applying a binaural effect to audio synchronized with an image reproduced through a mobile terminal to provide.

At this time, the application corresponds to a tool for authoring stereoscopic contents, and may correspond to a computer program installed and executed in the computer.

In this case, the binaural effect provides a three-dimensional sound effect as if the audio image is located outside when the user is listening to the audio through the headphone or earphone. Therefore, the contents including the audio with the binary effect can provide the user with a more realistic and realistic experience than the general contents.

At this time, the image may correspond to the image stored in the mobile terminal. For example, the image may correspond to a user's own image captured through a camera installed in the mobile terminal or an image downloaded based on the network.

At this time, the visual interface can be provided through a mobile terminal capable of executing an application because it can be provided based on execution of an application for authoring stereoscopic contents and providing advertisement contents.

At this time, the mobile terminal may correspond to a terminal capable of performing data communication based on the network.

For example, a mobile terminal can transmit and receive information necessary for stereophonic content authoring and advertisement content provision through a communication network such as a network. Particularly, in the present invention, applications and advertisement contents can be obtained from a server based on a network.

In addition, the mobile terminal can execute an application stored in a memory based on a processor corresponding to the central processing unit to provide stereoscopic content and advertisement contents.

At this time, various applications including an application according to an embodiment of the present invention can be stored in the memory of the mobile terminal together with an operating system (OS). Accordingly, an application according to an embodiment of the present invention may correspond to a computer program installed and executed in a mobile terminal.

In addition, the memory of the mobile terminal may support a function for providing an advertisement according to an embodiment of the present invention. At this time, the memory may operate as a separate mass storage and may include control functions for performing operations. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In various different implementations, the memory may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

In addition, an application stored in a computer-readable recording medium according to an exemplary embodiment of the present invention provides advertisement contents along with rendering status feedback while rendering binaural sound content based on a visual interface.

For example, advertisement content 310, 410 may be provided along with rendering status feedback 320, 420 as shown in FIGS. 3-4.

At this time, it is possible to provide the rendering progress information up to the present through the rendering state feedback 320, 420. At this time, the rendering state feedbacks 320 and 420 are not limited to the shapes shown in FIGS. 3 to 4, but may be provided in various ways that the user can easily recognize.

At this time, the rendering may be performed based on the touch input to the visual interface and based on the metadata that is synchronized with the image.

For example, the metadata may be generated based on a touch input value to the visual interface. That is, the user of the mobile terminal can control the sound image of the audio synchronized with the image by operating the visual interface while watching the motion of the object included in the image.

At this time, the touch input value may be a multi-touch input value. In order to effectively specify a position on a three-dimensional coordinate, a single touch for setting a single point on the screen is not sufficient, and a multi-touch capable of simultaneously setting a plurality of points on the screen may be required.

That is, when the multi-touch is not supported or when the metadata is generated by using the input values using the keyboard or the mouse instead of the touch input to the screen, it is difficult for the user to generate the metadata intuitively and efficiently while viewing the image have.

At this time, the application according to an embodiment of the present invention can immediately output the feedback audio with the binary effect by reacting to the touch input.

At this time, the feedback audio may correspond to a modification of the existing audio on which the binary effect is not applied based on the metadata. That is, when the touch input by the user occurs, the feedback audio can be output by reflecting the generated metadata in real time. If immediate feedback audio is not output, the user may apply a binary effect so that it does not correspond to the image because the user can not hear the modified audio until rendering is complete.

Accordingly, in the present invention, the feedback audio is immediately output, so that the user can perform the touch input corresponding to the motion of the object included in the image.

At this time, the feedback audio to which the binary effect is applied can be output through an external sound device such as an earphone or a headphone, which allows the user to feel a stereoscopic effect due to a binary effect.

At this time, the metadata may correspond to the three-dimensional position of the sound image corresponding to the binary effect. In this case, the three-dimensional position of the sound image may be based on the user of the mobile terminal experiencing the binary effect.

For example, the three-dimensional position of the sound image according to the present invention can be displayed on a hemispherical three-dimensional space with respect to the user's head. Therefore, in the case of the audio with the binary effect, it is possible to provide a three-dimensional sound effect at 360 degrees based on the user.

As described above, in the present invention, the sound image is arranged on the three-dimensional space using the X-axis, the Y-axis, and the Z-axis to provide a deeper feeling of immersion than the conventional sound technique.

At this time, the visual interface according to an embodiment of the present invention includes a first interface for designating the position on the two-dimensional plane of the sound image and a second interface for specifying the position on the straight line perpendicular to the two-dimensional plane of the sound image And the 3D position may be generated by combining the user's first input to the first interface and the user's second input to the second interface.

For example, referring to FIG. 5, the user may operate the first interface 510 and the second interface 520 according to the movement of the object 500 included in the image. At this time, an operation key of the first interface 510 and an operation key of the second interface 520 can be moved to the touch input according to the direction in which the object 500 included in the image moves. At this time, the three-dimensional position generated by combining the first input and the second input according to the touch input can be recorded as metadata.

At this time, the visual interface is not limited to the form shown in FIG. 5, and can be provided in various forms in which the three-dimensional position of the sound image can be input.

At this time, it is possible to selectively provide any one of a play mode for playing the image by the feedback audio based on the mode change button included in the visual interface or an edit mode for adjusting the position of the sound image by the visual interface . At this time, the mode change button can be operated in a toggle manner.

At this time, the visual interface can overlay the image corresponding to the meta data on the image and display it in real time.

For example, a user can manipulate the first interface and the second interface to overlay and display the changed sound image on an object displayed in the image. In this case, when the user operates the first interface or the second interface based on the touch input, the position of the sound image may be displayed while moving in real time.

At this time, the sound image corresponding to the meta data may be expressed as a hemispheric position set with reference to the head position of the user.

For example, the sound track tracking interface can track changing sound images and show the location of sound images. At this time, the sound image tracking interface may be provided in a form corresponding to the hemispherical space created based on the user's head position. At this time, the sound image may be displayed while rotating the direction of the user's head so as to more intuitively identify the three-dimensional position of the sound image.

At this time, the visual interface may include an object tracking interface for highlighting the object corresponding to the sound image among the objects of the image.

At this time, the object trapping interface can determine the object corresponding to the sound image based on the correlation with the audio among the objects of the image. That is, at a moment when the audio synchronized with the video changes significantly, the object determined to be highly correlated with the audio change can be determined as the object corresponding to the sound image, taking into consideration the change of the object in the image.

For example, the object tracking interface can judge and highlight a newly emerging object as an object corresponding to a sound image at a time when audio synchronized with the image changes by a predetermined reference change amount or more.

The object trapping interface may also change the position of the sound image correspondingly to the motion of the recognized object corresponding to the sound image. That is, even if the user does not directly change the sound image by manipulating the visual interface, it is possible to automatically generate the metadata for the sound image while tracking the object corresponding to the sound image recognized through the object trapping interface.

At this time, the metadata according to an embodiment of the present invention can be generated in synchronization with an image reproduced in the mobile terminal.

For example, as shown in FIG. 6, the metadata 620-1 to 620-N generated during the reproduction of the image may be respectively recorded in the frames 610-1 to 610-N of the reproduced image at the touch input point It is possible to synchronize metadata and video by matching.

In this case, the stereophonic contents according to an embodiment of the present invention may correspond to contents that can be reproduced through the mobile terminal, and when a user views stereophonic contents, a plurality of speakers are installed around the user It is possible to feel the same sound effect as when the audio is outputted in the environment.

In this case, the audio to which the binary effect is applied may be two channel audio corresponding to the left channel and the right channel. Therefore, a terminal or a device that supports only 2-channel audio can provide stereophonic contents that can feel a sound effect like 5.1 channel or 10.2 channel.

At this time, the audio to which the binary effect according to an embodiment of the present invention is applied can be generated by applying existing technologies and technologies that can be developed in the future.

Korean Registered Patent Publication No. 10-1599554 discloses a method for outputting a three-dimensional binary signal based on a multi-channel audio encoding technology of MPEG standard called " MPEG Surround ". 10-1599554 extracts multichannel audio reproduction characteristic parameters based on the MPEG-Surround (MPS) international standard, performs HRTF (Head Related Transfer function) filtering on the downmix audio signal using the audio reproduction characteristic parameters, And outputs a binary signal. Here, the HRTF filtering may be filtering using the dummy header microphones modeled by the human auditory organ to obtain impulse responses on both sides of the respective positions with a specific interval between the azimuth angle of 360 degrees and the altitude angle of 180 degrees.

In this case, the multi-channel audio reproduction characteristic parameter relates to the output level difference of the front and rear channel signals according to the frequency band. In the MPEG-Surround (MPS) international standard, a multi-channel audio signal is received, And the like, and the like.

In Korean Registered Patent Publication No. 10-0971700, left and right audio signals are filtered in the frequency domain based on the position information of virtual sound sources and binaural filter coefficients for each channel, and the filtered signals are decoded into binaural stereo signals The contents are disclosed. At this time, the stereo left / right audio signal in the input time domain is converted into a frequency domain signal by using DFT (Discrete Fourier Transform) or FFT (Fast Fourier Transform), and the sub- The binaural stereo signal can be used to filter the stereo left / right signal corresponding to the frequency domain based on the power gain value of each channel for each band and the left / right HRTF coefficient block of the frequency domain for each channel.

In this case, the power gain value of each channel can be calculated by combining spatial cue information based on VSLI (Virtual Source Location Information), and VSLI-based spatial queue information for a stereo signal The left half-plane angle (LHA (m)), the left subconscious angle (LSA (LSA (m)) and the right half angle (LSA RHA (Right Half-Plane Angle) RHA (m) and Right Subsequent Angle (RSA (m)).

Accordingly, in the present invention, it is also possible to generate audio to which a binary effect is applied according to the metadata based on the above-described technique.

For example, spatial parameters for audio synchronized to an image are extracted based on the MPEG-Surround (MPS) international standard, and HRTF (Head Related) is calculated for the audio based on the 3D position and spatial parameters of the sound source corresponding to the metadata. Transfer function filtering can be performed to generate an audio signal with a binary effect.

In another example, the audio synchronized with the image is converted into a stereo left / right audio signal in the frequency domain, and the spatial cue information is synthesized based on the position information corresponding to the metadata, And then generate a binarized audio based on the left / right HRTF coefficient block in the frequency domain for each channel.

At this time, a container including an image, audio and binary data to which a binary effect is applied and additional data is generated through rendering, and audio with a binary effect can be synchronized.

For example, referring to FIG. 7, the metadata 720 for the audio 712 S ₁ synchronized with the image 711 M ₁ based on the touch input to the visual interface according to an embodiment of the present invention Can be assumed. At this time, the application according to an embodiment of the present invention generates an audio 721 S ₂ to which a _binary effect is applied by using the audio 712 S ₁ and the metadata 720 synchronized with the video 711 M ₁ may be, it is possible to perform rendering to generate an image (711), M _1, a container 740 containing S ₂ and the additional data 730, audio 721 by the neoreol effects.

In this case, the additional data 730 may include information related to the format of stereophonic contents, parameters for rendering, and the like.

At this time, the rendering can be started based on the touch input to the visual interface.

For example, when the user touches the save button (SAVE) included in the visual interface, the user can start rendering to generate stereophonic content. At this time, the rendered stereoscopic sound content can be stored in the mobile terminal based on the stereoscopic sound content authoring application.

At this time, the rendering may be performed together with the process of generating the audio effect 721 S ₂ to which the _binary effect is applied.

In addition, when the user stops or ends the visual interface without performing rendering, the generated metadata may be stored or deleted.

For example, when the user touches the stop button PAUSE included in the visual interface, the metadata generated so far may be stored and stored together with the image.

For example, when the user touches the end button (End) or the exit button (EXIT) included in the visual interface, the generated metadata may be deleted and the visual interface may be terminated.

At this time, the advertisement content may be selected based on at least one of the rendering time factor and the content profile factor from the advertisement content database.

Further, the advertisement contents can be selected in consideration of the sound source position control propensity factor.

For example, as shown in FIG. 8, among the plurality of advertisement contents stored in the advertisement content database 800, a rendering time factor 810, a content profile factor 820, and an acoustic location control propensity factor 830 It is possible to select an advertisement content to be output at the time of rendering.

In this case, the rendering time factor may be calculated in consideration of one or more of the reproduction time corresponding to the image and the file size corresponding to the image.

For example, the longer the reproduction time corresponding to the image or the larger the file size corresponding to the image, the longer the time required for rendering may be. Therefore, in this case, it is possible to select and output the advertisement contents which are relatively long in the advertisement contents stored in the advertisement contents database 800.

At this time, the content profile parameter may be calculated based on the history of the user rendering stereoscopic contents.

For example, when the user mainly renders stereophonic contents for a game image, it is determined that the user prefers the game, and the user can select and output the game advertisement contents among the advertisement contents stored in the advertisement contents database 800 have.

In this case, the sound source position control propensity factor can be calculated based on the position change value of the sound image corresponding to the meta data.

For example, when the positional change of the sound image corresponding to the meta data generated by the user's touch input is large, it is determined that the user prefers a stereoscopic image having many movements, and the advertisement contents stored in the advertisement content database 800 It is possible to select and output the advertisement contents of the image having a large change in the scene or dynamic of the scenes.

Accordingly, the advertisement content database 800 can store and manage a plurality of advertisement contents in consideration of the reproduction time, the content category, and the static / dynamic content.

At this time, the advertisement contents according to an embodiment of the present invention can be selected through a mobile terminal or a server providing an application.

For example, when selecting the advertisement content in the mobile terminal, the application 911 stored in the mobile terminal 910 may select the advertisement content output based on the advertisement content metadata 912 as shown in FIG. 9 have. At this time, the advertisement content metadata 912 may correspond to meta information about a plurality of advertisement contents stored in the advertisement content database 900. [ Accordingly, the mobile terminal 910 may request the advertisement content database 900 to transmit the advertisement content already selected, and the advertisement content database 900 may transmit the advertisement content corresponding to the request to the mobile terminal 910, 911).

In another example, when selecting ad content on the server, the relevancy factors to be considered for selecting ad content from the mobile terminal 1020 to the server 1010, such as the rendering time factor, Content profile parameters, and sound source location control propensity factors. Thereafter, the server 1010 may select any one of a plurality of advertisement contents stored in the advertisement content database 1000 and provide the selected advertisement contents to the mobile terminal 1020 using the transmitted related parameters. Thereafter, the mobile terminal 1020 can output the advertisement contents provided from the server 1010 through the application for stereoscopic sound content authoring and advertisement provisioning.

At this time, the advertisement contents may include audio to which a binary effect is applied.

In addition, the advertisement content may include audio that varies depending on the peripheral device connection status to the mobile terminal.

For example, an application according to an embodiment of the present invention provides advertisement contents including audio with a binary effect when an external audio equipment is connected to the mobile terminal, and an external audio equipment is connected to the mobile terminal It is possible to provide the advertisement contents including the audio to which the binary effect is not applied.

By using an application stored in such a computer-readable recording medium, revenue can be generated by providing advertisement contents while providing services for authoring stereophonic contents.

11 is a view showing an example of an application execution screen for providing user compensation according to the present invention.

Referring to FIG. 11, an application according to an exemplary embodiment of the present invention may provide user compensation according to advertisement contents provided during rendering stereophonic contents.

At this time, the user compensation may be provided in consideration of the reproduction time of the advertisement contents.

For example, it can be assumed that the preset reference playback time is 5 minutes. 11, when the user watches the advertisement contents having a reproduction time of less than 5 minutes while rendering the stereophonic contents, it is possible to provide four notes as a unit corresponding to the user compensation. However, if the playback time of the advertisement content viewed by the user exceeds 5 minutes, eight notes corresponding to double compensation may be provided.

In this case, the criterion for paying the user compensation can be freely set and changed according to the embodiment of the present invention, and the unit degree corresponding to the user compensation is not limited to the note.

At this time, the user compensation can be used to skip other advertisement contents.

For example, referring to FIG. 12, an application according to an exemplary embodiment of the present invention may provide a skip button 1210 for skipping advertisement contents. If the user of the mobile terminal touches the skip button 1210, the user may be prompted to skip the advertisement by consuming a note through the skip guide pop-up 1220 as shown in FIG. At this time, the number of notes required for skipping the advertisement contents may be displayed together with the number of notes held by the current user.

At this time, while the user selects whether to skip the advertisement, the reproduction of the advertisement contents can be temporarily stopped, but the rendering can be continued.

User compensation may also be used to download content through an application according to an embodiment of the present invention.

For example, as shown in FIG. 13, movie contents can be downloaded or purchased and reproduced using a note corresponding to user compensation.

14 is a flowchart illustrating an advertisement providing method using a stereophonic content authoring tool according to an embodiment of the present invention.

Referring to FIG. 14, an advertisement providing method using a stereophonic content authoring tool according to an embodiment of the present invention includes a visual interface for applying a binaural effect to audio synchronized with an image reproduced through a mobile terminal, (S1410).

In this case, the binaural effect provides a three-dimensional sound effect as if the audio image is located outside when the user is listening to the audio through the headphone or earphone. Therefore, the contents including the audio with the binary effect can provide the user with a more realistic and realistic experience than the general contents.

At this time, the image may correspond to the image stored in the mobile terminal. For example, the image may correspond to a user's own image captured through a camera installed in the mobile terminal or an image downloaded based on the network.

At this time, the visual interface can be provided through a mobile terminal capable of executing an application because it can be provided based on execution of an application for authoring stereoscopic contents and providing advertisement contents.

At this time, the mobile terminal may correspond to a terminal capable of performing data communication based on the network.

For example, a mobile terminal can transmit and receive information necessary for stereophonic content authoring and advertisement content provision through a communication network such as a network. Particularly, in the present invention, applications and advertisement contents can be obtained from a server based on a network.

In addition, the mobile terminal can execute an application stored in a memory based on a processor corresponding to the central processing unit to provide stereoscopic content and advertisement contents.

At this time, various applications including an application according to an embodiment of the present invention can be stored in the memory of the mobile terminal together with an operating system (OS). Accordingly, an application according to an embodiment of the present invention may correspond to a computer program installed and executed in a mobile terminal.

In addition, the memory of the mobile terminal may support a function for providing an advertisement according to an embodiment of the present invention. At this time, the memory may operate as a separate mass storage and may include control functions for performing operations. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In various different implementations, the memory may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

Also, an advertisement providing method using a stereophonic content authoring tool according to an exemplary embodiment of the present invention provides advertisement contents along with rendering status feedback while rendering stereophonic contents applied with a binary effect based on a visual interface S1420).

For example, advertisement content 310, 410 may be provided along with rendering status feedback 320, 420 as shown in FIGS. 3-4.

At this time, it is possible to provide the rendering progress information up to the present through the rendering state feedback 320, 420. At this time, the rendering state feedbacks 320 and 420 are not limited to the shapes shown in FIGS. 3 to 4, but may be provided in various ways that the user can easily recognize.

At this time, the rendering may be performed based on the touch input to the visual interface and based on the metadata that is synchronized with the image.

For example, the metadata may be generated based on a touch input value to the visual interface. That is, the user of the mobile terminal can control the sound image of the audio synchronized with the image by operating the visual interface while watching the motion of the object included in the image.

At this time, the touch input value may be a multi-touch input value. In order to effectively specify a position on a three-dimensional coordinate, a single touch for setting a single point on the screen is not sufficient, and a multi-touch capable of simultaneously setting a plurality of points on the screen may be required.

That is, when the multi-touch is not supported or when the metadata is generated by using the input values using the keyboard or the mouse instead of the touch input to the screen, it is difficult for the user to generate the metadata intuitively and efficiently while viewing the image have.

In this case, the advertisement enhancement method according to an embodiment of the present invention can output feedback audio to which a binary effect is applied by immediately reacting to a touch input.

At this time, the feedback audio may correspond to a modification of the existing audio on which the binary effect is not applied based on the metadata. That is, when the touch input by the user occurs, the feedback audio can be output by reflecting the generated metadata in real time. If immediate feedback audio is not output, the user may apply a binary effect so that it does not correspond to the image because the user can not hear the modified audio until rendering is complete.

Accordingly, in the present invention, the feedback audio is immediately output, so that the user can perform the touch input corresponding to the motion of the object included in the image.

At this time, the feedback audio to which the binary effect is applied can be output through an external sound device such as an earphone or a headphone, which allows the user to feel a stereoscopic effect due to a binary effect.

At this time, the metadata may correspond to the three-dimensional position of the sound image corresponding to the binary effect. In this case, the three-dimensional position of the sound image may be based on the user of the mobile terminal experiencing the binary effect.

For example, the three-dimensional position of the sound image according to the present invention can be displayed on a hemispherical three-dimensional space with respect to the user's head. Therefore, in the case of the audio with the binary effect, it is possible to provide a three-dimensional sound effect at 360 degrees based on the user.

As described above, in the present invention, the sound image is arranged on the three-dimensional space using the X-axis, the Y-axis, and the Z-axis to provide a deeper feeling of immersion than the conventional sound technique.

At this time, the visual interface according to an embodiment of the present invention includes a first interface for designating the position on the two-dimensional plane of the sound image and a second interface for specifying the position on the straight line perpendicular to the two-dimensional plane of the sound image And the 3D position may be generated by combining the user's first input to the first interface and the user's second input to the second interface.

For example, referring to FIG. 5, the user may operate the first interface 510 and the second interface 520 according to the movement of the object 500 included in the image. At this time, an operation key of the first interface 510 and an operation key of the second interface 520 can be moved to the touch input according to the direction in which the object 500 included in the image moves. At this time, the three-dimensional position generated by combining the first input and the second input according to the touch input can be recorded as metadata.

At this time, the visual interface is not limited to the form shown in FIG. 5, and can be provided in various forms in which the three-dimensional position of the sound image can be input.

At this time, it is possible to selectively provide any one of a play mode for playing the image by the feedback audio based on the mode change button included in the visual interface or an edit mode for adjusting the position of the sound image by the visual interface . At this time, the mode change button can be operated in a toggle manner.

At this time, the visual interface can overlay the image corresponding to the meta data on the image and display it in real time.

For example, a user can manipulate the first interface and the second interface to overlay and display the changed sound image on an object displayed in the image. In this case, when the user operates the first interface or the second interface based on the touch input, the position of the sound image may be displayed while moving in real time.

At this time, the sound image corresponding to the meta data may be expressed as a hemispheric position set with reference to the head position of the user.

For example, the sound track tracking interface can track changing sound images and show the location of sound images. At this time, the sound image tracking interface may be provided in a form corresponding to the hemispherical space created based on the user's head position. At this time, the sound image may be displayed while rotating the direction of the user's head so as to more intuitively identify the three-dimensional position of the sound image.

At this time, the visual interface may include an object tracking interface for highlighting the object corresponding to the sound image among the objects of the image.

At this time, the object trapping interface can determine the object corresponding to the sound image based on the correlation with the audio among the objects of the image. That is, at a moment when the audio synchronized with the video changes significantly, the object determined to be highly correlated with the audio change can be determined as the object corresponding to the sound image, taking into consideration the change of the object in the image.

For example, the object tracking interface can judge and highlight a newly emerging object as an object corresponding to a sound image at a time when audio synchronized with the image changes by a predetermined reference change amount or more.

The object trapping interface may also change the position of the sound image correspondingly to the motion of the recognized object corresponding to the sound image. That is, even if the user does not directly change the sound image by manipulating the visual interface, it is possible to automatically generate the metadata for the sound image while tracking the object corresponding to the sound image recognized through the object trapping interface.

At this time, the metadata according to an embodiment of the present invention can be generated in synchronization with an image reproduced in the mobile terminal.

For example, as shown in FIG. 6, the metadata 620-1 to 620-N generated during the reproduction of the image may be respectively recorded in the frames 610-1 to 610-N of the reproduced image at the touch input point It is possible to synchronize metadata and video by matching.

In this case, the stereophonic contents according to an embodiment of the present invention may correspond to contents that can be reproduced through the mobile terminal, and when a user views stereophonic contents, a plurality of speakers are installed around the user It is possible to feel the same sound effect as when the audio is outputted in the environment.

In this case, the audio to which the binary effect is applied may be two channel audio corresponding to the left channel and the right channel. Therefore, a terminal or a device that supports only 2-channel audio can provide stereophonic contents that can feel a sound effect like 5.1 channel or 10.2 channel.

At this time, the audio to which the binary effect according to an embodiment of the present invention is applied can be generated by applying existing technologies and technologies that can be developed in the future.

Korean Registered Patent Publication No. 10-1599554 discloses a method for outputting a three-dimensional binary signal based on a multi-channel audio encoding technology of MPEG standard called " MPEG Surround ". 10-1599554 extracts multichannel audio reproduction characteristic parameters based on the MPEG-Surround (MPS) international standard, performs HRTF (Head Related Transfer function) filtering on the downmix audio signal using the audio reproduction characteristic parameters, And outputs a binary signal. Here, the HRTF filtering may be filtering using the dummy header microphones modeled by the human auditory organ to obtain impulse responses on both sides of the respective positions with a specific interval between the azimuth angle of 360 degrees and the altitude angle of 180 degrees.

In this case, the multi-channel audio reproduction characteristic parameter relates to the output level difference of the front and rear channel signals according to the frequency band. In the MPEG-Surround (MPS) international standard, a multi-channel audio signal is received, And the like, and the like.

In Korean Registered Patent Publication No. 10-0971700, left and right audio signals are filtered in the frequency domain based on the position information of virtual sound sources and binaural filter coefficients for each channel, and the filtered signals are decoded into binaural stereo signals The contents are disclosed. At this time, the stereo left / right audio signal in the input time domain is converted into a frequency domain signal by using DFT (Discrete Fourier Transform) or FFT (Fast Fourier Transform), and the sub- The binaural stereo signal can be used to filter the stereo left / right signal corresponding to the frequency domain based on the power gain value of each channel for each band and the left / right HRTF coefficient block of the frequency domain for each channel.

In this case, the power gain value of each channel can be calculated by combining spatial cue information based on VSLI (Virtual Source Location Information), and VSLI-based spatial queue information for a stereo signal The left half-plane angle (LHA (m)), the left subconscious angle (LSA (LSA (m)) and the right half angle (LSA RHA (Right Half-Plane Angle) RHA (m) and Right Subsequent Angle (RSA (m)).

Accordingly, in the present invention, it is also possible to generate audio to which a binary effect is applied according to the metadata based on the above-described technique.

For example, spatial parameters for audio synchronized to an image are extracted based on the MPEG-Surround (MPS) international standard, and HRTF (Head Related) is calculated for the audio based on the 3D position and spatial parameters of the sound source corresponding to the metadata. Transfer function filtering can be performed to generate an audio signal with a binary effect.

In another example, the audio synchronized with the image is converted into a stereo left / right audio signal in the frequency domain, and the spatial cue information is synthesized based on the position information corresponding to the metadata, And then generate a binarized audio based on the left / right HRTF coefficient block in the frequency domain for each channel.

At this time, a container including an image, audio and binary data to which a binary effect is applied and additional data is generated through rendering, and audio with a binary effect can be synchronized.

For example, referring to FIG. 7, the metadata 720 for the audio 712 S1 to be synchronized with the image 711 M1 is generated based on the touch input to the visual interface according to an embodiment of the present invention Can be assumed. At this time, the advertisement enhancement method according to an embodiment of the present invention generates an audio 721 S2 to which a binary effect is applied by using the audio 712 S1 and the metadata 720 synchronized with the image 711 M1 And may perform rendering to create a container 740 containing the image 711, the audio 721 S2 and the additional data 730 with the binarized effect.

In this case, the additional data 730 may include information related to the format of stereophonic contents, parameters for rendering, and the like.

At this time, the rendering can be started based on the touch input to the visual interface.

For example, when the user touches the save button (SAVE) included in the visual interface, the user can start rendering to generate stereophonic content. At this time, the rendered stereoscopic sound content can be stored in the mobile terminal based on the stereoscopic sound content authoring application.

At this time, the rendering may be performed together with the process of generating the audio effect 721 S2 to which the binary effect is applied.

In addition, when the user stops or ends the visual interface without performing rendering, the generated metadata may be stored or deleted.

For example, when the user touches the stop button PAUSE included in the visual interface, the metadata generated so far may be stored and stored together with the image.

For example, when the user touches the end button (End) or the exit button (EXIT) included in the visual interface, the generated metadata may be deleted and the visual interface may be terminated.

At this time, the advertisement content may be selected based on at least one of the rendering time factor and the content profile factor from the advertisement content database.

Further, the advertisement contents can be selected in consideration of the sound source position control propensity factor.

For example, as shown in FIG. 8, among the plurality of advertisement contents stored in the advertisement content database 800, a rendering time factor 810, a content profile factor 820, and an acoustic location control propensity factor 830 It is possible to select an advertisement content to be output at the time of rendering.

In this case, the rendering time factor may be calculated in consideration of one or more of the reproduction time corresponding to the image and the file size corresponding to the image.

For example, the longer the reproduction time corresponding to the image or the larger the file size corresponding to the image, the longer the time required for rendering may be. Therefore, in this case, it is possible to select and output the advertisement contents which are relatively long in the advertisement contents stored in the advertisement contents database 800.

At this time, the content profile parameter may be calculated based on the history of the user rendering stereoscopic contents.

For example, when the user mainly renders stereophonic contents for a game image, it is determined that the user prefers the game, and the user can select and output the game advertisement contents among the advertisement contents stored in the advertisement contents database 800 have.

In this case, the sound source position control propensity factor can be calculated based on the position change value of the sound image corresponding to the meta data.

For example, when the positional change of the sound image corresponding to the meta data generated by the user's touch input is large, it is determined that the user prefers a stereoscopic image having many movements, and the advertisement contents stored in the advertisement content database 800 It is possible to select and output the advertisement contents of the image having a large change in the scene or dynamic of the scenes.

Accordingly, the advertisement content database 800 can store and manage a plurality of advertisement contents in consideration of the reproduction time, the content category, and the static / dynamic content.

At this time, the advertisement contents according to an embodiment of the present invention can be selected through a mobile terminal or a server providing an application.

For example, when selecting the advertisement content in the mobile terminal, the application 911 stored in the mobile terminal 910 may select the advertisement content output based on the advertisement content metadata 912 as shown in FIG. 9 have. At this time, the advertisement content metadata 912 may correspond to meta information about a plurality of advertisement contents stored in the advertisement content database 900. [ Accordingly, the mobile terminal 910 may request the advertisement content database 900 to transmit the advertisement content already selected, and the advertisement content database 900 may transmit the advertisement content corresponding to the request to the mobile terminal 910, 911).

In another example, when selecting ad content on the server, the relevancy factors to be considered for selecting ad content from the mobile terminal 1020 to the server 1010, such as the rendering time factor, Content profile parameters, and sound source location control propensity factors. Thereafter, the server 1010 may select any one of a plurality of advertisement contents stored in the advertisement content database 1000 and provide the selected advertisement contents to the mobile terminal 1020 using the transmitted related parameters. Thereafter, the mobile terminal 1020 can output the advertisement contents provided from the server 1010 through the application for stereoscopic sound content authoring and advertisement provisioning.

At this time, it may be possible to provide user compensation corresponding to the provision of advertisement contents, and to use the user compensation to skip other advertisement contents.

At this time, the user compensation may be provided in consideration of the reproduction time of the advertisement contents.

For example, it can be assumed that the preset reference playback time is 5 minutes. 11, when the user watches the advertisement contents having a reproduction time of less than 5 minutes while rendering the stereophonic contents, it is possible to provide four notes as a unit corresponding to the user compensation. However, if the playback time of the advertisement content viewed by the user exceeds 5 minutes, eight notes corresponding to double compensation may be provided.

In this case, the criterion for paying the user compensation can be freely set and changed according to the embodiment of the present invention, and the unit degree corresponding to the user compensation is not limited to the note.

At this time, the user compensation can be used to skip other advertisement contents.

For example, referring to FIG. 12, an advertisement providing method according to an embodiment of the present invention may provide a skip button 1210 for skipping advertisement contents. If the user of the mobile terminal touches the skip button 1210, the user may be prompted to skip the advertisement by consuming a note through the skip guide pop-up 1220 as shown in FIG. At this time, the number of notes required for skipping the advertisement contents may be displayed together with the number of notes held by the current user.

At this time, while the user selects whether to skip the advertisement, the reproduction of the advertisement contents can be temporarily stopped, but the rendering can be continued.

At this time, the advertisement contents may include audio to which a binary effect is applied.

In addition, the advertisement content may include audio that varies depending on the peripheral device connection status to the mobile terminal.

For example, an application according to an embodiment of the present invention provides advertisement contents including audio with a binary effect when an external audio equipment is connected to the mobile terminal, and an external audio equipment is connected to the mobile terminal It is possible to provide the advertisement contents including the audio to which the binary effect is not applied.

In addition, although not shown in FIG. 14, the advertisement providing method using the stereophonic content authoring tool according to an embodiment of the present invention may store various information generated in the advertisement providing process in a separate storage module .

With such an advertisement providing method, it is possible to provide a service for authoring stereophonic contents and to generate revenue from providing advertisement contents for a service provider.

15 is a flowchart illustrating an advertisement providing method according to an exemplary embodiment of the present invention.

Referring to FIG. 15, an advertisement providing method according to an embodiment of the present invention is first performed when a stereophonic content authoring tool installed in a mobile terminal, that is, an application according to an embodiment of the present invention is executed (S1510) And provides a visual interface based on the image selected by the user (S1520).

At this time, the visual interface may include interfaces for applying a binary effect to the audio synchronized with the image selected by the user.

If rendering is requested by the user (S1530), the advertisement content to be provided during the rendering time is selected in consideration of the rendering time factor, the content profile factor, and the sound source position control propensity factor in operation S1540.

At this time, either the mobile terminal or the server providing the application can select the advertisement contents.

Thereafter, rendering is performed to generate a container including audio and additional data to which a video, a binary effect is applied, and at the same time, the advertisement content is output to the mobile terminal (S1550).

At this time, the video and the audio to which the binary effect is applied can be synchronized with respect to time.

Thereafter, when the advertisement contents together with the rendering are concurrently terminated (S1560), the stereophonic contents corresponding to the created container can be stored in the memory of the mobile terminal (S1570).

In this case, when stereophonic contents are reproduced by a mobile terminal connected to an external sound apparatus such as an earphone or a headphone, it is possible to appreciate the contents realistically as if the sound exists in the external audio.

As described above, the method and apparatus for providing advertisement using stereophonic content authoring tool according to the present invention are not limited to the configuration and method of the embodiments described above, All or some of the embodiments may be selectively combined.

110, 910, 1020: mobile terminal 120, 1010: application server
130: advertisement server 140: network
310, 410: advertisement contents 320, 420: rendering status feedback
500: object 510: first interface
520: second interface 610-1 to 610-N: frame
620-1 to 620-N, 720: metadata 711: video
712: Audio 721: Audio with binaural effect
730: Additional data 740: Container
800, 900, 1000: Ad contents database
810: rendering time factor 820: content profile factor
830: Sound source location control propensity factor 911: Application
912: Advertisement content metadata 1110: User-provided compensation pop-up
1210: Skip button 1220: Skip guidance pop-up
1610: communication unit 1620: processor
1630: Memory

Claims (20)

Providing a visual interface for applying a binaural effect to audio that is synchronized with an image played through a mobile terminal; And
Providing ad content along with rendering status feedback while rendering binaural sound content based on the visual interface;
The method of claim 1, further comprising: The method according to claim 1,
The rendering
Wherein the method is performed based on meta data generated based on a touch input to the visual interface and synchronized with the image. The method of claim 2,
The advertisement content
Wherein the at least one content is selected based on at least one of a rendering time factor and a content profile factor from the advertisement content database. The method of claim 3,
The rendering time factor
And a file size corresponding to the image. The method of claim 1, wherein the at least one of the at least one of the first, The method of claim 3,
The content profile parameter
And the user is calculated based on a history of rendering the stereophonic contents. The method of claim 3,
The advertisement content
And the sound source position control tendency factor is further taken into consideration. The method of claim 6,
The sound source position control propensity factor
And calculating the positional change value of the sound image corresponding to the meta data. The method of claim 3,
The step of providing the advertisement content
And providing user compensation corresponding to the provision of the advertisement content, wherein the user compensation is available for skipping other advertisement content. The method of claim 3,
The advertisement content
Wherein the audio effect includes audio applied with the binaural effect. The method of claim 9,
The advertisement content
And audio depending on a connection state of a peripheral device to the mobile terminal. The method of claim 9,
Wherein the audio to which the binary effect is applied is two-channel audio corresponding to a left channel and a right channel. Providing a visual interface for applying a binaural effect to audio that is synchronized with an image played through a mobile terminal; And
And providing the advertisement content along with the rendering status feedback while rendering the binaural effected stereophonic content based on the visual interface. The method of claim 12,
The rendering
Wherein the metadata is generated based on a touch input to the visual interface, and is performed based on metadata that is synchronized with the image. 14. The method of claim 13,
The advertisement content
Wherein the at least one parameter is selected based on at least one of a rendering time factor and a content profile factor from an advertising content database. 15. The method of claim 14,
The rendering time factor
And a file size corresponding to the image. The computer-readable recording medium according to claim 1, 15. The method of claim 14,
The content profile parameter
Characterized in that the user is calculated on the basis of a history of rendering the stereophonic contents. 15. The method of claim 14,
The advertisement content
And the sound source position control propensity factor is further taken into consideration. 18. The method of claim 17,
The sound source position control propensity factor
Wherein the calculation is performed on the basis of the position change value of the sound image corresponding to the metadata. 15. The method of claim 14,
The advertisement content
Characterized in that it comprises audio with the binary effect applied thereto. The method of claim 19,
The advertisement content
And audio dependent on a peripheral device connection state to the mobile terminal.

Images