JP2019537791A - Advertising based on spatial sound in virtual or augmented reality video streams - Google Patents

Abstract

An advertisement based on spatial audio is provided in the VR video stream. In various embodiments, a spatial audio cue is played to a user. Spatial acoustic cues have an apparent position in the virtual environment. The user's view is tracked toward the apparent position of the acoustic cue. The sponsored content is displayed to the user at the approximate apparent location of the audio cue. [Selection diagram] Fig. 1

Description

  Embodiments of the present invention relate to virtual reality presentation, and more specifically, to advertisements based on spatial sound in virtual reality or augmented reality video streams.

  According to an embodiment of the present disclosure, there is provided a method of audio-based advertising and a computer program product therefor. A spatial acoustic cue is played to the user. Spatial acoustic cues have an apparent position in the virtual environment. The user's view is tracked toward the apparent position of the acoustic cue. The sponsored content is displayed to the user at the approximate apparent location of the audio cue.

  In some embodiments, the user's field of view is determined by tracking the user's eye movements. In some embodiments, the user's field of view is determined by tracking the user's head movement. In some embodiments, the user's field of view is determined by tracking the movement of the handheld device.

  In some embodiments, sponsored content includes virtual objects in a virtual environment. In some such embodiments, interaction between the user and the virtual object is detected and additional sponsored content is presented to the user.

Fig. 4 illustrates a process for interaction between a user and a spatial sound based advertising system according to the present disclosure. 4 illustrates a method for spatial sound-based advertising according to an embodiment of the present disclosure. 4 shows a computation node according to an embodiment of the present invention.

  In an immersive environment, a user does not necessarily notice the presence of content outside his or her field of view. This raises certain challenges for advertising. Even if the advertiser may have placed the content in a virtual reality (VR) or augmented reality (AR) environment, the interactivity of the VR or AR environment results in the user unnoticed to miss the sponsored content. There are cases. Since the sponsored content is not visible to the user during the VR experience, the user does not necessarily know that it is located throughout the experience. Accordingly, the present disclosure provides a spatial audio cue that informs and guides a user to sponsored content.

  According to various embodiments of the present disclosure, spatial sound is provided to guide a user to sponsored content located throughout the VR experience. Interactive advertisements are customized presentations that are developed and embedded within the body of VR content. The user can interact with the form of the 3D object and the augmented reality animation in the VR experience.

  In various embodiments, an enhanced graphic animation that appears in a VR environment is provided. Thus, the sponsored content and the 3D object that renders the sponsored content may be displayed when the user turns their eyes during the VR experience. For example, a product placement may be included directly in a 3D scene, for example, a brand cola can. Also, a video screen that displays sponsored content when the user is nearby or when the user looks at the screen may be included in the VR environment. More complex arrangements of sponsored objects, such as dynamically adding brand decals to existing objects in a VR scene (eg, to display a logo on passing vehicles) are also available.

  In various embodiments, users of a VR or AR system may indicate the sponsored content or type of sponsored content that they are interested in. The user may also initiate an interaction with the sponsored content and be led to further sponsored content in the VR experience, or direct them to a favorite sponsor's platform outside of the VR environment.

  In this way, spatial sound, interactive 3D objects and augmented reality graphics are merged to create new forms of advertising in a VR experience.

  Of course, various virtual and augmented reality devices are known in the art. For example, various vendors offer various head mounted displays that provide either immersive video or video overlays. Some such devices integrate a smartphone into a headset, which provides computing and wireless communication resources for each virtual or augmented reality application. Some such devices connect to an external computing node, such as a personal computer, via a wired or wireless connection. Still other devices may include integrated computing nodes that provide some or all of the necessary computation and connectivity for a given application.

  The virtual or augmented reality display may be linked to various motion sensors to track the movement of the user in the virtual environment. Such motion tracking may be used to navigate within the virtual environment, manipulate the user's avatar in the virtual environment, and interact with other objects in the virtual environment. In some devices that integrate a smartphone, head tracking may be performed by sensors integrated into the smartphone, such as an orientation sensor, gyroscope, accelerometer, or geomagnetic field sensor. The sensors may be integrated into the headset, or may be carried by the user, and may be mounted on various body parts to obtain detailed information about the user's positioning.

  Of course, various embodiments have application to virtual and augmented reality environments, including those presented generally without a headset. For example, VR or AR magic window implementations use a display on a handheld device, such as a telephone, as a window into virtual space. By moving, swiping, or otherwise interacting with the handheld device, the user moves the view of the screen within the virtual environment. The center of the user's field of view can be determined based on the orientation of the virtual window in virtual space without requiring eye tracking. However, higher accuracy can be obtained with devices that include eye tracking.

  Referring now to FIG. 1, an exemplary user interaction with an advertising system based on spatial sound is depicted in accordance with an embodiment of the present disclosure. A 3D or 360 ° video feed 101 is provided. A spatial audio component 102 is embedded in the video feed. In some embodiments, the audio component includes an audible cue that indicates the presence of sponsored content, such as an advertisement in a VR or AR environment. The acoustic components are spatially oriented within the VR or AR environment to make it appear to originate from the direction of the sponsored content. As the user looks around (103) and directs their attention to the sponsored content, an interactive object may be activated in the VR environment (104) or expanded graphics may appear (105). ). The user may continue with the VR or AR experience (108) or may jump to its sponsored content in a VR or AR environment (106). Once the user interacts with the sponsored content, they may be transferred to the sponsor's platform (107).

  For example, the position sound of opening the cola can may direct the user's attention as a sponsored object to the cola can placed in the VR environment. When it reaches the center of the user's field of view, the can may be opened. Additional sponsored content may be played when the user activates the can, or they may be relocated to a sponsored location in a virtual environment. In addition to presenting the sponsored content in the virtual environment, further interactions may lead the user to content outside the virtual environment, such as a sponsored website.

  Referring now to FIG. 2, a method for spatial audio-based advertising according to an embodiment of the present disclosure is shown. At 201, a spatial audio cue is played to a user. Spatial acoustic cues have an apparent position in the virtual environment. At 202, the user's view is tracked toward the apparent location of the acoustic cue. At 203, the sponsored content is displayed to the user at the approximate apparent location of the audio cue.

  Referring now to FIG. 3, a schematic diagram of an example of a compute node is shown. Cloud computing node 10 is merely an example of a suitable computing node and does not imply any limitation as to the scope of use or functionality of the embodiments of the invention described herein. Nevertheless, computing node 10 may implement and / or perform any of the functionality described above.

  Compute node 10 has a computer system / server 12 that operates with many other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with the computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handhelds. Or laptop device, multiprocessor system, microprocessor based system, set top box, programmable consumer electronics, network PC, mini computer system, mainframe computer system, and distributed comprising any of the above systems or devices Type cloud computing environment.

  Computer system / server 12 may be described in the general context of computer system executable instructions, such as program modules executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer system / server 12 may execute in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, such as memory storage.

  As shown in FIG. 3, the computer system / server 12 of the computing node 10 is shown in the form of a general-purpose computing device. Components of the computer system / server 12 include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components such as the system memory 28 to the processor 16. Can be

  Bus 18 may be any of several types of bus structures, such as a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. Represents one or more of By way of example, and not limitation, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Extended ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus , And a peripheral component interconnect (PCI) bus.

  Computer system / server 12 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system / server 12 and includes both volatile and nonvolatile media, removable and non-removable media.

  System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer system / server 12 may further comprise other removable / non-removable volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be provided for reading from and writing to non-removable, nonvolatile magnetic media (not shown, typically referred to as a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (eg, a "floppy disk"), and a removable non-volatile such as a CD-ROM, DVD-ROM or other optical medium An optical disk drive for reading from and writing to the optical disk can be provided. In such a case, each may be connected to bus 18 by one or more data media interfaces. As further shown and described below, memory 28 includes at least one program product having a set (eg, at least one) of program modules configured to perform the functions of embodiments of the present invention. May be included.

  A program / utility 40 having a set (at least one) of program modules 42 includes, by way of example and not limitation, memory 28 and an operating system, one or more application programs, other program modules and program data. It may be stored. Each of the operating system, one or more application programs, other program modules and program data, or some combination thereof, may include an implementation of a networking environment. Program module 42 generally performs the functions and / or methodologies of the embodiments of the invention described herein.

  Also, computer system / server 12 may include one or more external devices 14, such as a keyboard, pointing device, display 24, one or more devices that allow a user to interact with computer system / server 12, and / or The computer system / server 12 may be in communication with any device that enables the computer system / server 12 to communicate with one or more other computing devices (eg, a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 22. Still further, computer system / server 12 may be connected via network adapter 20 to one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (eg, the Internet). Can communicate. As shown, network adapter 20 communicates with other components of computer system / server 12 via bus 18. Although not shown, it should be understood that other hardware and / or software components may be used with computer system / server 12. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

  The invention may be a system, method and / or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions for causing a processor to perform aspects of the present invention.

  The computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, without limitation, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media is: portable computer diskette, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch card or raised structure having grooves in which instructions are recorded, etc. It includes a mechanically encoded device, and any suitable combination of the above. As used herein, a computer readable storage medium is itself a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, an optical pulse passing through a fiber optic cable). ) Or transient signals such as electrical signals transmitted over wires.

  The computer readable program instructions described herein may be transmitted from a computer readable storage medium to a respective computing / processing device or to an external computer via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Alternatively, it can be downloaded to an external storage device. A network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage on a computer readable storage medium in the respective computing / processing device. I do.

  Computer readable program instructions for performing the operations of the present invention include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, and state-setting data. setting data), or any combination of one or more programming languages, such as an object-oriented programming language such as SmallTalk, C ++, and a conventional procedural programming language such as a "C" programming language or similar programming language. It may be either source code or object code. The computer readable program instructions may be entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely. Alternatively, it may be executed on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, such as a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer. (Eg, over the Internet using an Internet service provider). In some embodiments, an electronic circuit, such as, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), converts state information of computer readable program instructions to implement aspects of the present invention. Utilization may be used to personalize electronic circuitry to execute computer readable program instructions.

  Aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be performed by computer readable program instructions.

  These computers or other programmable data processing devices execute instructions such that the instructions executed via the processors create means for implementing the functions / acts specified in the blocks of the flowcharts and / or block diagrams. The readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to create a machine. These computer-readable program instructions may also include instructions such that the stored computer-readable storage medium includes instructions that implement aspects of the functions / operations specified in the blocks of the flowcharts and / or block diagrams. May be stored on a computer readable storage medium that can direct a computer, a programmable data processing device, and / or other devices to function in a particular manner.

  Also, the computer-readable program instructions may be executed on a computer, such that the instructions executed on the computer, other programmable devices, or other devices implement the functions / acts specified in the blocks of the flowcharts and / or block diagrams. Loading another programmable data processing device, or a series of operating steps, into a computer, another programmable device or other device for execution on another device to create a computer-implemented process. You may.

  The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions, for implementing the specified logical functions. In some other implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or they may sometimes be executed in the reverse order, depending on the functionality involved. Good. Each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may implement dedicated functions or operations or execute dedicated hardware and computer instruction combinations. It should also be noted that this can be implemented by the following system.

  Although descriptions of various embodiments of the present invention have been provided by way of illustration, they are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein best describes the principles of the embodiments, their practical application or technical improvements over the technology found in the marketplace, or the embodiments disclosed by others skilled in the art. Or to be able to understand.

Claims (18)

Playing a spatial audio cue to a user, wherein the spatial audio cue has an apparent position in a virtual environment;
Tracking the user's field of view towards the apparent position of the acoustic cue;
Displaying sponsored content to the user at the approximate location of the audio cue in the virtual environment;
A method that includes   The method of claim 1, wherein the field of view is determined by tracking eye movements of the user.   The method of claim 1, wherein the field of view is determined by tracking a head movement of the user.   The method of claim 1, wherein the field of view is determined by tracking movement of a handheld device.   The method of claim 1, wherein the sponsored content comprises a virtual object in the virtual environment. Detecting interaction between a user and the virtual object;
Presenting further sponsored content to the user;
The method of claim 5, further comprising: A computer program product for spatial audio-based advertising, the computer program product including a computer readable storage medium having program instructions embodied therewith, wherein the program instructions comprise:
Playing a spatial audio cue to a user, wherein the spatial audio cue has an apparent position in a virtual environment;
Tracking the user's field of view towards the apparent position of the acoustic cue;
Displaying sponsored content to the user at the approximate location of the audio cue in the virtual environment;
A computer program product executable by a processor to cause the processor to perform the method comprising:   The computer program product of claim 7, wherein the field of view is determined by tracking eye movements of the user.   The computer program product of claim 7, wherein the field of view is determined by tracking a head movement of the user.   The computer program product of claim 7, wherein the field of view is determined by tracking movement of a handheld device.   The computer program product of claim 7, wherein the sponsored content includes a virtual object in the virtual environment. The method comprises:
Detecting interaction between a user and the virtual object;
Presenting further sponsored content to the user;
The computer program product of claim 11, further comprising: A computing node comprising a computer readable storage medium having program instructions embodied therewith, wherein said program instructions comprise:
Playing a spatial audio cue to a user, wherein the spatial audio cue has an apparent position in a virtual environment;
Tracking the user's field of view towards the apparent position of the acoustic cue;
Displaying sponsored content to the user at the approximate location of the audio cue in the virtual environment;
A system comprising a compute node executable by the processor to cause the processor of the compute node to perform the method comprising:   14. The system of claim 13, wherein the field of view is determined by tracking eye movements of the user.   14. The system of claim 13, wherein the field of view is determined by tracking head movement of the user.   14. The system of claim 13, wherein the field of view is determined by tracking movement of a handheld device.   14. The system of claim 13, wherein the sponsored content comprises a virtual object in the virtual environment. The method comprises:
Detecting interaction between a user and the virtual object;
Presenting further sponsored content to the user;
18. The system of claim 17, further comprising:

Images