TWI779961B - Sound playback method equipment and computer-readable storage medium - Google Patents

Abstract

The embodiments of the present disclosure relate to a sound playback method, equipment, and computer-readable storage medium. The method includes: determining a first position of the AR device in an augmented reality AR scene; and determining a virtual object in the AR scene according to the first position, obtain the weather conditions of the real geographic location where the AR device is located; in the case that the virtual object can make a sound based on the weather conditions, play the sound through the AR device.

Description

Sound playback method, device, and computer-readable storage medium

The present invention relates to the field of computer technology, in particular to a sound playing method, device, and computer-readable storage medium.

Augmented reality (augment reality, AR) technology can combine real world information and virtual world information, apply virtual information to the real world, and be perceived by human senses, so as to achieve a sensory experience beyond reality. In AR technology, the real environment and virtual objects are superimposed on the same screen in real time, and in the screen, the virtual objects can match the real environment.

In AR technology, better combining virtual information with the real environment can better improve user experience.

The embodiment of the present invention proposes a sound playback technical solution.

According to an aspect of the embodiments of the present invention, a sound playing method is provided, including: In the augmented reality AR scene, determine the first position of the AR device; determining a virtual object in the AR scene according to the first position; Obtaining the weather condition of the real geographic location where the AR device is located; If the virtual object can emit a sound based on the weather condition, the AR device is used to play the sound.

According to an aspect of the embodiments of the present invention, a sound playback device is provided, including: The first position determination module is configured to determine the first position of the AR device in the augmented reality AR scene; A virtual object determination module configured to determine a virtual object in the AR scene according to the first position; The weather condition determining module is configured to obtain the weather condition of the real geographic location where the AR device is located; The sound playing module is configured to play the sound through the AR device when the virtual object can make a sound based on the weather condition.

According to an aspect of an embodiment of the present invention, there is provided an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory, to perform the above method.

According to an aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.

According to an aspect of the embodiments of the present invention, a computer program is provided, including computer readable codes, and when the computer readable codes are run in an electronic device, a processor in the electronic device is executed to implement the above method.

According to an aspect of the embodiments of the present invention, a computer program product is provided, the computer program product includes one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the steps in the above method.

In the embodiment of the present invention, when the user is using the AR device, the sound played by the AR device takes into account the influence of the weather conditions of the user's location on the virtual objects in the AR scene, and the virtual objects can emit sounds based on the weather conditions. In some cases, the sound is played through the AR device, so that the virtual information played by the AR device can be better combined with the real situation, the user experience is more real, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. Other features and aspects of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Various exemplary embodiments, features, and aspects of the invention will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just a description of the associated relationship between related objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. three conditions. In addition, the term "at least one" herein means any one or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including the composition consisting of A, B, and C Any one or more elements selected in the collection.

In addition, in order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. It will be understood by those skilled in the art that the present invention may be practiced without certain of the specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail in order to highlight the gist of the present invention.

Augmented reality, also known as mixed reality, is an emerging technology developed on the basis of virtual reality, which can superimpose a virtual scene generated by a computer on the real environment seen by the user.

Augmented reality technology superimposes computer-generated virtual objects or system prompt information on the real scene, thereby realizing the "amplification" of reality. For example, computer-generated virtual objects or information about real objects can be superimposed on images of the real world captured by the image acquisition device, thereby realizing the amplification of the real world.

In the embodiment of the present invention, when the user is using the AR device, the sound played by the AR device takes into account the influence of the weather conditions of the user's location on the virtual objects in the AR scene, and the virtual objects can emit sounds based on the weather conditions. In some cases, the sound is played through the AR device, so that the virtual information played by the AR device can be better combined with the real situation, the user experience is more real, and the user experience is improved.

The sound playing method provided by the embodiment of the present invention can be executed by electronic equipment such as a terminal device or a server, and the terminal device can be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, wireless phone, For a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc., the method can be realized by calling a computer-readable instruction stored in a memory by a processor. Alternatively, the method may be performed by a server.

Fig. 1 shows a flowchart of a sound playing method according to an embodiment of the present invention. As shown in Fig. 1 , the sound playing method includes the following steps.

In step S11, in the augmented reality AR scene, a first position of the AR device is determined.

The AR scene is a virtual scene, and the scene may be a virtual scene constructed based on a real scene. For example, in AR technology, the scene model will be constructed by spatially reconstructing the geographical objects in the real scene. The scene model is used to represent the 3D structure in the real scene, which can be obtained through 3D reconstruction technology, for example, through motion Restore structure (Structure From motion, SFM) space reconstruction technology to construct a three-dimensional scene model, where the scene model can be used as an AR scene.

When the AR device is at an actual position in the real scene, it can scan the surrounding scene information (such as image, depth information), and then use the scene information to match with the information in the AR scene (such as the scene model), and the match is successful After that, the first position of the AR device in the AR scene can be determined.

In step S12, an outdoor virtual object in the AR scene is determined according to the first position.

The virtual objects in the AR scene are displayed on the display interface of the AR device, and the virtual objects do not exist in the real scene where the user is. The image acquisition module of the AR device can also collect images of the real scene where the user is, and It is displayed in the display interface of the AR device in real time.

The arrangement of virtual objects in the AR scene can be realized based on the scene model mentioned above. In some implementations, in the process of displaying virtual objects in the real scene through the visual interface, the constructed scene model will be matched with the real scene collected by the AR device in real time. After the matching is successful, the scene model can be set The virtual object at a certain position in the AR device is displayed at the corresponding position in the real scene collected by the AR device in real time, realizing the placement of the virtual object in the real scene.

Obviously, the position of the virtual object in the AR scene is known, so the outdoor virtual object in the AR scene can be determined according to the first position of the AR device in the AR scene. The outdoor virtual object determined here may be a virtual object within a preset distance around the first location, or a virtual object visible to the user placed around the first location, and so on.

Considering that outdoor virtual objects are often affected by the weather, it is possible to determine the outdoor virtual objects in the AR scene. As mentioned above, the position of the virtual object is known. Then, according to the position of the virtual object, it can be Determine whether the virtual object is indoors or outdoors. In addition, the outdoor here can be the outdoor in the AR scene, and developers can pre-calibrate whether the virtual object is located indoors or outdoors according to the position of the virtual object to be placed.

In step S13, the weather condition of the real geographic location where the AR device is located is acquired.

The weather condition of the real geographic location where the AR device is located may be the weather condition of the user's geographic location in the real scene.

There are many ways to determine the weather conditions of the real geographic location where the AR device is located. For example, based on the geographic location of the AR device in the real scene, the AR device can be quickly obtained by obtaining the current weather conditions of the geographic location predicted by the weather forecast. The weather conditions of the real geographical location; in addition, the image acquisition equipment based on the AR device can also collect the outdoor image information in the real scene, and determine the weather in the image through image analysis technology according to the outdoor image information conditions, so as to determine the weather conditions of the real geographic location where the AR device is located, so as to accurately determine the weather conditions of the real geographic location where the AR device is located.

In step S14, if the virtual object can emit sound based on the weather condition, play the sound through the AR device.

The virtual object can emit sound based on the weather conditions. It can be that the real object corresponding to the virtual object can emit sound due to the influence of the weather condition. If it makes a sound when hit, it can be determined that the virtual object "tree" can make a sound based on the weather conditions.

Then, in the case that the virtual object can emit a sound based on the weather condition, the sound can be played through the AR device, and the user's interaction experience is better. The sound can be superimposed and played with other sounds in the AR scene, or played separately. The AR device can play the sound through the built-in speaker of the device, or can play the sound through the playback device connected to the device.

In the embodiment of the present invention, when the user is using the AR device, the sound played by the AR device takes into account the impact of the weather conditions at the user's location on the virtual objects in the AR scene. In the case that the virtual object can emit sound based on the weather conditions, the sound is played through the AR device, so that the virtual information played by the AR device can be better combined with the real situation, and the user experience is more realistic and the user experience is improved.

In a possible implementation manner, after acquiring the weather condition of the real geographic location where the AR device is located and before playing the sound through the AR device, the method further includes: detecting the weather condition Whether there is a sounding condition for generating sound; if the sounding condition is detected, it is determined that the virtual object can make a sound based on the weather condition.

Considering that virtual objects do not necessarily make sounds under certain weather conditions, for example, in sunny and windless weather conditions, outdoor trees and water surfaces often do not make sounds; while under certain weather conditions, they will make sounds, such as , the outdoor trees will make a sound when they are blown by the wind, and the outdoor trees, ground, and water will make a sound when they are slapped by rain. Therefore, it is possible to detect whether or not there is a utterance condition that produces sound.

The sounding condition may include at least one of the following: there is an airborne object in the weather condition; there is wind in the weather condition, and the virtual object can make a sound when it is blown by the wind. Airfalls include at least one of raindrops, hail, and snow. For further examples of sounding conditions, refer to the possible implementation manners provided later, which are not limited in this embodiment of the present invention.

In the embodiment of the present invention, after the virtual object in the AR scene is determined, it can be detected whether there is a sounding condition for producing sound, and if the sounding condition is detected, it is determined that the virtual object can make sound based on the weather condition By playing the sound through the AR device, the user experience is more real and the user experience is improved.

In a possible implementation manner, the sounding condition includes the existence of airborne falling objects under the weather conditions, and the airborne falling objects include at least one of raindrops, hail, and snow; The sounding condition of the sound includes: determining whether there is the falling object in the air according to the weather condition; in the case of detecting the sounding condition, determining that the virtual object can make a sound based on the weather condition, including: In the presence of the airborne object, it is determined that the virtual object is capable of making a sound based on the weather condition.

The falling objects in the air may be objects that fall naturally under certain weather conditions, for example, raindrops falling on a rainy day, hailstones falling on a hailstorm, snowflakes falling on a snowy day, and the like. Considering that airborne objects can often make sounds when they come into contact with outdoor virtual objects, for example, outdoor trees, ground, and water surfaces will make sounds when they are hit by rain. Therefore, the existence of airborne objects under weather conditions can be regarded as a possible sounding condition.

In the process of determining whether there is an airborne fallout according to the weather conditions, image analysis can be performed on the image collected by the image acquisition device of the AR device to determine whether there is an airborne fallout; the weather condition of the first location can also be acquired , according to the weather conditions to determine whether there is an airborne drop, when it is determined that the weather at the first location is rain, snow, hail, etc., then it can be determined that there is an airborne drop.

In the embodiment of the present invention, when it is determined based on the weather conditions that there is a falling object in the air, it can be determined that the virtual object can make a sound based on the weather conditions, which can more accurately reflect the situation in the real world to the AR scene. Therefore, the sound is played through the AR device, making the user experience more real and improving the user experience.

In a possible implementation manner, the sounding condition includes that there is wind under the weather condition, and the virtual object can make a sound when it is blown by the wind; the detecting whether there is a sounding condition for producing sound under the weather condition includes : Determine the wind level in the weather condition and the type of the virtual object; determine whether the virtual object can produce sound under the wind level according to the pre-stored sounding relationship library and the type of the virtual object.

Considering that an outdoor virtual object can often make a sound when it is blown by the wind, for example, an outdoor tree will make a sound when it is blown by the wind. Therefore, there is wind in weather conditions, and the virtual object can make a sound when it is blown by the wind, which can be regarded as a possible sounding condition.

Considering that some virtual objects may only be blown by the wind to produce sound when the wind force reaches a certain level, for example, a flag will not make a sound when it is blown by a breeze with a wind force level of one, but when the flag is blown by a wind force level of three When the wind blows, there is a high probability that a sound will be produced. Therefore, in the process of determining whether there is a sound producing condition, the wind level in the weather condition and the type of the virtual object can be determined, and then it is determined whether the object of this type can produce sound under the determined wind level.

The sounding relationship library can be built in advance, and the corresponding relationship between the object type and the lowest wind force level that can produce sound can be stored in the sounding relationship library. The example of the corresponding relationship is as follows: trees correspond to the second-level wind force, and textiles correspond to the third-level wind force . As for the type of virtual object, the type of virtual object placed in the AR scene can be pre-calibrated, and the type of virtual object can also be identified based on image recognition technology.

In the process of determining the wind force level in the weather conditions, the wind force level of the first position predicted in the weather forecast can be obtained, or the wind force level of the first position can also be detected based on the microphone of the AR device. The method of the wind level is not specifically limited in this embodiment of the present invention.

After determining the wind force level in the weather condition and the type of the virtual object, the corresponding wind force level can be determined from the sounding relationship library according to the type. If the determined wind level is smaller than the wind level in the weather condition, the virtual object can produce sound under the current weather condition; if the determined wind level is greater than the wind level in the weather condition, the virtual object cannot produce sound under the current weather condition.

In the embodiment of the present invention, it is determined whether the virtual object can make a sound based on the weather condition according to the wind level in the weather condition and the type of the virtual object, which can more accurately reflect the situation in the real world to the AR scene, so that the user can The user experience is more real and the user experience is improved.

In a possible implementation manner, after detecting whether there is a sounding condition for generating sound under the weather condition, the method further includes: if the sounding condition is detected, determining the sounding condition according to the sounding condition The virtual object emits a sound based on the weather condition.

Different sounding conditions may cause virtual objects to make different sounds based on weather conditions. Therefore, in the embodiment of the present invention, the sound emitted by the virtual object based on the weather condition may be determined according to the sound emission condition when the sound emission condition is detected.

For example, different virtual objects may produce different sounds under the influence of the same weather condition; falling objects in different weather conditions may produce different sounds when interacting with the same virtual object. Therefore, the sound produced by the virtual object based on the weather condition can be determined according to the sound production condition, so that the sound played by the AR device is more realistic and the user experience is improved. The embodiment of the present invention does not limit the sound emitted in different situations.

In a possible implementation manner, when the sounding condition includes the presence of airborne objects under the weather condition, the determining the sound produced by the virtual object based on the weather condition according to the sounding condition includes: : According to the obtained weather conditions of the real geographic location where the AR device is located, determine the type of the airborne drop under the weather conditions, and the weather level of the airborne drop; according to the airborne drop The type determines the sound produced by the airborne falling object falling on the virtual object; according to the weather level, the volume of the produced sound is determined.

The type of falling objects in the air can be determined according to the weather conditions of the real geographical location where the AR device is located. For example, when the weather is raining, the falling objects are raindrops; when the weather is snowing, the falling objects are snowflakes; when the weather is hail, then The fallout was hail.

When different types of falling objects interact with virtual objects, the sounds produced are different. You can pre-store the sounds of different types of falling objects interacting with virtual objects. When the type of falling objects is determined, you can obtain the pre-stored the sound of.

The meteorological level here can be the level used to indicate the strength of the weather. For example, when the weather is rainy, it is used to indicate the level of rainfall, which is often divided into multiple levels such as light rain, moderate rain, and heavy rain; when the weather is snowfall, it is often divided into Light snow, moderate snow, heavy snow and other levels. Considering that under different weather levels, the volume of the sound produced by the falling objects interacting with the virtual object is different. Therefore, the volume of the sound can be determined according to the weather level, and the volume is proportional to the weather level. For example, the greater the rainfall level, the louder the sound.

In the embodiment of the present invention, according to the acquired weather conditions of the real geographic location where the AR device is located, the type of airborne objects under the weather conditions and the meteorological level of the airborne objects are determined, and then determined according to the type of airborne objects The sound produced by the airborne falling object falling on the virtual object, and according to the weather level, determine the volume of the produced sound. The sound emitted by the AR device is determined according to the type of falling objects in the air and the weather level in the real weather conditions, which can more accurately reflect the situation in the real world to the AR scene. The AR device thus determined The sound played is more realistic, which improves the user experience.

In a possible implementation manner, when the sounding conditions include wind in the weather condition, and the virtual object can make a sound when it is blown by the wind, the determining the virtual object according to the sounding condition is based on The sound emitted by the weather conditions includes: according to the pre-stored sound relation library and the type of the virtual object, determining the sound type of the sound emitted by the virtual object when it is blown by the wind; The volume at which the sound is emitted.

The sounds produced by different types of virtual objects are different. For example, when a tree is blown by the wind, it will make a "rustle" sound, while a textile object (such as a flag) will make a "whoosh" sound when it is blown by the wind. sound. Therefore, based on the type of the virtual object, the sound type of the sound produced by the virtual object when it is blown by the wind can be determined.

The relationship between the type of the virtual object and the sound type of the sound produced when it is blown by the wind can be pre-stored in the sounding relationship library, based on the sounding relationship library, the sound type of the sound corresponding to the determined virtual object type can be obtained.

The wind level is used to indicate the strength of the wind. When the wind is stronger, the sound from the virtual object will be louder; when the wind is weaker, the sound from the virtual object will be smaller. Therefore, the volume of the sound emitted by the virtual object can be determined according to the wind force level.

In the embodiment of the present invention, the sound emitted by the virtual object when blown by the wind is determined according to the pre-stored sound emission relationship library and the type of the virtual object, and the volume of the sound emitted is determined according to the wind force level. Therefore, the sound emitted by the AR device is determined according to the real wind level and the type of the virtual object, which can more accurately reflect the situation in the real world to the AR scene, and the sound played by the AR device thus determined is more accurate. Authentic, improving the user experience.

In a possible implementation manner, after obtaining the weather conditions of the real geographic location where the AR device is located, the method further includes: in the case of falling objects in the air under the weather conditions, on the virtual object Attach the drop.

In the embodiment of the present invention, if there is a drop in the air under the weather conditions, the drop can be attached to the virtual object, and the amount of the drop attached to the virtual object can be positively correlated with the meteorological level of the drop in the air. For example, the higher the snowfall level, the greater the amount of falling matter attached to the virtual object. Thus, the situation in the real world can be reflected in the AR scene, so that the picture displayed by the AR device is more realistic, and the user experience is improved.

In a possible implementation manner, after acquiring the weather condition of the real geographic location where the AR device is located, the method further includes: determining the current outdoor weather condition according to the acquired weather condition of the real geographic location where the AR device is located. Visibility of the air: according to the distance between the virtual object and the first position, and the visibility, the virtual object is blurred.

The weather condition may also include visibility, which may be the maximum distance at which a person with normal vision can recognize the object from the background, and the visibility may be acquired through an interface of a third-party service that provides weather services.

The degree of blurring the virtual object can be determined according to the distance and visibility between the virtual object and the first position. When the distance between the virtual object and the first position is 0, no blurring process is performed; when the distance between the virtual object and the first position is greater than the visibility, the virtual object is completely blurred and not displayed; When the distance between the virtual object and the first position is less than the visibility, the percentage for blurring the virtual object may be: dividing the distance between the virtual object and the first position by the visibility. For example, assuming that the visibility is 100 meters and the virtual object is 50 meters away from the first position, the virtual object is displayed after being blurred by 50%.

In the embodiment of the present invention, virtual objects are blurred and displayed according to the visibility in weather conditions, thereby reflecting the situation in the real world to the AR scene, making the picture displayed by the AR device more realistic and improving the user experience.

It can be understood that the above-mentioned method embodiments mentioned in the present invention can all be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above-mentioned method in the specific implementation manner, the actual execution order of each step should be determined by its function and possible internal logic.

In addition, the embodiments of the present invention also provide sound playback devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any sound playback method provided by the embodiments of the present invention. For the corresponding technical solutions and descriptions, refer to the method section corresponding record.

Fig. 2 shows the block diagram of the sound playing device according to the embodiment of the present invention, as shown in Fig. 2, described device 20 comprises: The first position determination module 21 is configured to determine the first position of the AR device in the augmented reality AR scene; The virtual object determining module 22 is configured to determine the virtual object in the AR scene according to the first position; The weather condition determining module 23 is configured to obtain the weather condition of the real geographic location where the AR device is located; The sound playing module 24 is configured to play the sound through the AR device when the virtual object can make a sound based on the weather condition.

In a possible implementation manner, the device 20 further includes: A detection module configured to detect whether there is a sounding condition for generating sound under the weather conditions; The sounding determination module is configured to determine that the virtual object can make a sound based on the weather condition when the sounding condition is detected.

In a possible implementation manner, the sounding condition includes the existence of airborne falling objects under the weather conditions, and the airborne falling objects include at least one of raindrops, hail, and snow; The detection module is configured to determine whether there is the airborne drop according to the weather condition; The sounding determination module is configured to determine that the virtual object can make a sound based on the weather condition in the presence of the falling object in the air.

In a possible implementation manner, the sounding condition includes that there is wind in the weather condition, and the virtual object can make a sound when it is blown by the wind; The detection module is configured to determine the wind level in the weather conditions and the type of the virtual object; according to the pre-stored sounding relationship library and the type of the virtual object, determine that the virtual object is at the wind level to see if sound can be produced.

In a possible implementation manner, the device 20 further includes: The sound determination module is configured to determine the sound emitted by the virtual object based on the weather condition according to the sound emission condition when the sound emission condition is detected.

In a possible implementation manner, when the sounding condition includes the presence of airborne falling objects under the weather condition, the sound determination module includes: The falling object determining module is configured to determine the type of the airborne falling object under the weather condition and the meteorological level of the falling airborne falling object according to the acquired weather conditions of the real geographic location; The first sound type determination module is configured to determine the sound type of the sound made by the airborne object falling on the virtual object according to the type of the airborne object; The first volume determination module is configured to determine the volume of the emitted sound according to the weather level.

In a possible implementation manner, when the sounding conditions include wind under the weather condition, and the virtual object can make a sound when it is blown by the wind, the sound determining module includes: The second sound type determination module is configured to determine the sound type of the sound emitted by the virtual object when it is blown by the wind according to the pre-stored sounding relationship library and the type of the virtual object; The second volume determination module is configured to determine the volume of the emitted sound according to the wind force level.

In a possible implementation manner, the device 20 further includes: The additional module is configured to add the falling objects on the virtual object when there are falling objects in the air under the weather conditions.

In a possible implementation manner, the device 20 further includes: The visibility determining module is configured to determine the visibility according to the acquired weather conditions of the real geographic location; The blur display module is configured to perform blur display on the virtual object according to the distance between the virtual object and the first position and the visibility.

In some embodiments, the functions of the device provided in the embodiments of the present invention or the included modules can be configured to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments.

The embodiment of the present invention also proposes a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. The computer readable storage medium may be a non-volatile computer readable storage medium.

An embodiment of the present invention also proposes an electronic device, including: a processor; a memory configured to store instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present invention also provides a computer program product, including computer-readable codes. When the computer-readable codes are run on the device, the processor in the device executes the method for implementing the image detection method provided in any of the above embodiments. instruction.

The embodiment of the present invention also provides another computer program product, which is used for storing computer-readable instructions. When the instructions are executed, the computer executes the operation of the image detection method provided by any of the above-mentioned embodiments.

An embodiment of the present invention also provides a computer program, including computer-readable codes, when the computer-readable codes run in an electronic device, the processor in the electronic device executes the program for implementing the program provided by any of the above-mentioned embodiments. Image detection method.

Electronic devices may be provided as terminals, servers, or other types of devices.

Fig. 3 shows a block diagram of an electronic device 800 according to an embodiment of the present invention. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

Referring to FIG. 3, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (Input/Output, I/O) interface 812 , sensor component 814 and communication component 816 .

The processing component 802 generally controls the overall operations of the electronic device 800, such as those associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support operations of the electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact information, phonebook information, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM, Static Random-Access Memory), electrically erasable programmable read-only Memory (EEPROM, Static Random-Access Memory), Erasable Programmable Read-Only Memory (EPROM, Electrically Erasable Programmable Read-Only Memory), Programmable Read-Only Memory (PROM, Programmable Read-Only Memory) , Read Only Memory (ROM, Read Only Memory), Magnetic Memory, Flash Memory, Disk or CD.

The power supply component 806 provides power to various components of the electronic device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD, Liquid Crystal Display) and a touch panel (TP, Touch Panel). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect a duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC, Microphone). When the electronic device 800 is in an operation mode, such as a calling mode, a recording mode and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 804 or sent via the communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing unit 802 and peripheral interface modules, such as a keyboard, a click wheel, and buttons. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for electronic device 800 . For example, the sensor assembly 814 can detect the open/closed state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800, and the sensor assembly 814 can also detect the electronic device 800 or The position of a component of the electronic device 800 changes, the presence or absence of user contact with the electronic device 800 , the orientation or acceleration/deceleration of the electronic device 800 and the temperature of the electronic device 800 change. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor component 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS, Complementary Metal-Oxide-Semiconductor) or a charge-coupled device (CCD, Charge Coupled Device) image sensor, for imaging used in the application. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access wireless networks based on communication standards, such as wireless networks (Wi-Fi, Wireless Fidelity), second-generation mobile communication technologies (2G, The 2nd Generation) or third-generation mobile communication technologies (3G, The 3rd Generation), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC, Near Field Communication) module to facilitate short-range communication. For example, the NFC module can be based on radio frequency identification (RFID, Radio Frequency Identification) technology, infrared data association (IrDA, Infrared Data Association) technology, ultra-wideband (UWB, Ultra Wide Band) technology, Bluetooth (BT, Blue Tooth) technology and other technologies to achieve.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASIC, Application Specific Integrated Circuit), digital signal processors (DSP, Digital Signal Processor), digital signal processing devices (DSPD, Digital Signal Processing Device), Programmable Logic Device (PLD, Programmable Logic Device), Field Programmable Gate Array (FPGA, Field Programmable Gate Array), controller, microcontroller, microprocessor or other electronic components, using to perform the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to implement the above method.

Fig. 4 shows a block diagram of an electronic device 1900 according to an embodiment of the present invention. For example, electronic device 1900 may be provided as a server. Referring to FIG. 4 , the electronic device 1900 includes a processing component 1922 , which may include one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922 , such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input/output (I/O ) interface 1958. The electronic device 1900 can operate based on the operating system stored in the memory 1932, such as the Microsoft server operating system (Windows ServerTM), the operating system based on the graphical user interface (Mac OS XTM) launched by Apple Inc., multi-user and multi-process A computer operating system (UnixTM), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to implement the above method.

Embodiments of the invention may be systems, methods and/or computer program products. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present invention.

A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Computer-readable storage media can include: portable computer disk, hard disk, random access memory (RAM, Random Access Memory), read-only memory, erasable programmable read-only memory (EPROM or flash Memory), Static Random Access Memory, Portable Compressed Disk Read-Only Memory (CD-ROM, Compact Disc Read-Only Memory), Digital Versatile Disk (DVD, Digital Video Disc), Memory Stick, Soft Discs, mechanical encoding devices, such as punched cards with instructions stored thereon or raised structures in grooves, and any suitable combination of the foregoing. A computer-readable storage medium as used herein is not to be construed as a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other Electrical signals transmitted by wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in computer-readable storage in each computing/processing device medium.

The computer program instructions used to execute the operations of the embodiments of the present invention may be assembly instructions, instruction set architecture (ISA, Industry Standard Architecture) instructions, machine instructions, machine-related instructions, pseudocode, firmware instructions, state setting data, or in one or Source code or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" or similar programming language. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer. Execute on the terminal computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer via any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to An external computer (e.g. using an Internet service provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays, or programmable logic arrays, can be customized by utilizing state information of computer-readable program instructions, which can execute computer-programmable Program instructions are read, so as to realize various aspects of the embodiments of the present invention.

Aspects of embodiments of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart and/or block diagram and combinations of blocks in the flowchart and/or block diagram can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data-processing device, thereby producing a machine such that the instructions are executed by the processor of the computer or other programmable data-processing device When, means for realizing the function/action specified in one or more blocks in the flowchart and/or block diagram are produced. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing device and/or other equipment to operate in a specific manner, so that the computer-readable medium storing the instructions is An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in the flowchart and/or block diagrams is included.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer The process of implementation, so that instructions executed on computers, other programmable data processing devices, or other equipment implement the functions/actions specified in one or more blocks in the flowchart and/or block diagram.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction that contains one or more logic for implementing the specified Executable instructions for a function. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action. implemented, or may be implemented using a combination of dedicated hardware and computer instructions.

The computer program product can be realized by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Having described various embodiments of the present invention, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Industrial Applicability In the embodiment of the present invention, when the user is using the AR device, the sound played by the AR device takes into account the impact of the weather conditions at the user's location on the virtual objects in the AR scene. In the case that the virtual object can emit sound based on the weather conditions, the sound is played through the AR device, so that the virtual information played by the AR device can be better combined with the real situation, and the user experience is more realistic and the user experience is improved.

20: Sound playback device 21: The first position determination module 22: Virtual object determination module 23: Weather condition determination module 24: Sound player module 800: Electronic equipment 802: processing components 804: memory 806: Power components 808:Multimedia components 810:Audio components 812: input/output interface 814: Sensor component 816:Communication component 820: Processor 900: Electronic equipment 1922: Processing components 1926: Power components 1932: Memory 1950: Web interface 1958: Input/Output Interface S11~S14: Steps

The accompanying drawings here are incorporated into the specification and constitute a part of the specification. These drawings show embodiments consistent with the present invention, and are used together with the description to explain the technical solution of the present invention. Fig. 1 shows a flowchart of a sound playing method according to an embodiment of the present invention. Fig. 2 shows a block diagram of a sound playback device according to an embodiment of the present invention. Fig. 3 shows a block diagram of an electronic device according to an embodiment of the present invention. Fig. 4 shows a block diagram of an electronic device according to an embodiment of the present invention.

S11~S14: Steps

Claims (9)

A sound playing method for electronic devices, comprising: in an augmented reality AR scene, based on the scene information obtained by the AR device scanning around the real scene, determining a first position of the AR device; wherein, the first position is the position of the AR device in the AR scene; according to the first position, determine a virtual object whose position is known in the AR scene; wherein, the virtual object in the AR scene is superimposed on the same scene as the real scene In the screen; obtain the weather condition of the real geographical location where the AR device is located; determine whether there is the airborne object according to the weather condition; the airborne object includes at least one of raindrops, hail, and snow; In the case of the above-mentioned falling objects in the air, it is determined that the virtual object can emit sound based on the weather condition; The sound emitted by the weather state; wherein, different types of virtual objects emit different sounds based on the same weather state. According to the method described in claim 1, wherein, whether there is a sounding condition for producing sound under the weather condition includes wind under the weather condition, and the virtual object can produce sound when it is blown by the wind; the method further includes : determine the wind level in the weather condition, and the type of the virtual object; Determine whether the virtual object can produce sound under the wind level according to the pre-stored sounding relationship library and the type of the virtual object. The method according to claim 1, wherein, after determining whether there is the airborne drop according to the weather condition, the method further includes: if the presence of the airborne drop is detected, according to the The airborne object determines the sound made by the virtual object based on the weather condition. The method according to claim 3, wherein, in the case of detecting the presence of the airborne object, determining the sound made by the virtual object based on the weather condition according to the airborne object includes: according to the acquired According to the weather condition of the real geographic location, the type of the airborne drop under the weather conditions is determined, and the meteorological grade of the airborne drop is landed; the airborne landing is determined according to the type of the airborne drop The sound type of the sound emitted by the object falling on the virtual object; according to the weather level, the volume of the emitted sound is determined. The method according to claim 3, wherein, the method further includes: determining the wind level in the weather conditions and the type of the virtual object; determining the The sound type of the sound emitted by the virtual object when it is blown by the wind; the volume of the sound emitted is determined according to the wind force level. The method according to claim 1, wherein, after obtaining the weather conditions of the real geographic location where the AR device is located, the method further includes: in the case where the aerial drop exists under the weather conditions, The aerial drop is added to the virtual object. The method according to claim 1, wherein, after acquiring the weather conditions of the real geographic location where the AR device is located, the method further includes: determining the visibility according to the acquired weather conditions of the real geographic location; The distance between the virtual object and the first position, and the visibility are blurred for the virtual object, and the blurred virtual object is displayed. An electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to perform any one of request items 1 to 7 the method described. A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is realized.

Images