CN117202082A - Panoramic sound playing method, device, equipment, medium and head-mounted display equipment - Google Patents

Abstract

The embodiment of the invention discloses a panoramic sound playing method, a device, equipment, a medium and head-mounted display equipment. One embodiment of the method comprises the following steps: acquiring listening position information of a user terminal; determining listening sound field information corresponding to the listening position information; based on the listening sound field information, carrying out playback processing on preset panoramic play characteristic information to obtain panoramic play sound signals; and driving the loudspeaker terminal to play the audio so that the user terminal can hear the panoramic playing sound signal. According to the embodiment, the accuracy of panoramic sound playing and the picture-sound synchronization rate of the display device are improved.

Description

Panoramic sound playing method, device, equipment, medium and head-mounted display equipment

Technical Field

The embodiment of the disclosure relates to the technical field of computers, in particular to a panoramic sound playing method, a device, equipment, a medium and head-mounted display equipment.

Background

When a user uses a head-mounted display device, panoramic video and panoramic sound need to be played for the user. Currently, when panoramic sound playing is performed, the following methods are generally adopted: setting a microphone at a fixed position simulating a human head, recording panoramic sound audio, determining a virtual sound field at the fixed position, and playing panoramic sound according to the constructed historical virtual sound field when a user uses the panoramic sound field.

However, the inventors found that when panoramic sound playback is performed in the above manner, there are often the following technical problems:

when the panoramic audio is played, panoramic audio can be played only according to the historical virtual sound field on a limited plurality of fixed positions, for example, when the conversion range of the visual field of a user is large, the virtual sound field is not changed, so that the matching effect with the picture of the panoramic video is reduced, the accuracy of panoramic audio playing is reduced, and the picture-sound synchronization rate of the display equipment is deteriorated.

The above information disclosed in this background section is only for enhancement of understanding of the background of the inventive concept and, therefore, may contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country.

Disclosure of Invention

The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose panoramic sound playback methods, apparatuses, devices, media, and head-mounted display devices to address one or more of the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide a panoramic sound playing method, including: acquiring listening position information of a user terminal; determining listening sound field information corresponding to the listening position information; based on the listening sound field information, replaying the preset panoramic play characteristic information to obtain a panoramic play sound signal; and driving the speaker terminal to play the audio so that the user terminal can hear the panoramic play sound signal.

In a second aspect, some embodiments of the present disclosure provide a head mounted display device comprising: the position and orientation sensor is used for collecting the information of the listening position of the user terminal; the central processing unit is in communication connection with the pose sensor, and is used for determining virtual sound field information corresponding to the listening position information, playing back preset panoramic play characteristic information based on the virtual sound field information and determining panoramic play sound signals; the display is in communication connection with the central processing unit and is used for displaying virtual visual information corresponding to the virtual sound field information; and the loudspeaker is in communication connection with the central processing unit, and is used for playing audio so that the user terminal can hear the panoramic playing sound signal.

In a third aspect, some embodiments of the present disclosure provide a panoramic sound playback device, the device comprising: an acquisition unit configured to acquire listening position information of a user terminal; a determination unit configured to determine listening sound field information corresponding to the listening position information; a playback unit configured to perform playback processing on preset panoramic play feature information based on the listening sound field information to obtain a panoramic play sound signal; and the driving unit is configured to drive the loudspeaker terminal to play audio so that the user terminal can hear the panoramic playing sound signal.

In a fourth aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors causes the one or more processors to implement the method described in any of the implementations of the first aspect above.

In a fifth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

The above embodiments of the present disclosure have the following advantageous effects: by the panoramic sound playing method of some embodiments of the present disclosure, accuracy of panoramic sound playing may be improved. Specifically, the degradation of the accuracy of panoramic sound playback is caused by: when playing, panoramic audio can only be played according to the historical virtual sound field at a limited plurality of fixed positions, for example, when the conversion range of the visual field of a user is large, the virtual sound field is not changed, so that the matching effect with the picture of the panoramic video is reduced. Based on this, the panoramic sound playing method of some embodiments of the present disclosure first obtains the listening position information of the user terminal. Next, listening sound field information corresponding to the above-mentioned listening position information is determined. Thus, the position of the user and the sound field information of the position of the user can be confirmed in real time. And then, based on the listening sound field information, carrying out playback processing on the preset panoramic play characteristic information to obtain a panoramic play sound signal. Thus, the sound signal of the panoramic audio to be played at the user terminal can be determined according to the predetermined sound field characteristics. And finally, driving a loudspeaker terminal to play audio so that the user terminal can hear the panoramic play sound signal. Thus, panoramic audio can be played. Therefore, according to some panoramic sound playing methods of the present disclosure, the position information of the user terminal may be acquired in real time, and then, according to the predetermined characteristic information of the virtual sound field, the sound propagation signal of the panoramic sound frequency at the user terminal may be determined, so as to play the audio. Therefore, the sound field information of the position of the user terminal can be determined in real time, and the synchronization rate of the panoramic audio and the panoramic video received by the user terminal and the accuracy of panoramic sound playing can be improved.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.

FIG. 1 is a flow chart of some embodiments of a panoramic sound playback method according to the present disclosure;

FIG. 2 is a schematic structural diagram of some embodiments of a head mounted display device according to the present disclosure;

FIG. 3 is a schematic structural view of some embodiments of a panoramic sound playback device according to the present disclosure;

fig. 4 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates a flow 100 of some embodiments of a panoramic sound playback method according to the present disclosure. The panoramic sound playing method comprises the following steps:

step 101, obtaining the listening position information of the user terminal.

In some embodiments, the execution body of the panoramic sound playing method may acquire the listening position information of the user terminal from the pose sensor by means of wired connection or wireless connection. The pose sensor may be a sensor for acquiring the listening position information. The user terminal may be a terminal that wants to listen to a panoramic play sound signal, including: the auditory system of the user, the microphone system on the device worn by the user. The panorama playing sound signal may be a sound signal of which audio is formed at the user terminal. The listening position information may characterize coordinates of the user terminal in a spherical coordinate system. The above-mentioned listening position information may include, but is not limited to, at least one of: a first position coordinate parameter, a second position coordinate parameter, and a third position coordinate parameter. The first location coordinate parameter may be a radius coordinate of the user terminal in a spherical coordinate system. The second location coordinate parameter may be a polar angle coordinate of the user terminal in a spherical coordinate system. The third location coordinate parameter may be an azimuth coordinate of the ue in a spherical coordinate system.

It should be noted that the wireless connection may include, but is not limited to, 3G/4G connections, wiFi connections, bluetooth connections, wiMAX connections, zigbee connections, UWB (ultra wideband) connections, and other now known or later developed wireless connection means.

Step 102, determining listening sound field information corresponding to the listening position information.

In some embodiments, the execution body may determine the listening sound field information corresponding to the listening position information.

In some optional implementations of some embodiments, the executing body determining the listening sound field information corresponding to the listening position information may include the steps of:

first, a first sound field characteristic basis function sequence is generated based on the listening position information.

And a second step of performing splicing processing on a first sound field characteristic basic function of a first target number in the first sound field characteristic basic function sequence to obtain a first sound field characteristic transfer function. The first target number is equal to the dimension of the panoramic play feature vector included in the preset panoramic play feature information. Here, the first sound field characteristic basis function sequence described above may be truncated (e.g., 2 from 2p+1 degrees of freedom ^p The first sound field characteristic transfer function is obtained by performing nested linear combination processing (i.e., linear combination processing is performed on the internal degrees of freedom of each of the polar sub-functions first and then linear combination processing is performed on a plurality of the polar sub-functions) in order of sequence (for example, the polar sub-functions (ρ=0, 1 degree of freedom in total), the dipole sub-functions (p=1, 3 degrees of freedom in total), and the quadrupole sub-functions (p=2, 5 degrees of freedom in total).

And thirdly, determining the first sound field characteristic transfer function as the listening sound field information.

In some optional implementations of some embodiments, the executing body may generate the first sound field feature basis function sequence based on the listening position information, and may include the steps of:

first, determining a first playing sound field characteristic function corresponding to a first position coordinate parameter included in the listening position information. The first position coordinate parameter included in the listening position information may be input to a preset first feature function, to obtain the first playing sound field feature function.

As an example, the above-mentioned preset first characteristic function may be a first type of hanker function. The first type of hanker function is applied as a first feature function, and the input argument may be the product of the first position coordinate parameter (radius coordinate r) and the angular wave number ω.

And a second step of determining a second playing sound field characteristic function sequence corresponding to the second position coordinate parameter and the third position coordinate parameter included in the listening position information. The second position coordinate parameter and the third position coordinate parameter included in the listening position information may be input to each second feature function in a preset second feature function sequence to generate a second playing sound field feature function, so as to obtain a second playing sound field feature function sequence. Here, each of the above-described preset second feature function sequences may be a spherical harmonic basis function of a preset order value, a preset order value.

As an example, the above-mentioned preset order value may be any natural number p. The predetermined number of times may be any natural number between (-p, p). Each of the above-described predetermined sequences of second feature functions may be, but is not limited to, at least one of: a spherical harmonic basis function of 0 th order, a spherical harmonic basis function of 1 st order to 1 st order, a spherical harmonic basis function of 1 st order to 0 th order, or a spherical harmonic basis function of 1 st order to 1 st order.

And thirdly, determining the product of each second playing sound field characteristic function in the first playing sound field characteristic function and the second playing sound field characteristic function sequence as a first sound field characteristic basis function to obtain the first sound field characteristic basis function sequence. Wherein, the product of the function value of the first type Hank function and the function value of the spherical harmonic basis function of each preset order value and preset order value can be determined to obtain 2 degrees of freedom of 2p+1 ^p An infinite series of polar basis functions is formed and used as the first sound field characteristic basis function series.

Optionally, the preset panoramic play feature information may be generated by the following steps:

the first step, a recorded sound signal set collected by each microphone terminal and a recorded position information set of each microphone terminal are obtained. Wherein, the recording sound signals in the recording sound signal set and the recording position information in the recording position information set are in one-to-one correspondence. The microphone terminal may be a terminal provided on the head of the dummy for collecting and recording the sound signal. Each recorded sound signal in the set of recorded sound signals may be: pre-recorded audio for determining panoramic play characteristic information, a sound signal at a microphone terminal corresponding to the recorded sound signal. Each recording position information in the recording position information set may represent coordinates of the microphone terminal corresponding to the recording position information in a spherical coordinate system. Each recording location information in the recording location information set may include, but is not limited to, at least one of: a fourth position coordinate parameter, a fifth position coordinate parameter, and a sixth position coordinate parameter. The fourth position coordinate parameter may be a radius coordinate of the microphone terminal corresponding to the recording position information in a spherical coordinate system. The fifth position coordinate parameter may be a polar angle coordinate of the microphone terminal corresponding to the recording position information in a spherical coordinate system. The sixth position coordinate parameter may be an azimuth coordinate of the microphone terminal corresponding to the recording position information in a spherical coordinate system.

As an example, the above-mentioned microphone terminal may be a microphone.

And secondly, establishing a sound field recording equation corresponding to the recorded sound signal set and the recording position information set.

And thirdly, solving the sound field recording equation to obtain a panoramic play feature vector. The sound field recording equation can be solved by a linear algebra method to obtain the panoramic play feature vector.

And fourthly, determining the panoramic play characteristic vector as the panoramic play characteristic information.

In some optional implementations of some embodiments, the executing entity establishes a sound field recording equation corresponding to the recording sound signal set and the recording position information set, and may include the steps of:

and a first step of performing splicing processing on each recorded sound signal in the recorded sound signal set to obtain a recorded sound signal vector. Wherein each recorded sound signal in the set of recorded sound signals may be determined as an element in the recorded sound signal vector to obtain the recorded sound signal vector.

And secondly, determining the sound field information of the recording corresponding to the recording position information set. Wherein, the recording sound field information can include, but is not limited to, at least one of the following: a sound field characteristic transfer function matrix.

And thirdly, establishing the sound field recording equation based on the recorded sound signal vector and the recorded sound field information. The sound field recording equation may be:

wherein,representing the recorded sound signal vector. And B represents a sound field characteristic transfer function matrix included in the sound field information of the recording. />Representing the panoramic play feature vector.

In some optional implementations of some embodiments, the executing body determining recording sound field information corresponding to the recording position information set may include the following steps:

and determining a second sound field characteristic transfer function corresponding to each recording position information in the recording position information set to obtain a second sound field characteristic transfer function set.

And secondly, arranging the second sound field characteristic transfer functions in the second sound field characteristic transfer function set to obtain a sound field characteristic transfer function matrix. Wherein the row vectors of the sound field characteristic transfer function matrix are in one-to-one correspondence with the microphone terminals in the respective microphone terminals. Each second sound field characteristic transfer function in the second sound field characteristic transfer function set may be determined as a row vector in the sound field characteristic transfer function matrix, to obtain the sound field characteristic transfer function matrix.

And thirdly, determining the sound field characteristic transfer function matrix as the sound field information of the recording.

In some optional implementations of some embodiments, the executing body determining a second sound field feature transfer function corresponding to each recording position information in the recording position information set may include the following steps:

and determining a third playing sound field characteristic function corresponding to a fourth position coordinate parameter included in the recording position information. The fourth position coordinate parameter included in the recording position information may be input to the preset first feature function, to obtain the third playing sound field feature function.

And a second step of determining a fourth playing sound field characteristic function sequence corresponding to the fifth position coordinate parameter and the sixth position coordinate parameter included in the recording position information. And inputting a fifth position coordinate parameter and a sixth position coordinate parameter included in the recording position information into each second characteristic function in the preset second characteristic function sequence to generate a fourth playing sound field characteristic function, so as to obtain a fourth playing sound field characteristic function sequence.

And thirdly, determining the product of the third playing sound field characteristic function and each fourth playing sound field characteristic function in the fourth playing sound field characteristic function sequence as a second sound field characteristic basis function to obtain a second sound field characteristic basis function sequence.

And fourthly, splicing the second sound field characteristic basic functions of the first second target number in the second sound field characteristic basic function sequence to obtain the second sound field characteristic transfer function. The second target number is equal to the dimension of the panoramic play feature vector. And intercepting the first second target number of second sound field characteristic basic functions in the second sound field characteristic basic function sequence, and performing the nested linear combination processing according to the sequence to obtain the second sound field characteristic transfer function.

And 103, based on the listening sound field information, performing playback processing on the preset panoramic play characteristic information to obtain a panoramic play sound signal.

In some embodiments, the executing body may perform playback processing on the preset panoramic playing feature information based on the listening sound field information to obtain a panoramic playing sound signal. The product of the first sound field feature transfer function included in the listening sound field information and the panoramic play feature vector included in the preset panoramic play feature information may be determined as the panoramic play sound signal. The playback process described above can be expressed as follows:

wherein,representing a first sound field characteristic transfer function included in the above-mentioned listening sound field information in the form of a vector. />Representing the panoramic play feature vector. And o represents the panoramic broadcasting sound signal obtained by inversion.

The relevant content of steps 102-103 is taken as an invention point of the embodiment of the disclosure, and the technical problem of "accuracy reduction of panoramic sound playing" is solved. Among them, factors that cause the degradation of accuracy of panoramic sound playback tend to be as follows: in the panoramic audio recording stage, only sound signals at fixed positions (e.g., in the horizontal direction) are recorded, and the obtained virtual sound field information can only represent sound field information at a limited number of fixed positions, i.e., the spatial resolution of the recorded panoramic sound is low. In the panoramic audio playing stage, if the position of the user terminal is not recorded by panoramic sound, only a recorded sound signal corresponding to a certain fixed position can be searched for and played. If the above factors are solved, the effect of improving the accuracy of panoramic sound playing can be achieved. To achieve this effect, the present disclosure may place a microphone terminal at an arbitrary position on an artificial head during a panoramic audio recording stage to collect sound signals of the panoramic audio reaching various positions. The signal transfer function of the sound signal within the virtual sound field may then be determined based on the head-related propagation characteristics of the sound signal. Finally, global characteristic information which can characterize the sound propagation characteristics in the virtual sound field can be determined according to the respective sound signals and the signal transfer function corresponding to each sound signal. Therefore, the sound signal of any position where the user terminal possibly appears (even if the position is not subjected to panoramic sound recording) can be inverted in the panoramic sound audio playing stage based on the universe of the panoramic play characteristic information in the virtual sound field, and further, the accuracy of panoramic sound playing can be improved.

And 104, driving a speaker terminal to play audio so that the user terminal can hear the panoramic play sound signal.

In some embodiments, the executing body may drive the speaker terminal to play audio, so that the user terminal receives the panoramic playback sound signal. The speaker terminal may be a terminal for playing audio. The audio may form the same sound signal at the user terminal as the panoramic play sound signal.

The above embodiments of the present disclosure have the following advantageous effects: by the panoramic sound playing method of some embodiments of the present disclosure, accuracy of panoramic sound playing may be improved. Specifically, the degradation of the accuracy of panoramic sound playback is caused by: when playing, panoramic audio can only be played according to the historical virtual sound field at a limited plurality of fixed positions, for example, when the conversion range of the visual field of a user is large, the virtual sound field is not changed, so that the matching effect with the picture of the panoramic video is reduced. Based on this, the panoramic sound playing method of some embodiments of the present disclosure first obtains the listening position information of the user terminal. Next, listening sound field information corresponding to the above-mentioned listening position information is determined. Thus, the position of the user and the sound field information of the position of the user can be confirmed in real time. And then, based on the listening sound field information, carrying out playback processing on the preset panoramic play characteristic information to obtain a panoramic play sound signal. Thus, the sound signal of the panoramic audio to be played at the user terminal can be determined according to the predetermined sound field characteristics. And finally, driving a loudspeaker terminal to play audio so that the user terminal can hear the panoramic play sound signal. Thus, panoramic audio can be played. Therefore, according to some panoramic sound playing methods of the present disclosure, the position information of the user terminal may be acquired in real time, and then, according to the predetermined characteristic information of the virtual sound field, the sound propagation signal of the panoramic sound frequency at the user terminal may be determined, so as to play the audio. Therefore, the sound field information of the position of the user terminal can be determined in real time, and the synchronization rate of the panoramic audio and the panoramic video received by the user terminal and the accuracy of panoramic sound playing can be improved.

With further reference to fig. 2, a schematic structural diagram of a head mounted display device 200 suitable for use in implementing some embodiments of the present disclosure is shown. As shown in fig. 2, the head mounted display device 200 of some embodiments includes: a pose sensor 201. The pose sensor 201 is used for collecting information of a listening position of a user terminal. A central processing unit 202. The central processing unit 202 is communicatively connected to the pose sensor 201, and the central processing unit 202 is configured to determine virtual sound field information corresponding to the listening position information, and perform playback processing on preset panoramic play feature information based on the virtual sound field information to determine a panoramic play sound signal. A display 203. The display 203 is communicatively connected to the cpu 202, and the display 203 is configured to display virtual visual information corresponding to the virtual sound field information. A speaker 204. The speaker 204 is communicatively connected to the cpu 202, and the speaker 204 is configured to play audio so that the user terminal can hear the panoramic playback sound signal.

With further reference to fig. 3, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a panoramic sound playback apparatus, which apparatus embodiments correspond to those shown in fig. 1, and which panoramic sound playback apparatus is particularly applicable in a variety of electronic devices.

As shown in fig. 3, the panoramic sound playback device 300 of some embodiments includes: an acquisition unit 301, a determination unit 302, a playback unit 303, and a drive unit 304. Wherein the acquisition unit 301 is configured to acquire listening position information of the user terminal; a determining unit 302 configured to determine listening sound field information corresponding to the above-described listening position information; a playback unit 303 configured to perform playback processing on preset panoramic play feature information based on the listening sound field information described above, to obtain a panoramic play sound signal; and a driving unit 304 configured to drive the speaker terminal to perform audio playing, so that the user terminal receives the panoramic playback sound signal.

It will be appreciated that the elements described in this panoramic sound playback apparatus 300 correspond to the various steps in the panoramic sound playback method described with reference to fig. 1. Thus, the operations, features, and beneficial effects described above with respect to the panoramic sound playback method are equally applicable to the panoramic sound playback 300 and the units contained therein, and are not described herein.

Referring now to fig. 4, a schematic diagram of an electronic device 400 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), car terminals (e.g., car navigation terminals), and the like, as well as stationary terminals such as digital TVs, desktop computers, and the like. The terminal device shown in fig. 4 is only one example and should not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure.

As shown in fig. 4, the electronic device 400 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 401, which may perform various suitable actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage means 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

In general, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 408 including, for example, magnetic tape, hard disk, etc.; and a communication device 409. The communication means 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. While fig. 4 shows an electronic device 400 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 4 may represent one device or a plurality of devices as needed.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 409, or from storage 408, or from ROM 402. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 401.

It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring listening position information of a user terminal; determining listening sound field information corresponding to the listening position information; based on the listening sound field information, replaying the preset panoramic play characteristic information to obtain a panoramic play sound signal; and driving the speaker terminal to play the audio so that the user terminal can hear the panoramic play sound signal.

Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes an acquisition unit, a determination unit, a playback unit, and a drive unit. The names of these units do not constitute a limitation on the unit itself in some cases, and the acquisition unit may also be described as "a unit that acquires the listening position information of the user terminal", for example.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.

Claims (9)

1. A panoramic sound playing method, comprising:

acquiring listening position information of a user terminal;

determining listening sound field information corresponding to the listening position information;

based on the listening sound field information, replaying the preset panoramic play characteristic information to obtain a panoramic play sound signal;

and driving a loudspeaker terminal to play audio so that the user terminal can hear the panoramic play sound signal.

2. The method of claim 1, wherein the preset panorama playing feature information comprises: panoramic play feature vectors; and

the determining listening sound field information corresponding to the listening position information includes:

generating a first sound field feature basis function sequence based on the listening position information;

performing splicing processing on a first sound field feature basis function of a first target number in the first sound field feature basis function sequence to obtain a first sound field feature transfer function, wherein the first target number is equal to the dimension of a panoramic play feature vector included in the preset panoramic play feature information;

the first sound field feature transfer function is determined as the listening sound field information.

3. The method of claim 2, wherein the listening position information comprises: a first position coordinate parameter, a second position coordinate parameter, and a third position coordinate parameter; and

the generating a first sound field feature basis function sequence based on the listening position information includes:

determining a first playing sound field characteristic function corresponding to a first position coordinate parameter included in the listening position information;

determining a second playing sound field feature function sequence corresponding to a second position coordinate parameter and a third position coordinate parameter included in the listening position information;

and determining the product of each second playing sound field characteristic function in the first playing sound field characteristic function and the second playing sound field characteristic function sequence as a first sound field characteristic basis function to obtain the first sound field characteristic basis function sequence.

4. The method of claim 1, wherein the preset panoramic play characteristic information is generated by:

acquiring a recorded sound signal set acquired by each microphone terminal and a recording position information set of each microphone terminal, wherein the recorded sound signals in the recorded sound signal set and the recording position information in the recording position information set are in one-to-one correspondence;

establishing a sound field recording equation corresponding to the recorded sound signal set and the recording position information set;

solving the sound field recording equation to obtain a panoramic play feature vector;

and determining the panoramic play feature vector as the panoramic play feature information.

5. The method of claim 4, wherein the establishing sound field recording equations corresponding to the set of recorded sound signals and the set of recording location information comprises:

performing splicing processing on each recorded sound signal in the recorded sound signal set to obtain a recorded sound signal vector;

determining recording sound field information corresponding to the recording position information set;

and establishing the sound field recording equation based on the recorded sound signal vector and the recorded sound field information.

6. A head mounted display device for implementing the panoramic sound playback method of any one of claims 1-3, comprising:

the position and orientation sensor is used for collecting the information of the listening position of the user terminal;

the central processing unit is in communication connection with the pose sensor and is used for determining virtual sound field information corresponding to the listening position information, playing back preset panoramic play characteristic information based on the virtual sound field information and determining panoramic play sound signals;

the display is in communication connection with the central processing unit and is used for displaying virtual visual information corresponding to the virtual sound field information;

and the loudspeaker is in communication connection with the central processing unit and is used for playing audio so that the user terminal can hear the panoramic playing sound signal.

7. A panoramic sound playback device, comprising:

an acquisition unit configured to acquire listening position information of a user terminal;

a determination unit configured to determine listening sound field information corresponding to the listening position information;

a playback unit configured to perform playback processing on preset panoramic play feature information based on the listening sound field information to obtain a panoramic play sound signal;

and the driving unit is configured to drive the loudspeaker terminal to play audio so that the user terminal can hear the panoramic playing sound signal.

8. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon;

when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.

9. A computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-5.