CN113115175B

CN113115175B - 3D sound effect processing method and related product

Info

Publication number: CN113115175B
Application number: CN202110395644.5A
Authority: CN
Inventors: 严锋贵
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2018-09-25
Filing date: 2018-09-25
Publication date: 2022-05-10
Anticipated expiration: 2038-09-25
Also published as: CN113115175A; CN109327766A; WO2020063027A1; CN109327766B

Abstract

The embodiment of the application discloses a 3D sound effect processing method and a related product, wherein the method comprises the following steps: acquiring single track data of a sound source; determining a target content scene type corresponding to the single sound channel data; determining a target reverberation effect parameter according to the target content scene type; and processing the single sound channel data according to the target reverberation effect parameter to obtain target reverberation double sound channel data. By adopting the method and the device, the reverberation effect parameter corresponding to the content scene can be determined, and the reverberation binaural data can be generated according to the reverberation effect parameter, so that the reverberation effect suitable for the content scene is realized, and the stereoscopic impression is more real.

Description

3D sound effect processing method and related product

Technical Field

The application relates to the technical field of virtual/augmented reality, in particular to a 3D sound effect processing method and a related product.

Background

With the widespread use of electronic devices (such as mobile phones, tablet computers, and the like), the electronic devices have more and more applications and more powerful functions, and the electronic devices are developed towards diversification and personalization, and become indispensable electronic products in the life of users.

With the development of the technology, the virtual reality is also developed rapidly in the electronic device, however, in the virtual reality product, the audio data received by the earphone in the prior art is often 2D audio data, so that the sound reality sense cannot be brought to the user, and the user experience is reduced.

Disclosure of Invention

The embodiment of the application provides a 3D sound effect processing method and a related product, which can synthesize a 3D sound effect and improve user experience.

In a first aspect, an embodiment of the present application provides a 3D sound effect processing method, including:

acquiring single sound channel data of a sound source;

determining a target content scene type corresponding to the single sound channel data;

determining a target reverberation effect parameter according to the target content scene type;

and processing the single sound channel data according to the target reverberation effect parameter to obtain target reverberation double sound channel data.

In a second aspect, an embodiment of the present application provides a 3D sound effect processing apparatus, where the 3D sound effect processing apparatus includes: an acquisition unit, a first determination unit, a second determination unit and a processing unit, wherein,

the acquisition unit is used for acquiring single sound channel data of a sound source;

the first determining unit is configured to determine a target content scene type corresponding to the monaural data;

the second determining unit is used for determining a target reverberation effect parameter according to the target content scene type;

and the processing unit is used for processing the single-channel data according to the target reverberation effect parameter to obtain target reverberation double-channel data.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program enables a computer to perform some or all of the steps described in the first aspect of the embodiment of the present application.

In a fifth aspect, embodiments of the present application provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, where the computer program is operable to cause a computer to perform some or all of the steps as described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

It can be seen that, according to the 3D sound effect processing method and the related product described in the embodiments of the present application, the monaural data of the sound source is obtained, the target content scene type corresponding to the monaural data is determined, the target reverberation effect parameter is determined according to the target content scene type, and the monaural data is processed according to the target reverberation effect parameter to obtain the target reverberation binaural data.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 1B is a schematic flow chart illustrating a 3D sound effect processing method according to an embodiment of the present disclosure;

fig. 1C is a schematic diagram illustrating a multi-channel binaural data partitioning manner disclosed in the embodiment of the present application;

FIG. 1D is a schematic diagram illustrating a 3D sound processing method according to an embodiment of the present disclosure;

FIG. 1E is a schematic diagram illustrating another 3D sound effect processing method disclosed in an embodiment of the present application;

FIG. 2 is a schematic flow chart illustrating another 3D sound effect processing method disclosed in the present application;

FIG. 3 is a schematic flow chart illustrating another 3D sound effect processing method disclosed in the present application;

fig. 4 is a schematic structural diagram of another electronic device disclosed in the embodiments of the present application;

fig. 5 is a schematic structural diagram of a 3D sound effect processing device according to an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The electronic device according to the embodiment of the present application may include various handheld devices (e.g., smart phones), vehicle-mounted devices, Virtual Reality (VR)/Augmented Reality (AR) devices, wearable devices, computing devices or other processing devices connected to wireless modems, and various forms of User Equipment (UE), Mobile Stations (MSs), terminal devices (terminal devices), development/test platforms, servers, and so on, which have wireless communication functions. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

In specific implementation, in this embodiment, the electronic device may filter audio data (sound generated by a sound source) by using an HRTF (Head Related Transfer Function) filter to obtain virtual surround sound, which is also called surround sound or panoramic sound, so as to implement a three-dimensional stereo sound effect. The name of the HRTF in the time domain is hrir (head Related Impulse response). Or convolve the audio data with a Binaural Room Impulse Response (BRIR), which consists of three parts: direct sound, early reflected sound and reverberation (reverb).

Referring to fig. 1A, fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device includes a control circuit and an input-output circuit, and the input-output circuit is connected to the control circuit.

The control circuitry may include, among other things, storage and processing circuitry. The storage circuit in the storage and processing circuit may be a memory, such as a hard disk drive memory, a non-volatile memory (e.g., a flash memory or other electronically programmable read only memory used to form a solid state drive, etc.), a volatile memory (e.g., a static or dynamic random access memory, etc.), etc., and the embodiments of the present application are not limited thereto. Processing circuitry in the storage and processing circuitry may be used to control the operation of the electronic device. The processing circuitry may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application specific integrated circuits, display driver integrated circuits, and the like.

The storage and processing circuitry may be used to run software in the electronic device, such as play incoming call alert ringing application, play short message alert ringing application, play alarm alert ringing application, play media file application, Voice Over Internet Protocol (VOIP) phone call application, operating system functions, and so forth. The software may be used to perform some control operations, such as playing an incoming alert ring, playing a short message alert ring, playing an alarm alert ring, playing a media file, making a voice phone call, and performing other functions in the electronic device, and the embodiments of the present application are not limited.

The input-output circuit can be used for enabling the electronic device to input and output data, namely allowing the electronic device to receive data from the external device and allowing the electronic device to output data from the electronic device to the external device.

The input-output circuit may further include a sensor. The sensors may include ambient light sensors, optical and capacitive based infrared proximity sensors, ultrasonic sensors, touch sensors (e.g., optical based touch sensors and/or capacitive touch sensors, where the touch sensors may be part of a touch display screen or may be used independently as a touch sensor structure), acceleration sensors, gravity sensors, and other sensors, etc. The input-output circuit may further include audio components that may be used to provide audio input and output functionality for the electronic device. The audio components may also include a tone generator and other components for generating and detecting sound.

The input-output circuitry may also include one or more display screens. The display screen can comprise one or a combination of a liquid crystal display screen, an organic light emitting diode display screen, an electronic ink display screen, a plasma display screen and a display screen using other display technologies. The display screen may include an array of touch sensors (i.e., the display screen may be a touch display screen). The touch sensor may be a capacitive touch sensor formed by a transparent touch sensor electrode (e.g., an Indium Tin Oxide (ITO) electrode) array, or may be a touch sensor formed using other touch technologies, such as acoustic wave touch, pressure sensitive touch, resistive touch, optical touch, and the like, and the embodiments of the present application are not limited thereto.

The input-output circuitry may further include communications circuitry that may be used to provide the electronic device with the ability to communicate with external devices. The communication circuitry may include analog and digital input-output interface circuitry, and wireless communication circuitry based on radio frequency signals and/or optical signals. The wireless communication circuitry in the communication circuitry may include radio frequency transceiver circuitry, power amplifier circuitry, low noise amplifiers, switches, filters, and antennas. For example, the wireless communication circuitry in the communication circuitry may include circuitry to support Near Field Communication (NFC) by transmitting and receiving near field coupled electromagnetic signals. For example, the communication circuit may include a near field communication antenna and a near field communication transceiver. The communications circuitry may also include cellular telephone transceiver and antennas, wireless local area network transceiver circuitry and antennas, and so forth.

The input-output circuit may further include other input-output units. Input-output units may include buttons, joysticks, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light emitting diodes and other status indicators, and the like.

The electronic device may further include a battery (not shown) for supplying power to the electronic device.

The following describes embodiments of the present application in detail.

Referring to fig. 1B, fig. 1B is a schematic flow chart of a 3D sound effect processing method disclosed in the present embodiment, which is applied to the electronic device described in fig. 1A, wherein the 3D sound effect processing method includes the following

steps

101 and 103.

101. Monophonic data of a sound source is acquired.

The embodiment of the application can be applied to virtual reality/augmented reality scenes or 3D recording scenes. In the embodiment of the present application, the sound source may be a sounding body in a virtual scene, for example, an airplane in a game scene, and the sound source may be a fixed sound source, or a mobile sound source, or the sound source may also be a sounding body in a physical environment.

102. And determining the target content scene type corresponding to the single sound channel data.

In this embodiment of the present application, the content scene type may be at least one of the following: movies, life, entertainment, military, life, astronomy, geography, etc., without limitation. For example, each piece of monaural data may correspond to a frequency band, and different frequency bands may correspond to different content scene types, and specifically, a mapping relationship between a frequency band and a content scene type is pre-stored in the electronic device, and then a target content scene type corresponding to the frequency band of the monaural data is determined according to the mapping relationship.

Optionally, in the step 102, determining the type of the target content scene corresponding to the mono data may include the following steps:

21. performing semantic analysis on the single sound channel data to obtain a plurality of keywords;

22. determining a content scene type corresponding to each keyword in the plurality of keywords according to a mapping relation between preset keywords and the content scene type to obtain a plurality of content scene types;

23. and taking the content scene type with the largest occurrence number in the plurality of content scene types as the target content scene type.

The electronic equipment can perform semantic analysis on the monophonic data to obtain a plurality of keywords, can also pre-store a mapping relation between preset keywords and content scene types, further obtains a plurality of content scene types according to the content scene type corresponding to each keyword in the plurality of keywords, and takes the content scene type with the largest occurrence frequency in the plurality of content scene types as a target content scene type.

103. And determining a target reverberation effect parameter according to the target content scene type.

Wherein the reverberation effect parameter may include at least one of: input level, low frequency cut point, high frequency cut point, early reflection time, diffusion degree, low mixing ratio, reverberation time, high frequency attenuation point, frequency dividing point, original dry sound volume, early reflection sound volume, reverberation volume, sound field width, output sound field, tail sound, etc., without limitation. In the concrete realization, different content scenes can correspond to different reverberation effect parameters, so that under different scenes, the reverberation effect is different, the reverberation effect suitable for the scenes can be realized, and the 3D feeling is more real.

Optionally, in step 103, determining a target reverberation effect parameter according to the target content scene type, which may be implemented as follows:

and determining the target reverberation effect parameter corresponding to the target content scene type according to a preset mapping relation between the content scene type and the reverberation effect parameter.

The electronic device may pre-store a mapping relationship between a preset content scene type and the reverberation effect parameter, and then determine a target reverberation effect parameter corresponding to the target content scene type according to the mapping relationship.

104. And processing the single sound channel data according to the target reverberation effect parameter to obtain target reverberation double sound channel data.

The electronic device can process the single-channel data based on the HRTF algorithm to obtain the double-channel data, and can process the double-channel data through the target reverberation effect parameter to obtain the reverberation double-channel data.

Optionally, in the step 104, processing the mono channel data according to the target reverberation effect parameter to obtain the target reverberation binaural data may include the following steps:

41. acquiring a first three-dimensional coordinate of the sound source;

42. acquiring a second three-dimensional coordinate of the target object, wherein the first three-dimensional coordinate and the second three-dimensional coordinate are based on the same coordinate origin;

43. and generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the single sound channel data and the target reverberation effect parameter.

Taking a virtual scene as an example, each object in the virtual scene may correspond to one three-dimensional coordinate, so that the first three-dimensional coordinate of the sound source may be obtained, and when the sound source makes a sound, the monaural data generated by the sound source may be obtained. The target object may also correspond to a three-dimensional coordinate, that is, a second three-dimensional coordinate, where the first three-dimensional coordinate and the second three-dimensional coordinate are different positions and are based on the same coordinate origin. And further, generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the monophonic data and the target reverberation effect parameter, and specifically, realizing by using an HRTF algorithm.

Optionally, when the target object is in a game scene, the step 42 of acquiring the second three-dimensional coordinates of the target object may include the following steps:

421. acquiring a map corresponding to the game scene;

422. and determining the coordinate position corresponding to the target object in the map to obtain the second three-dimensional coordinate.

The target object can be regarded as a role in the game when the target object is in the game scene, and certainly, in specific implementation, the game scene can correspond to a three-dimensional map, so that the electronic device can obtain the map corresponding to the game scene, determine the coordinate position corresponding to the target object in the map, and obtain the second three-dimensional coordinate.

Optionally, in step 43, generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the mono data, and the target reverberation effect parameter may include the following steps:

431. generating multi-channel two-channel data between the first three-dimensional coordinate and the second three-dimensional coordinate by using the single-channel data, wherein each channel of two-channel data corresponds to a unique propagation direction;

432. and synthesizing the multi-channel binaural data into the target reverberation binaural data according to the target reverberation effect parameters.

The original sound data of the sound source is monaural data, and binaural data can be obtained through algorithm processing (for example, HRTF algorithm), and since sound is propagated along various directions in a real environment, and phenomena such as reflection, refraction, interference, diffraction and the like also occur in the propagation process, in this embodiment of the present application, only multichannel binaural data passing through between the first three-dimensional coordinate and the second three-dimensional coordinate is used for finally synthesizing target binaural data, and the multichannel binaural data is synthesized into target reverberation binaural data according to the target reverberation effect parameters.

Optionally, in the step 432, synthesizing the multi-channel binaural data into the target reverberation binaural data according to the target reverberation effect parameter may include the following steps:

a11, taking the first three-dimensional coordinate and the second three-dimensional coordinate as axes to make a cross section, dividing the multichannel two-channel data to obtain a first two-channel data set and a second two-channel data set, wherein the first two-channel data set and the second two-channel data set both comprise at least one channel of two-channel data;

a12, synthesizing the first double-channel data set to obtain first single-channel data;

a13, synthesizing the second double-channel data set to obtain second single-channel data;

and A14, synthesizing the first single-channel data and the second single-channel data according to the target reverberation effect parameter to obtain the reverberation binaural data.

Wherein, after knowing the first three-dimensional coordinate and the second three-dimensional coordinate, the first three-dimensional coordinate and the second three-dimensional coordinate can be taken as axes to make a cross section, and since the sound propagation direction is fixed, the propagation track can also have a certain symmetry along a certain symmetry axis, as shown in fig. 1C, the first three-dimensional coordinate and the second three-dimensional coordinate form an axis, and the axis is taken as a cross section, so that the multi-channel binaural data can be divided to obtain a first binaural data set and a second binaural data set, and without considering external factors such as refraction, reflection, diffraction and the like, the first binaural data set and the second binaural data set can also be binaural data containing the same number of channels, and the binaural data of different sets are also in a symmetric relationship, and the first binaural data set and the second binaural data set both include at least one channel of binaural data, in a specific implementation, the electronic device may synthesize a first set of binaural data to obtain first mono data, the electronic device may include left and right earphones, the first mono data may be played mainly by the left earphone, and accordingly, the electronic device may synthesize a second set of binaural data to obtain second mono data, the second mono data may be played mainly by the right earphone, and finally, synthesize the first mono data and the second mono data according to a target reverberation effect parameter to obtain target reverberation channel data, specifically, as shown in fig. 1D, the electronic device may perform channel processing on the mono data (for example, the first mono data and the second mono data) (channel processing mainly refers to converting the mono data into binaural data) to obtain 2 binaural data, and process each path of mono data according to the target reverberation effect parameter, and obtaining reverberation audio data, and synthesizing the 2 pieces of binaural data and the reverberation audio data to obtain target reverberation audio data.

Optionally, in the step 4322, the synthesizing the first binaural data set to obtain the first binaural data may include the following steps:

b1, obtaining a plurality of energy values according to the energy value of each path of double-channel data in the first double-channel data set;

b2, selecting an energy value larger than a first energy threshold value from the plurality of energy values to obtain a plurality of first target energy values;

b3, determining first multichannel data corresponding to the plurality of first target energy values, and synthesizing the first multichannel data to obtain the first multichannel data.

The first energy threshold value can be set by the user or defaulted by the system. In specific implementation, the electronic device may obtain a plurality of energy values from an energy value of each path of binaural data in the first binaural data set, further select an energy value greater than the first energy threshold from the plurality of energy values to obtain a plurality of first target energy values, determine first binaural data corresponding to the plurality of first target energy values, and synthesize the first binaural data to obtain first monophonic data.

Optionally, based on the step a 1-step A3, the step 4323 may also be implemented, which is not described herein again.

Optionally, between the above steps 41 to 43, the following steps may be further included:

acquiring the face orientation of the target object;

then, in step 43, generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the mono data, and the target reverberation effect parameter may be implemented as follows:

and generating target reverberation binaural data according to the face orientation, the first three-dimensional coordinate, the second three-dimensional coordinate, the mono data and the target reverberation effect parameter.

In this embodiment, the electronic device may detect the face orientation of the target object by considering the face orientation of the target object, specifically, may detect the orientation of the target object relative to the sound source as the face orientation of the target object if the game scene is the case, and may consider a user head-mounted device, for example, head-mounted virtual reality glasses, a virtual reality helmet, a virtual reality headband display device, and the like if the electronic device is the game scene. The detection of the human head direction can use various sensors, including but not limited to resistive sensors, mechanical sensors, photosensitive sensors, ultrasonic sensors, muscle sensors, etc., and is not limited herein. The sensor can be one kind of sensor, or a combination of several kinds of sensors, or one sensor or a combination of several sensors. The detection of the human head direction can be performed at preset time intervals, and the preset time intervals can be set by a user or default by a system.

a21, determining the energy value of each path of double-channel data in the multi-path double-channel data to obtain a plurality of energy values;

a22, determining a reverberation effect parameter adjusting coefficient corresponding to each energy value in the multiple energy values according to a mapping relation between preset energy values and reverberation effect parameter adjusting coefficients to obtain multiple reverberation effect parameter adjusting coefficients;

a23, determining a reverberation effect parameter corresponding to each path of binaural data according to the reverberation effect parameter adjusting coefficients and the target reverberation effect parameter to obtain a plurality of first reverberation effect parameters;

a24, processing the multichannel two-channel data according to the multiple first reverberation effect parameters to obtain multichannel reverberation two-channel data, wherein each first reverberation effect parameter corresponds to a unique channel two-channel data;

and A25, synthesizing the multi-channel binaural data to obtain the target reverberation binaural data.

The electronic device can pre-store a mapping relation between a preset energy value and a reverberation effect parameter adjusting coefficient, the reverberation effect parameter adjusting coefficient is used for a reverberation effect parameter, the value range of the reverberation effect parameter adjusting coefficient is 0-1, specifically, the reverberation effect parameter is the actual reverberation effect parameter, and corresponding dual-channel data is processed through the actual reverberation effect parameter to obtain the reverberation dual-channel data. In the specific implementation, the electronic device may determine an energy value of each of the multichannel binaural data to obtain a plurality of energy values, determine a reverberation effect parameter adjustment coefficient corresponding to each of the plurality of energy values according to the mapping relationship to obtain a plurality of reverberation effect parameter adjustment coefficients, determine a reverberation effect parameter corresponding to each of the multichannel binaural data according to the plurality of reverberation effect parameter adjustment coefficients and the target reverberation effect parameter to obtain a plurality of first reverberation effect parameters, i.e., a first reverberation effect parameter is a target reverberation effect parameter and a reverberation effect parameter adjustment coefficient, further process the multichannel binaural data according to the plurality of first reverberation effect parameters to obtain multichannel reverberation binaural data, where each first reverberation effect parameter corresponds to a unique one of the multichannel binaural data, synthesize the multichannel binaural data to obtain target reverberation binaural data, therefore, different reverberation effects in different directions are realized according to each path of energy value, finally synthesized reverberation binaural data is more real in third dimension. Specifically, as shown in fig. 1E, the electronic device may perform channel processing on the mono channel data (e.g., the first mono channel data and the second mono channel data) (the channel processing mainly refers to converting the mono channel data into the binaural data) to obtain a plurality of binaural data, process each channel of channel data according to the reverberation effect parameter to obtain a plurality of reverberation audio data, and synthesize the plurality of binaural data and the plurality of reverberation audio data to obtain the target reverberation channel data.

It can be seen that, in the 3D sound effect processing method described in the embodiment of the present application, the monaural data of the sound source is obtained, the target content scene type corresponding to the monaural data is determined, the target reverberation effect parameter is determined according to the target content scene type, and the monaural data is processed according to the target reverberation effect parameter to obtain the target reverberation binaural data.

In accordance with the above, fig. 2 is a schematic flow chart of a 3D sound effect processing method disclosed in the embodiment of the present application. Applied to the electronic device shown in FIG. 1A, the 3D sound effect processing method includes the following

steps

201 and 206.

201. Monophonic data of a sound source is acquired.

202. And determining the target content scene type corresponding to the single sound channel data.

203. And determining a target reverberation effect parameter according to the target content scene type.

204. Acquiring a first three-dimensional coordinate of the sound source.

205. And acquiring a second three-dimensional coordinate of the target object, wherein the first three-dimensional coordinate and the second three-dimensional coordinate are based on the same coordinate origin.

206. And generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the single sound channel data and the target reverberation effect parameter.

The detailed descriptions of the steps 201 to 206 may refer to the corresponding descriptions of the 3D sound effect processing method described in fig. 1B, and are not repeated herein.

It can be seen that, in the 3D sound effect processing method described in the embodiment of the present application, the monaural data of the sound source is obtained, the target content scene type corresponding to the monaural data is determined, the target reverberation effect parameter is determined according to the target content scene type, the first three-dimensional coordinate of the sound source is obtained, the second three-dimensional coordinate of the target object is obtained, the first three-dimensional coordinate and the second three-dimensional coordinate are based on the same coordinate origin, and the target reverberation two-channel data is generated according to the first three-dimensional coordinate, the second three-dimensional coordinate, the monaural data, and the target reverberation effect parameter.

In accordance with the above, fig. 3 is a schematic flow chart of a 3D sound effect processing method disclosed in the embodiment of the present application. Applied to the electronic device shown in FIG. 1A, the 3D sound effect processing method includes the following steps 301-306.

301. Monophonic data of a sound source is acquired.

302. And performing semantic analysis on the single-channel data to obtain a plurality of keywords.

303. And determining the content scene type corresponding to each keyword in the plurality of keywords according to a mapping relation between preset keywords and the content scene type to obtain a plurality of content scene types.

304. And taking the content scene type with the largest occurrence frequency in the plurality of content scene types as a target content scene type.

305. And determining a target reverberation effect parameter corresponding to the target content scene type according to a preset mapping relation between the content scene type and the reverberation effect parameter.

306. And processing the single sound channel data according to the target reverberation effect parameter to obtain target reverberation double sound channel data.

The detailed descriptions of steps 301 to 306 may refer to the corresponding descriptions of the 3D audio processing method described in fig. 1B, and are not repeated herein.

It can be seen that, in the 3D sound effect processing method described in the embodiment of the present application, the monaural data of the sound source is obtained, the monaural data is subjected to semantic analysis to obtain a plurality of keywords, the content scene type corresponding to each keyword in the plurality of keywords is determined according to the mapping relationship between the preset keywords and the content scene types to obtain a plurality of content scene types, the content scene type with the largest occurrence frequency in the plurality of content scene types is taken as the target content scene type, the target reverberation effect parameter corresponding to the target content scene type is determined according to the mapping relationship between the preset content scene type and the reverberation effect parameter, the monaural data is processed according to the target reverberation effect parameter to obtain the target reverberation binaural data, so that the reverberation effect parameter corresponding to the content scene can be determined, and the reverberation binaural data is generated according to the reverberation effect parameter, therefore, the reverberation effect suitable for the content scene is realized, and the stereoscopic impression is more real.

Referring to fig. 4, fig. 4 is a schematic structural diagram of another electronic device disclosed in the embodiment of the present application, and as shown in the drawing, the electronic device includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for performing the following steps:

acquiring single track data of a sound source;

It can be seen that, in the electronic device described in the embodiment of the present application, the monaural data of the sound source is obtained, the target content scene type corresponding to the monaural data is determined, the target reverberation effect parameter is determined according to the target content scene type, and the monaural data is processed according to the target reverberation effect parameter to obtain the target reverberation binaural data.

In one possible example, in the determining the target content scene type corresponding to the mono data, the program includes instructions for:

performing semantic analysis on the single sound channel data to obtain a plurality of keywords;

determining a content scene type corresponding to each keyword in the plurality of keywords according to a mapping relation between preset keywords and the content scene type to obtain a plurality of content scene types;

and taking the content scene type with the largest occurrence number in the plurality of content scene types as the target content scene type.

In one possible example, in said determining a target reverberation effect parameter according to the target content scene type, the above procedure comprises instructions for performing the following steps:

In one possible example, in the aspect that the mono data is processed according to the target reverberation effect parameter to obtain target binaural data, the program includes instructions for:

acquiring a first three-dimensional coordinate of the sound source;

acquiring a second three-dimensional coordinate of the target object, wherein the first three-dimensional coordinate and the second three-dimensional coordinate are based on the same coordinate origin;

and generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the single sound channel data and the target reverberation effect parameter.

In one possible example, in the generating target reverberant binaural data from the first three-dimensional coordinates, the second three-dimensional coordinates, and the mono data, the target reverberation effect parameters, the program includes instructions for:

generating multi-channel two-channel data between the first three-dimensional coordinate and the second three-dimensional coordinate by using the single-channel data, wherein each channel of two-channel data corresponds to a unique propagation direction;

and synthesizing the multi-channel two-channel data into the target reverberation two-channel data according to the target reverberation effect parameter.

In one possible example, in the synthesizing of the multi-channel binaural data into the target reverberation binaural data according to the target reverberation effect parameter, the program includes instructions for:

taking the first three-dimensional coordinate and the second three-dimensional coordinate as axes to make a cross section, and dividing the multichannel two-channel data to obtain a first two-channel data set and a second two-channel data set, wherein the first two-channel data set and the second two-channel data set both comprise at least one channel of two-channel data;

synthesizing the first double-channel data set to obtain first single-channel data;

synthesizing the second double-channel data set to obtain second single-channel data;

and synthesizing the first single-channel data and the second single-channel data according to the target reverberation effect parameter to obtain the reverberation double-channel data.

In one possible example, in said synthesizing the multi-channel binaural data into the target reverberant binaural data according to the target reverberation effect parameter, the above procedure comprises instructions for:

determining the energy value of each path of double-channel data in the multi-path double-channel data to obtain a plurality of energy values;

determining a reverberation effect parameter adjusting coefficient corresponding to each energy value in the plurality of energy values according to a mapping relation between preset energy values and reverberation effect parameter adjusting coefficients to obtain a plurality of reverberation effect parameter adjusting coefficients;

determining a reverberation effect parameter corresponding to each path of binaural data according to the reverberation effect parameter adjusting coefficients and the target reverberation effect parameter to obtain a plurality of first reverberation effect parameters;

processing the multichannel two-channel data according to the plurality of first reverberation effect parameters to obtain multichannel reverberation two-channel data, wherein each first reverberation effect parameter corresponds to a unique channel of two-channel data;

and synthesizing the multi-channel binaural data to obtain the target reverberation binaural data.

The above description has introduced the solution of the embodiment of the present application mainly from the perspective of the method-side implementation process. It is understood that the electronic device comprises corresponding hardware structures and/or software modules for performing the respective functions in order to realize the above-mentioned functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative elements and algorithm steps described in connection with the embodiments provided herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a 3D sound effect processing device according to an embodiment of the present application, applied to the electronic device shown in fig. 1A, where the 3D sound effect processing device 500 includes: an acquisition unit 501, a first determination unit 502, a second determination unit 503, and a processing unit 504, wherein,

the acquiring unit 501 is configured to acquire monaural data of a sound source;

the first determining unit 502 is configured to determine a target content scene type corresponding to the monaural data;

the second determining unit 503 is configured to determine a target reverberation effect parameter according to the target content scene type;

the processing unit 504 is configured to process the mono channel data according to the target reverberation effect parameter, so as to obtain target reverberation binaural data.

In one possible example, in the aspect of determining the target content scene type corresponding to the monaural data, the first determining unit 502 is specifically configured to:

In one possible example, in the aspect of determining the target reverberation effect parameter according to the target content scene type, the second determining unit 503 is specifically configured to:

In a possible example, in terms of processing the mono data according to the target reverberation effect parameter to obtain target binaural data, the processing unit 504 is specifically configured to:

acquiring a first three-dimensional coordinate of the sound source;

In one possible example, in the aspect of generating the target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the mono data and the target reverberation effect parameter, the processing unit 504 is specifically configured to:

In one possible example, in the aspect of synthesizing the multi-channel binaural data into the target reverberation binaural data according to the target reverberation effect parameter, the processing unit 504 is specifically configured to:

It can be seen that the 3D sound effect processing apparatus described in the embodiment of the present application is applied to an electronic device, obtains monaural data of a sound source, determines a target content scene type corresponding to the monaural data, determines a target reverberation effect parameter according to the target content scene type, and processes the monaural data according to the target reverberation effect parameter to obtain target reverberation binaural data.

It should be noted that the electronic device described in the embodiments of the present application is presented in the form of a functional unit. The term "unit" as used herein is to be understood in its broadest possible sense, and objects used to implement the functions described by the respective "unit" may be, for example, an integrated circuit ASIC, a single circuit, a processor (shared, dedicated, or chipset) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.

The obtaining unit 501, the first determining unit 502, the second determining unit 503, and the processing unit 504 may be a control circuit or a processor.

Embodiments of the present application also provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the 3D sound effect processing methods as described in the above method embodiments.

Embodiments of the present application also provide a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, the computer program being operable to cause a computer to execute some or all of the steps of any of the 3D sound effect processing methods as described in the above method embodiments.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.

The integrated unit, if implemented in the form of a software program module and sold or used as a stand-alone product, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: various media capable of storing program codes, such as a usb disk, a read-only memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and the like.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash disk, ROM, RAM, magnetic or optical disk, and the like.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A3D sound effect processing method is characterized by comprising the following steps:

acquiring single track data of a sound source;

processing the mono channel data according to a target reverberation effect parameter to obtain target reverberation binaural data, wherein the target reverberation effect parameter depends on a target content scene type, and the target content scene type corresponds to the mono channel data;

wherein the content of the first and second substances,

the target reverberation effect parameter comprises at least one of: the method comprises the steps of inputting level, low-frequency cut points, high-frequency cut points, early reflection time, diffusion degree, low mixing ratio, reverberation time, original dry sound volume, early reflection sound volume, high-frequency attenuation points, frequency dividing points, reverberation volume, sound field width, output sound field and tail sound;

determining a target content scene type corresponding to the mono data includes:

wherein the target content scene type is a content scene type that occurs the most frequently among the plurality of content scene types.

2. The method of claim 1, wherein the target reverberation effect parameter depends on the target content scene type and comprises:

and determining the target reverberation effect parameter corresponding to the target content scene type according to a mapping relation between a preset content scene type and the reverberation effect parameter.

3. The method of any of claims 1-2, wherein the processing the mono data to obtain the target binaural data according to the target reverberation effect parameter comprises:

acquiring a first three-dimensional coordinate of the sound source;

4. The method of claim 3,

generating target reverberation binaural data according to the first three-dimensional coordinate, the second three-dimensional coordinate, the mono data, and the target reverberation effect parameter, including:

5. The method of claim 4, wherein said synthesizing the multi-channel binaural data into the target reverberant binaural data according to the target reverberation effect parameter comprises:

taking the first three-dimensional coordinate and the second three-dimensional coordinate as axes to be taken as cross sections, dividing the multichannel two-channel data to obtain a first two-channel data set and a second two-channel data set, wherein the first two-channel data set and the second two-channel data set both comprise at least one channel of two-channel data;

6. The method of claim 4, wherein said synthesizing the multi-channel binaural data into the target reverberant binaural data according to the target reverberation effect parameter comprises:

7. A3D sound effect processing method is characterized by comprising the following steps:

acquiring single track data of a sound source;

processing the single-channel data according to a target reverberation effect parameter to obtain target reverberation binaural data, wherein the target reverberation effect parameter depends on a target content scene type, and the target content scene type corresponds to the single-channel data;

wherein the content of the first and second substances,

wherein the target reverberation effect parameter depends on a target content scene type, including: and determining the target reverberation effect parameter corresponding to the target content scene type according to a preset mapping relation between the content scene type and the reverberation effect parameter.

8. A3D sound effect processing method is characterized by comprising the following steps:

acquiring single track data of a sound source;

wherein the content of the first and second substances,

wherein the processing the mono channel data according to the target reverberation effect parameter to obtain the target binaural data includes:

acquiring a first three-dimensional coordinate of the sound source;

9. A3D sound effect processing device, wherein the 3D sound effect processing device comprises: an acquisition unit, a first determination unit, a second determination unit and a processing unit, wherein,

the processing unit is configured to process the mono channel data according to a target reverberation effect parameter to obtain target reverberation binaural data, where the target reverberation effect parameter depends on a target content scene type, and the target content scene type corresponds to the mono channel data;

wherein the content of the first and second substances,

10. A3D sound effect processing device, wherein the 3D sound effect processing device comprises: an acquisition unit, a first determination unit, a second determination unit and a processing unit, wherein,

wherein the content of the first and second substances,

11. A3D sound effect processing device, wherein the 3D sound effect processing device comprises: an acquisition unit, a first determination unit, a second determination unit and a processing unit, wherein,

wherein the content of the first and second substances,

acquiring a first three-dimensional coordinate of the sound source;

12. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-8.

13. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-8.