CN106331977A - Virtual reality panoramic sound processing method for network karaoke - Google Patents
Virtual reality panoramic sound processing method for network karaoke Download PDFInfo
- Publication number
- CN106331977A CN106331977A CN201610704412.2A CN201610704412A CN106331977A CN 106331977 A CN106331977 A CN 106331977A CN 201610704412 A CN201610704412 A CN 201610704412A CN 106331977 A CN106331977 A CN 106331977A
- Authority
- CN
- China
- Prior art keywords
- processing unit
- song
- earphone
- network
- transmitting terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 170
- 210000005069 ears Anatomy 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims description 70
- 230000008569 process Effects 0.000 claims description 54
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 230000008054 signal transmission Effects 0.000 claims description 12
- 230000001360 synchronised effect Effects 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 11
- 240000006409 Acacia auriculiformis Species 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 8
- 238000012546 transfer Methods 0.000 claims description 8
- 238000000527 sonication Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 5
- 239000011159 matrix material Substances 0.000 description 8
- 230000006855 networking Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 238000001914 filtration Methods 0.000 description 4
- 241001342895 Chorus Species 0.000 description 3
- HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical compound N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
- H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation of the pseudo five- or more-channel type, e.g. virtual surround
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
Abstract
The invention discloses a virtual reality panoramic sound processing method for network karaoke. The virtual reality panoramic sound processing method comprises the following steps that a sending end audio collection device collects audio data, and a sensing unit collects a real-time location of the sending end with respect to orientation of ears of a listening end; a second processing unit performs superposition processing; a first processing unit performs operation processing; a processed signal is transmitted to a playing device of the listening end for playing; variation of location of a network karaoke singer with respect to the ears of a listener is monitored in real time by using a sensing unit, and the audio data of the singer is processed in real time by using a spatial location of the sound of the singer with respect to the ears of the listener, thus achieving a vivid network karaoke effect.
Description
Technical field
The present invention relates to technical field of virtual reality, be specifically related to the virtual reality panorama sonication side of a kind of network K song
Method.
Background technology
Network K song refers to move KTV to a kind of entertainment way of the Internet.Network is play the accompaniment of song, display
The lyrics of song, user sings recording according to accompaniment and the lyrics.Finally voice data recording and accompanying song are mixed,
Form the song that user individual sings.In addition to individual sings, also two people mode to many people chorus of joining in the chorus.Network K sings
In addition to the demand of satisfied individual's K song, it also it is a kind of important the Internet social application.But, in the application of current network K song
Individual or many people recording there is no direction feeling, have impact on the discrimination of voice data and social entertainment orientation.
When presenting content with virtual reality helmet (head-mounted display, HMD) to user, in audio frequency
Hold and play to user by stereophone.We face the problem improving virtual surrounding sound effect.In virtual reality applications,
When playing audio content by stereophone, virtual 3D audio frequency purpose is intended to reach a kind of effect allows user just as with raising
As sound device array environment is listened to, even true as listening the sound in reality.
When making virtual reality audio content, it usually needs the sound element of multiple different azimuth.Generally, improve when participating in the cintest
The method of sense is to follow the tracks of user's headwork, processes sound accordingly.If original sound perceived as from
Dead ahead, when, after user's rotary head to the left 90 degree, sound should be processed so that user's perception sound is from front-right 90 degree.Use
The virtual reality device of this processing mode has many types, the display device followed the tracks of including headed or headed tracking sensing
Stereophone of device etc..Realizing head tracking and also have multiple method, relatively common is to use multiple sensors.Motion passes
Sensor external member generally includes accelerometer, gyroscope and magnetometric sensor.Every kind of sensing in terms of motion tracking and absolute direction
Device has oneself intrinsic strong point and weakness, and therefore practices well is to use sensor " to merge " (sensor fusion) in the future
Combine from the signal of each sensor, produce a more accurate motion detection result.Obtaining end rotation angle
After, it would be desirable to sound is changed accordingly.The feature of network K song is that the multi-user being distributed in diverse geographic location wants
Interactive participation, network K song participant also has various limb actions, position to move while singing, needs according to network K song
Singer, relative to the change in location of listener's ear, changes the sound locus at earphone of singer, just in real time
The network K that can reach true to nature sings effect.
It will be seen that voice data spatial impression and social entertainmentization that virtual reality panorama sound is sung for network K are the heaviest
Want, but still there is no suitable technical scheme at present in this area.In view of this, need a kind of effective network K song virtual existing
The solution of real panorama sound.
Summary of the invention
It is an object of the invention to provide the virtual reality panorama acoustic processing method of a kind of network K song, existing in order to solve
The problem that technology cannot provide virtual reality panorama sound for network K song
For achieving the above object, the invention provides the virtual reality panorama acoustic processing method of a kind of network K song, described side
Method includes:
Audio frequency is sung in transmitting terminal audio collecting device collection；
Sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
First processing unit carries out calculation process；
Second processing unit is overlapped processing；
Signal transmission after process plays out to the playing device of earphone.
Audio frequency is sung in described transmitting terminal audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
Described sensing unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
With the ears line midpoint of earphone human ear as limit, limit and auris dextra line are pole axis positive direction, arrange pole and sit
Mark system, the polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal；
Described sensing unit determines polar coordinate based on GPS location data；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
When background music is panorama sound, first by the second processing unit, performance audio frequency is overlapped with background music, so
After carried out calculation process by the first processing unit the voice data after described superposition be converted to binaural signal；
When background music is stereophonic signal, first carried out calculation process by the first processing unit, audio frequency conversion will be sung
For binaural signal, then by the second processing unit, described binaural signal is overlapped with background music signal.
Described first processing unit carries out calculation process, including:
Described first processing unit is according to described polar coordinate, and it is empty that the voice data after singing audio frequency or superposition is placed on 3D
Certain direction between, the voice data after singing audio frequency or superposition is converted to binaural signal；
Described will sing audio frequency or superposition after the voice data method that is converted to binaural signal, for using HRTF
(Head Related Transfer Function, head related transfer function) wave filter processes, or for using
The conversion of Ambisonic sound field processes.
Described second processing unit is overlapped processing, including:
The described performance audio frequency from specific senders is overlapped with panorama sound background music, obtains final K song
Song content；
It is overlapped described from the performance audio frequency binaural signal of specific senders and the signal of background music, obtains
Whole K sings song content；
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
Signal transmission after described process plays out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server enter
Row distribution transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The first described processing unit may be disposed at transmitting terminal, or is arranged at server, or is arranged at earphone；Institute
The second processing unit stated may be disposed at transmitting terminal, or is arranged at server, or is arranged at earphone.
Described first processing unit is according to described polar coordinate, and it is empty that the voice data after singing audio frequency or superposition is placed on 3D
Certain direction between, the voice data after singing audio frequency or superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised,
And by server, the described polar coordinate being manually set or revising are transmitted to the first processing unit.
Each network K song participant is provided with transmitting terminal and earphone.
The inventive method has the advantage that and is distributed in diverse geographic location, multiuser interactive according to network K song participant
Participating in, network K song participant also has various limb actions and position to move the feature waited while singing, and uses sensing list
Unit, real time monitoring network K song singer is relative to the change in location of listener's ear, and the sound using singer in real time is relative
Processing the voice data of singer in the locus of listener's ear, the network K reaching true to nature sings effect.
Accompanying drawing explanation
The virtual reality panorama acoustic processing method schematic flow sheet of Fig. 1 inventive network K song.
The polar coordinate system schematic diagram that Fig. 2 present invention is arranged with the ears line midpoint of earphone human ear for limit.
Detailed description of the invention
Following example are used for illustrating the present invention, but are not limited to the scope of the present invention.
Embodiment 1
Refer to the virtual reality panorama acoustic processing method of Fig. 1, a kind of network K song, described virtual reality panorama sonication
Method comprises the steps:
Step S101: audio frequency is sung in transmitting terminal audio collecting device collection；
Step S102: sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
Step S103: the second processing unit is overlapped processing；
Step S104: the first processing unit carries out calculation process；
Step S105: the playing device of the signal transmission after process to earphone plays out.
Audio frequency is sung in described transmitting terminal audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
Described sensing unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
Refer to Fig. 2, with the line midpoint of the left ear of earphone 1 and auris dextra 2 as limit, limit and auris dextra 2 line are pole axis
Positive direction, arranges polar coordinate system, the polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal, described sensing
Unit determines polar coordinate based on GPS location data, such as, at certain time point, K song participant 3 in networking is at the coordinate of this polar coordinate system
For (ρ1,θ1), K song participant 4 in networking is (ρ at the coordinate of this polar coordinate system2,θ2)；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
Described background music is panorama sound, is first overlapped with background music by performance audio frequency by the second processing unit, so
After carried out calculation process by the first processing unit the voice data after described superposition be converted to binaural signal；
Described first processing unit carries out calculation process, including:
Described first processing unit is according to described polar coordinate, certain side being placed in 3d space by the voice data after superposition
To, the voice data after superposition is converted to binaural signal；
The described method that voice data after superposition is converted to binaural signal, for using HRTF (Head Related
Transfer Function, head related transfer function) wave filter processes.The selection of hrtf filter is relative by transmitting terminal
In earphone human ear towards real time position direction determine, process the binaural signal B that obtains and represent,
B=H S
Wherein, H represents HRTF filtering matrix, and S represents that the voice data of collection, S here can represent to drill corresponding to multiple
Voice data after the superposition of the person of singing.
Described second processing unit is overlapped processing, including:
The described performance audio frequency from specific senders is overlapped with panorama sound background music, obtains final K song
Song content；
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
Signal transmission after described process plays out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server enter
Row distribution transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The first described processing unit is arranged at transmitting terminal, and the second processing unit is arranged at transmitting terminal；Or first processes
Unit is arranged at transmitting terminal, and the second processing unit is arranged at server；Or the first processing unit is arranged at transmitting terminal, at second
Reason unit is arranged at earphone；Or the first processing unit is arranged at server, the second processing unit is arranged at transmitting terminal；Or
First processing unit is arranged at server, and the second processing unit is arranged at server；Or the first processing unit is arranged at service
Device, the second processing unit is arranged at earphone；Or the first processing unit is arranged at earphone, the second processing unit is arranged to be sent out
Sending end；Or the first processing unit is arranged at earphone, the second processing unit is arranged at server；Or the first processing unit sets
Being placed in earphone, the second processing unit is arranged at earphone.
Described first processing unit is according to described polar coordinate, certain side being placed in 3d space by the voice data after superposition
To, the voice data after superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised,
And it is manually set and revises transmission to the first processing unit by server by described.
Each network K song participant is provided with transmitting terminal and earphone.
Embodiment 2
Refer to the virtual reality panorama acoustic processing method of Fig. 1, a kind of network K song, described virtual reality panorama sonication
Method comprises the steps:
Step S101: audio frequency is sung in transmitting terminal audio collecting device collection；
Step S102: sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
Step S103: the second processing unit is overlapped processing；
Step S104: the first processing unit carries out calculation process；
Step S105: the playing device of the signal transmission after process to earphone plays out.
Audio frequency is sung in described transmitting terminal audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
Described sensing unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
Refer to Fig. 2, with the line midpoint of the left ear of earphone 1 and auris dextra 2 as limit, limit and auris dextra 2 line are pole axis
Positive direction, arranges polar coordinate system, the polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal, described sensing
Unit determines polar coordinate based on GPS location data, such as, at certain time point, K song participant 3 in networking is at the coordinate of this polar coordinate system
For (ρ1,θ1), K song participant 4 in networking is (ρ at the coordinate of this polar coordinate system2,θ2)；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
Described background music is panorama sound, is first overlapped with background music by performance audio frequency by the second processing unit, so
After carried out calculation process by the first processing unit the voice data after described superposition be converted to binaural signal；
Described first processing unit carries out calculation process, including:
Described first processing unit is according to described polar coordinate, certain side being placed in 3d space by the voice data after superposition
To, the voice data after superposition is converted to binaural signal；
The described method that voice data after superposition is converted to binaural signal, for use Ambisonic sound field convert into
Row processes.Voice data after superposition is converted to acoustic field signal, then acoustic field signal is converted to virtual speaker array signal,
Virtual speaker array signal is filtered by hrtf filter, obtains binaural signal, process the binaural signal B obtained
Represent,
B=H D T S
H represents HRTF filtering matrix,
D represents sound field decoding matrix,
T represents sound field transition matrix,
S represents that the voice data after superposition, S here can represent the voice data from multiple different singers.
This processing mode is advantageous in that, when the number of chorus is more, the efficiency of this processing mode is higher.
Described second processing unit is overlapped processing, including:
The described performance audio frequency from specific senders is overlapped with panorama sound background music, obtains final K song
Song content；
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
Signal transmission after described process plays out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server enter
Row distribution transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The first described processing unit is arranged at transmitting terminal, and the second processing unit is arranged at transmitting terminal；Or first processes
Unit is arranged at transmitting terminal, and the second processing unit is arranged at server；Or the first processing unit is arranged at transmitting terminal, at second
Reason unit is arranged at earphone；Or the first processing unit is arranged at server, the second processing unit is arranged at transmitting terminal；Or
First processing unit is arranged at server, and the second processing unit is arranged at server；Or the first processing unit is arranged at service
Device, the second processing unit is arranged at earphone；Or the first processing unit is arranged at earphone, the second processing unit is arranged to be sent out
Sending end；Or the first processing unit is arranged at earphone, the second processing unit is arranged at server；Or the first processing unit sets
Being placed in earphone, the second processing unit is arranged at earphone.
Described first processing unit is according to described polar coordinate, certain side being placed in 3d space by the voice data after superposition
To, the voice data after superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised,
And it is manually set and revises transmission to the first processing unit by server by described.
Each network K song participant is provided with transmitting terminal and earphone.
Embodiment 3
Refer to the virtual reality panorama acoustic processing method of Fig. 1, a kind of network K song, described virtual reality panorama sonication
Method comprises the steps:
Step S101: audio frequency is sung in transmitting terminal audio collecting device collection；
Step S102: sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
Step S104: the first processing unit carries out calculation process；
Step S103: the second processing unit is overlapped processing；
Step S105: the playing device of the signal transmission after process to earphone plays out.
Audio frequency is sung in described transmitting terminal audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
Described sensing unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
Refer to Fig. 2, with the line midpoint of the left ear of earphone 1 and auris dextra 2 as limit, limit and auris dextra 2 line are pole axis
Positive direction, arranges polar coordinate system, the polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal, described sensing
Unit determines polar coordinate based on GPS location data, such as, at certain time point, K song participant 3 in networking is at the coordinate of this polar coordinate system
For (ρ1,θ1), K song participant 4 in networking is (ρ at the coordinate of this polar coordinate system2,θ2)；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
Described background music is stereophonic signal, is first carried out calculation process by the first processing unit, will sing audio frequency conversion
For binaural signal；By the second processing unit, described binaural signal is overlapped with background music again.
Described first processing unit carries out calculation process, including:
Described first processing unit, according to described polar coordinate, by singing certain direction that audio frequency is placed in 3d space, will be drilled
Sing audio conversion and be changed to binaural signal；
Described will sing the audio conversion method that is changed to binaural signal, for using HRTF (Head Related Transfer
Function, head related transfer function) performance audio conversion is changed to four road ears (Quad binaural) signal by wave filter, because of
For to process the binaural signal in 4 directions, all it is filtered so singing the hrtf filter that audio frequency will be corresponding with 4 directions.
Process the binaural signal B obtainediRepresent,
Bi=Hi·S
Wherein: i=1～N,
HiRepresenting the HRTF filtering matrix on the i-th tunnel, S represents the performance audio frequency of input.
Described second processing unit is overlapped processing, including:
The described performance audio frequency binaural signal from specific senders is overlapped with background music signal, obtains final
K sing song content, the signal after superposition, represent with B',
B'=B+M
B represents performance audio frequency binaural signal, and M represents background music signal.
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
Signal transmission after described process plays out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server enter
Row distribution transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The first described processing unit is arranged at transmitting terminal, and the second processing unit is arranged at transmitting terminal；Or first processes
Unit is arranged at transmitting terminal, and the second processing unit is arranged at server；Or the first processing unit is arranged at transmitting terminal, at second
Reason unit is arranged at earphone；Or the first processing unit is arranged at server, the second processing unit is arranged at transmitting terminal；Or
First processing unit is arranged at server, and the second processing unit is arranged at server；Or the first processing unit is arranged at service
Device, the second processing unit is arranged at earphone；Or the first processing unit is arranged at earphone, the second processing unit is arranged to be sent out
Sending end；Or the first processing unit is arranged at earphone, the second processing unit is arranged at server；Or the first processing unit sets
Being placed in earphone, the second processing unit is arranged at earphone.
Described first processing unit is according to described polar coordinate, and it is empty that the voice data after singing audio frequency or superposition is placed on 3D
Certain direction between, the voice data after singing audio frequency or superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised,
And it is manually set and revises transmission to the first processing unit by server by described.
Each network K song participant is provided with transmitting terminal and earphone.
Embodiment 4
Refer to the virtual reality panorama acoustic processing method of Fig. 1, a kind of network K song, described virtual reality panorama sonication
Method comprises the steps:
Step S101: audio frequency is sung in transmitting terminal audio collecting device collection；
Step S102: sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
Step S104: the first processing unit carries out calculation process；
Step S103: the second processing unit is overlapped processing；
Step S105: the playing device of the signal transmission after process to earphone plays out.
Audio frequency is sung in described transmitting terminal audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
Described sensing unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
Refer to Fig. 2, with the line midpoint of the left ear of earphone 1 and auris dextra 2 as limit, limit and auris dextra 2 line are pole axis
Positive direction, arranges polar coordinate system, the polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal, described sensing
Unit determines polar coordinate based on GPS location data, such as, at certain time point, K song participant 3 in networking is at the coordinate of this polar coordinate system
For (ρ1,θ1), K song participant 4 in networking is (ρ at the coordinate of this polar coordinate system2,θ2)；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
Described background music is stereophonic signal, is first carried out calculation process by the first processing unit, will sing audio frequency conversion
For binaural signal；By the second processing unit, described binaural signal is overlapped with background music again.
Described first processing unit carries out calculation process, including:
Described first processing unit, according to described polar coordinate, by singing certain direction that audio frequency is placed in 3d space, will be drilled
Sing audio conversion and be changed to binaural signal；
Described will sing the audio conversion method that is changed to binaural signal, for using the conversion of Ambisonic sound field to process, will
The audio conversion of singing gathered is changed to the acoustic field signal in N=4 direction, then acoustic field signal is converted to virtual speaker array letter
Number, virtual speaker array signal is filtered by hrtf filter, obtains the binaural signal B in N=4 directioni,
Bi=Hi·Di·Ri·SAmb
Wherein: i=1～N,
HiRepresent the HRTF filtering matrix on the i-th tunnel,
DiRepresent the decoding matrix on the i-th tunnel,
RiRepresent the spin matrix on the i-th tunnel,
SAmbRepresent the audio frequency sound field of input.
Described second processing unit is overlapped processing, including:
According to transmitting terminal relative to earphone human ear towards real time position, by four road binaural signals are carried out interpolation,
Restore panorama acoustic field signal B,
Wherein: i=1～N, GiRepresent the interpolation coefficient on the i-th tunnel.
In order to keep signal energy, cosine signal can be used as interpolation coefficient, cosine signal GiFunction expression
For,
If Gi＜ 0, then Gi=0；
Wherein: i=1～N, θ represent the level angle that the number of people rotates.
The described performance audio frequency binaural signal from specific senders is overlapped with background music signal, obtains final
K sing song content, the signal after superposition, represent with B',
B'=B+M
B represents performance audio frequency binaural signal, and M represents background music signal.
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
Signal transmission after described process plays out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server enter
Row distribution transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The first described processing unit is arranged at transmitting terminal, and the second processing unit is arranged at transmitting terminal；Or first processes
Unit is arranged at transmitting terminal, and the second processing unit is arranged at server；Or the first processing unit is arranged at transmitting terminal, at second
Reason unit is arranged at earphone；Or the first processing unit is arranged at server, the second processing unit is arranged at transmitting terminal；Or
First processing unit is arranged at server, and the second processing unit is arranged at server；Or the first processing unit is arranged at service
Device, the second processing unit is arranged at earphone；Or the first processing unit is arranged at earphone, the second processing unit is arranged to be sent out
Sending end；Or the first processing unit is arranged at earphone, the second processing unit is arranged at server；Or the first processing unit sets
Being placed in earphone, the second processing unit is arranged at earphone.
Described first processing unit is according to described polar coordinate, and it is empty that the voice data after singing audio frequency or superposition is placed on 3D
Certain direction between, the voice data after singing audio frequency or superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised,
And it is manually set and revises transmission to the first processing unit by server by described.
Each network K song participant is provided with transmitting terminal and earphone.
Although, the present invention is described in detail to have used general explanation and specific embodiment, but at this
On the basis of invention, can make some modifications or improvements it, this will be apparent to those skilled in the art.Therefore,
These modifications or improvements without departing from theon the basis of the spirit of the present invention, belong to the scope of protection of present invention.
Claims (10)
1. the virtual reality panorama acoustic processing method of a network K song, it is characterised in that described virtual reality panorama sonication side
Method includes:
Audio frequency is sung in transmitting terminal audio collecting device collection；
Sensing unit gather transmitting terminal relative to earphone human ear towards real time position；
First processing unit carries out calculation process；
Second processing unit is overlapped processing；
Signal transmission after process plays out to the playing device of earphone.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described transmission
Audio frequency is sung in end audio collecting device collection, including:
Described transmitting terminal is arranged on the microphone apparatus that network K song singer uses；
Described transmitting terminal audio collecting device records the sound of singer, and is converted to the performance audio frequency of number format.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described sensing
Unit gather transmitting terminal relative to earphone human ear towards real time position, including:
Transmitting terminal and earphone are both provided with sensing unit；
Between described sensing unit, between sensing unit and the first processing unit, carry out data exchange by server；
With the ears line midpoint of earphone human ear as limit, limit and auris dextra line are pole axis positive direction, arrange polar coordinate system,
The polar coordinate change in this polar coordinate system of the sensing unit Real-time Collection transmitting terminal；
Described sensing unit determines polar coordinate based on GPS location data；
Described polar coordinate are sent to the first processing unit by described sensing unit in real time.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that
When background music is panorama sound, first by the second processing unit, performance audio frequency is overlapped with background music, then by
First processing unit carries out calculation process and the voice data after described superposition is converted to binaural signal；
When background music is stereophonic signal, first carried out calculation process by the first processing unit, performance audio conversion is changed to double
Ear signal, then by the second processing unit, described binaural signal is overlapped with background music signal.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described first
Processing unit carries out calculation process, including:
Described first processing unit is placed in 3d space according to described polar coordinate, the voice data after singing audio frequency or superposition
Certain direction, the voice data after singing audio frequency or superposition is converted to binaural signal；
Described will sing audio frequency or superposition after the voice data method that is converted to binaural signal, for using HRTF (Head
Related Transfer Function, head related transfer function) wave filter processes, or for using Ambisonic sound
Field transformation processes.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described second
Processing unit is overlapped processing, including:
The described performance audio frequency from specific senders is overlapped with panorama sound background music, obtains final K and sing song
Content；
It is overlapped described from the performance audio frequency binaural signal of specific senders and the signal of background music, obtains final
K sings song content；
Described specific senders can be one, it is also possible to is multiple；
The audio frequency of singing of the plurality of specific senders can be synchronous acquisition, it is also possible to be non-synchronous acquisition.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described process
After signal transmission play out to the playing device of earphone, including:
Signal after described process is sent to server by the first processing unit or the second processing unit, server carry out point
Send out transmission；
Described earphone is arranged on the playing device that network K song singer uses；
Described playing device is earphone.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that described the
One processing unit may be disposed at transmitting terminal, or is arranged at server, or is arranged at earphone；The second described processing unit
May be disposed at transmitting terminal, or be arranged at server, or be arranged at earphone.
The virtual reality panorama acoustic processing method of network K the most according to claim 5 song, it is characterised in that described first
Processing unit according to described polar coordinate, certain direction that the voice data after singing audio frequency or superposition is placed in 3d space,
Voice data after singing audio frequency or superposition is converted to binaural signal, also includes:
Described polar coordinate, can be set on handheld mobile device artificially by network K song participant or be revised, and lead to
Cross server to transmit the described polar coordinate being manually set or revising to the first processing unit.
The virtual reality panorama acoustic processing method of network K the most according to claim 1 song, it is characterised in that each network
K song participant is provided with transmitting terminal and earphone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610704412.2A CN106331977B (en) | 2016-08-22 | 2016-08-22 | A kind of virtual reality panorama acoustic processing method of network K songs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610704412.2A CN106331977B (en) | 2016-08-22 | 2016-08-22 | A kind of virtual reality panorama acoustic processing method of network K songs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106331977A true CN106331977A (en) | 2017-01-11 |
| CN106331977B CN106331977B (en) | 2018-06-12 |
Family
ID=57742711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610704412.2A Active CN106331977B (en) | 2016-08-22 | 2016-08-22 | A kind of virtual reality panorama acoustic processing method of network K songs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106331977B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106851482A (en) * | 2017-03-24 | 2017-06-13 | 北京时代拓灵科技有限公司 | A kind of panorama sound loudspeaker body-sensing real-time interaction system and exchange method |
| CN107016990A (en) * | 2017-03-21 | 2017-08-04 | 腾讯科技（深圳）有限公司 | Audio signal generation method and device |
| CN110832884A (en) * | 2017-07-05 | 2020-02-21 | 索尼公司 | Signal processing device and method, and program |
| CN111158459A (en) * | 2018-11-07 | 2020-05-15 | 辉达公司 | Application of geometric acoustics in immersive Virtual Reality (VR) |
| WO2021196337A1 (en) * | 2020-04-03 | 2021-10-07 | 上海唯二网络科技有限公司 | Method for processing interactive voice data in multi-person vr scene |
| WO2022228220A1 (en) * | 2021-04-27 | 2022-11-03 | 腾讯音乐娱乐科技（深圳）有限公司 | Method and device for processing chorus audio, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101384105A (en) * | 2008-10-27 | 2009-03-11 | 深圳华为通信技术有限公司 | Three dimensional sound reproducing method, device and system |
| CN101933344A (en) * | 2007-10-09 | 2010-12-29 | 荷兰皇家飞利浦电子公司 | Method and apparatus for generating a binaural audio signal |
| CN103607550A (en) * | 2013-11-27 | 2014-02-26 | 北京海尔集成电路设计有限公司 | Method for adjusting virtual sound track of television according to position of watcher and television |
| CN105101027A (en) * | 2014-05-08 | 2015-11-25 | 大北公司 | Real-time Control Of An Acoustic Environment |
| CN105376690A (en) * | 2015-11-04 | 2016-03-02 | 北京时代拓灵科技有限公司 | Method and device of generating virtual surround sound |
| CN105611481A (en) * | 2015-12-30 | 2016-05-25 | 北京时代拓灵科技有限公司 | Man-machine interaction method and system based on space voices |
| CN105797366A (en) * | 2016-03-25 | 2016-07-27 | 中国传媒大学 | Head-wearing type interactive audio game terminal based on sound source location |
| CN105808710A (en) * | 2016-03-05 | 2016-07-27 | 上海斐讯数据通信技术有限公司 | Remote karaoke terminal, remote karaoke system and remote karaoke method |
-
2016
- 2016-08-22 CN CN201610704412.2A patent/CN106331977B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101933344A (en) * | 2007-10-09 | 2010-12-29 | 荷兰皇家飞利浦电子公司 | Method and apparatus for generating a binaural audio signal |
| CN101384105A (en) * | 2008-10-27 | 2009-03-11 | 深圳华为通信技术有限公司 | Three dimensional sound reproducing method, device and system |
| CN103607550A (en) * | 2013-11-27 | 2014-02-26 | 北京海尔集成电路设计有限公司 | Method for adjusting virtual sound track of television according to position of watcher and television |
| CN105101027A (en) * | 2014-05-08 | 2015-11-25 | 大北公司 | Real-time Control Of An Acoustic Environment |
| CN105376690A (en) * | 2015-11-04 | 2016-03-02 | 北京时代拓灵科技有限公司 | Method and device of generating virtual surround sound |
| CN105611481A (en) * | 2015-12-30 | 2016-05-25 | 北京时代拓灵科技有限公司 | Man-machine interaction method and system based on space voices |
| CN105808710A (en) * | 2016-03-05 | 2016-07-27 | 上海斐讯数据通信技术有限公司 | Remote karaoke terminal, remote karaoke system and remote karaoke method |
| CN105797366A (en) * | 2016-03-25 | 2016-07-27 | 中国传媒大学 | Head-wearing type interactive audio game terminal based on sound source location |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107016990A (en) * | 2017-03-21 | 2017-08-04 | 腾讯科技（深圳）有限公司 | Audio signal generation method and device |
| CN106851482A (en) * | 2017-03-24 | 2017-06-13 | 北京时代拓灵科技有限公司 | A kind of panorama sound loudspeaker body-sensing real-time interaction system and exchange method |
| CN110832884A (en) * | 2017-07-05 | 2020-02-21 | 索尼公司 | Signal processing device and method, and program |
| CN110832884B (en) * | 2017-07-05 | 2022-04-08 | 索尼公司 | Signal processing apparatus and method, and computer-readable storage medium |
| CN111158459A (en) * | 2018-11-07 | 2020-05-15 | 辉达公司 | Application of geometric acoustics in immersive Virtual Reality (VR) |
| US11809773B2 (en) | 2018-11-07 | 2023-11-07 | Nvidia Corporation | Application of geometric acoustics for immersive virtual reality (VR) |
| WO2021196337A1 (en) * | 2020-04-03 | 2021-10-07 | 上海唯二网络科技有限公司 | Method for processing interactive voice data in multi-person vr scene |
| WO2022228220A1 (en) * | 2021-04-27 | 2022-11-03 | 腾讯音乐娱乐科技（深圳）有限公司 | Method and device for processing chorus audio, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106331977B (en) | 2018-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106331977A (en) | Virtual reality panoramic sound processing method for network karaoke | |
| CN105872940B (en) | A kind of virtual reality sound field generation method and system | |
| US6021206A (en) | Methods and apparatus for processing spatialised audio | |
| CN105101027A (en) | Real-time Control Of An Acoustic Environment | |
| CN105163242B (en) | A kind of multi-angle 3D sound back method and device | |
| CN106210990B (en) | A kind of panorama sound audio processing method | |
| CN107040843A (en) | The method and collecting device of same source of sound are obtained by two microphones | |
| CN102100089A (en) | Angle-dependent operating device or method for obtaining a pseudo-stereophonic audio signal | |
| CN105120418B (en) | Double-sound-channel 3D audio generation device and method | |
| CN106454686A (en) | Multi-channel surround sound dynamic binaural replaying method based on body-sensing camera | |
| Bujacz et al. | Sound of Vision-Spatial audio output and sonification approaches | |
| CN104363555A (en) | Method and device for reconstructing directions of 5.1 multi-channel sound sources | |
| CN105509691B (en) | The detection method of multisensor group fusion and the circular method for acoustic for supporting head tracking | |
| JP2020088516A (en) | Video conference system | |
| Malham | Toward reality equivalence in spatial sound diffusion | |
| CN106658345A (en) | Virtual surround sound playing method, device and equipment | |
| CN116456247A (en) | Stereo playback method, apparatus, microphone device, sound box device, and medium | |
| Yuan et al. | Sound image externalization for headphone based real-time 3D audio | |
| Jenny et al. | Can I trust my ears in VR? Literature review of head-related transfer functions and valuation methods with descriptive attributes in virtual reality | |
| CN106851482A (en) | A kind of panorama sound loudspeaker body-sensing real-time interaction system and exchange method | |
| JP6587047B2 (en) | Realistic transmission system and realistic reproduction device | |
| CN113347530A (en) | Panoramic audio processing method for panoramic camera | |
| CN105307086A (en) | Method and system for simulating surround sound for two-channel headset | |
| CN109168125A (en) | A kind of 3D sound effect system | |
| CN101656525A (en) | Method for acquiring filter and filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |