CN108574925A - The method and apparatus that audio signal output is controlled in virtual auditory environment - Google Patents

The method and apparatus that audio signal output is controlled in virtual auditory environment Download PDF

Info

Publication number
CN108574925A
CN108574925A CN201710149313.7A CN201710149313A CN108574925A CN 108574925 A CN108574925 A CN 108574925A CN 201710149313 A CN201710149313 A CN 201710149313A CN 108574925 A CN108574925 A CN 108574925A
Authority
CN
China
Prior art keywords
azimuth information
sound
variation
orientation
bearing data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710149313.7A
Other languages
Chinese (zh)
Inventor
王彦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dynabook Inc
Original Assignee
Toshiba Corp
Toshiba Client Solutions Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp, Toshiba Client Solutions Co Ltd filed Critical Toshiba Corp
Priority to CN201710149313.7A priority Critical patent/CN108574925A/en
Priority to JP2017113712A priority patent/JP2018152834A/en
Publication of CN108574925A publication Critical patent/CN108574925A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)

Abstract

The present invention provides a kind of method and apparatus controlling audio signal output in virtual auditory environment.The device includes：Audio input unit is configured as receiving one or more audio signals corresponding with one or more of virtual auditory environment sound source；Azimuth information acquiring unit is configured as obtaining the azimuth information of the object in the virtual auditory environment；Determination unit, it is configured as determining that one or more sound bearings data acquisition system corresponding with the azimuth information, one or more of sound bearing data acquisition systems characterize one or more of virtual auditory environment sound source transmission sound wave to the transmission characteristic in orientation corresponding with the azimuth information respectively for the object；Processing unit is configured as being based on one or more of sound bearing data acquisition systems, handle respectively one or more of audio signals, to obtain binaural signals；Audio output unit is configured as exporting the binaural signals.

Description

The method and apparatus that audio signal output is controlled in virtual auditory environment
Technical field
The present invention relates to virtual auditory technologies, more particularly to the method for audio signal output is controlled in virtual auditory environment And device.
Background technology
The sense of hearing of the mankind is that another important channel of external information is perceived in addition to vision.In real acoustic environments, sound The sound wave of source radiation is transferred to attentive listener through through and Ambient approach, and the head through attentive listener, auricle, trunk etc. are raw Ears are transferred to after reason structural scattering or reflection, thus the sound pressure signal received by ears includes the acoustics sky of sound source and environment Between information.Auditory system (including high-rise nervous system) generates corresponding spatial hearing using these information, and to sound source Positioning and to the subjective sensation of ambient acoustic environment.
Virtual auditory environment is so that attentive listener is generated like placing oneself in the midst of nature acoustics by artificially generated or control acoustic enviroment The feeling of environment.Since binaural signals include the main information of sound, thus manual simulation's binaural signals can be passed through The method that earphone or speaker playback is used in combination makes attentive listener generate the subjective sensation like placing oneself in the midst of certain acoustic environment.
In the prior art, some are disclosed, corresponding spatial hearing is fictionalized to attentive listener by earphone or loud speaker Scheme.For example, in patent document CN 1402593A, a kind of signal processing of the earphone repeat of 5.1 access surround sound is provided Method orientation effect in the head of earphone repeat can be eliminated, and can be without simulating listening room by using this method Room reflections sound under the premise of, retransmit out more natural 5.1 access surround sound effect.
In these current schemes, it is only capable of realization and provides natural spatial hearing by earphone or loud speaker for attentive listener Effect.And when attentive listener itself is moved or is rotated, the sound field experienced without changing therewith, to obtain Immersive effects more true to nature.
Invention content
The present invention provides a kind of method and apparatus of control audio signal output.The method and device can be built virtually Acoustic environments, make the difference for being in attentive listener therein according to own physical orientation, can hear and be consistent with its physical orientation Thus the sound effect of conjunction makes attentive listener obtain true virtual auditory experience.Further, which can be with The physical orientation variation for attentive listener is adjusted, and so that attentive listener is experienced the sound being consistent with the variation of its physical orientation and is imitated Fruit, to enhance the virtual auditory experience of attentive listener.
The method according to an embodiment of the invention that audio signal output is controlled in virtual auditory environment, including： First obtaining step obtains the azimuth information of the object in the virtual auditory environment；First determines step, is that the object is true Fixed one or more sound bearings data acquisition system corresponding with the azimuth information, one or more of sound bearing data It is extremely corresponding with the azimuth information that set characterizes one or more of virtual auditory environment sound source transmission sound wave respectively Orientation transmission characteristic；And first processing step, be based on one or more of sound bearing data acquisition systems, respectively pair with The corresponding one or more audio signals of one or more of sound sources are handled, to obtain the binaural sound letter for output Number.
In a further embodiment, the above method further includes：Second obtaining step obtains in the virtual auditory environment The object variation after azimuth information；Second determines step, determines for the object and believes with the orientation after the variation Manner of breathing corresponding another or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set point It is extremely opposite with the azimuth information after the variation that one or more of virtual auditory environment sound source transmission sound wave is not characterized The transmission characteristic in the orientation answered；And second processing step, based on another or the another multi-acoustical bearing data set, point Other pair of one or more audio signals corresponding with one or more of sound sources are handled, to obtain for the double of output Otoacoustic signal.
The device according to an embodiment of the invention that audio signal output is controlled in virtual auditory environment, including： Audio input unit, be configured as receiving and one or more of virtual auditory environment sound source it is corresponding one or Multiple audio signals；Azimuth information acquiring unit is configured as obtaining the orientation letter of the object in the virtual auditory environment Breath；Determination unit is configured as determining one or more sound bearings corresponding with the azimuth information for the object Data acquisition system, one or more of sound bearing data acquisition systems characterize one or more of described virtual auditory environment respectively Sound source transmits sound wave to the transmission characteristic in orientation corresponding with the azimuth information；Processing unit is configured as being based on institute One or more sound bearing data acquisition systems are stated, one or more of audio signals are handled respectively, to obtain ears Acoustical signal；Audio output unit is configured as exporting the binaural signals.
In a further embodiment, the azimuth information acquiring unit is configured to：It obtains and described virtually listens Feel the azimuth information after the variation of the object in environment；The determination unit is configured to：It is true for the object Fixed corresponding with the azimuth information after the variation another or another multi-acoustical bearing data set, it is described another or separately Multi-acoustical bearing data set characterize respectively one or more of the virtual auditory environment sound source transmission sound wave to institute State the transmission characteristic in the corresponding orientation of azimuth information after variation；The processing unit is configured to：Based on described Another or another multi-acoustical bearing data set, are respectively handled one or more of audio signals, double to obtain Otoacoustic signal.
Description of the drawings
Fig. 1 shows the control audio signal output according to an embodiment of the invention in single object virtual auditory environment The structure diagram of device；
Fig. 2 shows the schematic diagrames of single object virtual reality system according to an embodiment of the invention；
Fig. 3 shows the control audio signal output according to an embodiment of the invention in single object virtual auditory environment The flow chart of method；
Fig. 4 shows that the audio signal according to an embodiment of the invention that controlled in the multipair environment as virtual auditory exports The structure diagram of device；
Fig. 5 shows the multipair schematic diagram as virtual reality system according to an embodiment of the invention；And
Fig. 6 shows that the audio signal according to an embodiment of the invention that controlled in more single object virtual auditory environment exports Method flow chart.
Specific implementation mode
The transmission characteristic in sound source transmission sound wave to a certain specific orientation can be expressed as function data set, this characterization The function data set of sound transmission characteristics can be used for handling audio signal, and audio signal is made to embody the sound source transmission sound wave To the transmission characteristic in the orientation.When this treated audio signal is converted into sound via playback equipment, which is It shows the sound source and transmits sound wave to the transmission characteristic in the orientation, attentive listener is enable to experience virtual sound source space side Position.If characterizing sound source respectively with multiple function data set transmits sound wave to the transmission characteristic in multiple specific orientation, this is used in combination Multiple function data set handle audio signal respectively, so that it may so that audio signal shows the sound source transmission sound wave extremely respectively The transmission characteristic in multiple specific orientation.Virtual acoustic environments can be built by this scheme, on this basis, if will incline The real physical orientation projection of hearer is the specific orientation in virtual auditory environment, attentive listener the real physical orientations of difference and Correspondence is established between the specific orientation of difference in virtual auditory environment, so that it may so that attentive listener is according to own physical orientation Difference hears the sound effect being consistent with own physical orientation.This technical solution proposed by the invention can make to listen attentively to The set sense of space direction of the no longer unalterable embodiment of the sound that person experiences, and the physical orientation with attentive listener can be embodied The sense of space direction being consistent.
In the inventive solutions, the function data set for characterizing sound transmission characteristics is referred to as sound source Bearing data set.It can be such as head-related transfer function data to be suitable for the invention sound bearing data acquisition system (Head Related Transfer Function, HRTF) gathers, interaural difference data (Interaural Time Difference, ITD) set, any sound wave appropriate that can be characterized sound source and send out such as interaural intensity difference data (IID) set It is transmitted to the data acquisition system of the transmission characteristic in a certain orientation.
ITD refers to the distance difference from left and right ear due to sound source so that acoustical signal reaches time difference when ears.IID is Refer to the distance difference due to sound source from left and right ear so that acoustical signal reaches intensity difference when ears.ITD and IID is sound source position Set the function with frequency of sound wave.When sound source localization data collection is combined into ITD and IID data acquisition systems, user can differentiate sound source position In its left side or right side.
HRTF is the acoustic transmission function from sound source to ears in the case of free field, is used for describing in free found field The sound wave that sound source is sent out is incident on the changing features occurred when certain point in duct at a certain angle.HRTF is sound source position, sound The function of wave frequency rate and human body surface shape and property.From sound source to anthropological measuring, the unit impulse response of point is known as head phase Close impulse response (Head Related Impulse Response, HRIR).HRTF is the Fourier transform of HRIR.Work as sound source When location data collection is combined into HRTF data acquisition systems, user can differentiate sound source and be located at its front, rear, top, lower section, left side also It is right side.
For a better understanding of the present invention, hereinafter will by taking HRTF data acquisition systems as an example to sound bearing data acquisition system this Application in inventive technique scheme illustrates.Before each embodiment of the detailed narration present invention, HRTF is obtained first Mode is taken to briefly introduce.
HRTF can be acquired by testing to measure.It measures and is usually carried out in free field (for example, anechoic room).HRTF's Measuring principle is similar with the common measurement of acoustic transmission function.Sound source (for example, miniature loudspeaker system) generates pumping signal, Sound pressure signal at subject's ears is picked up using a pair of of microphone, then finds out transfer function with the method for signal processing.It is logical The relative position changed between sound source and subject is crossed, then can obtain the HRTF of different sound source positions.As for acoustic pressure at ears Microphone can be placed on the end of duct simulator, be equivalent to eardrum by the pickup of signal to dummy head model subject Position measures；And to true man subject, micro microphone can be placed on to ear canal mouth and measured.
HRTF can also be acquired by way of calculating.For example, head can be reduced to rigid ball, by ears letter Opposite on spherical surface 2 points are turned to, HRTF is then calculated to the Raylei scattering formulas of plane incidence wave using rigid ball.Example again Such as, in the case where considering that trunk influences, head and trunk can be reduced to the ball of 2 different radiis, then use Green The method of function and multiple scattering calculates HRTF.Further, it is also possible to using the methods of laser or CT scan, by the outer of research object Shape is converted into computer picture, then calculates HRTF with approximation method.
It the above is only and enumerated for made by several known modes for obtaining HRTF, it should be appreciated that by any The HRTF that the known or following mode that may be developed obtains can be applied to the basic conception of the present invention, fall therefrom Enter within protection scope of the present invention.
It is directed to single object virtual auditory environment and multipair as virtual auditory environment is to the present invention respectively below with reference to attached drawing Each embodiment illustrate.The virtual auditory environment can be applied to several scenes, for example, fictitious air space flight is instructed Experienced, virtual military training, virtual entertainment system, Virtual meeting etc..Under normal conditions, various application scenarios listed above It works in coordination including virtual auditory environment and virtual vision environment, the two simultaneously and constructs relatively complete virtual reality ring Border.In the reality environment, user is projected as experiencer as the object in reality environment, the behavior phase of user It is projected the activity in reality environment for object with answering, for example, user can pass through the fortune certainly in actual environment Move the movement to realize object in virtual reality scenario.It should be noted that referred to below for being easy to understand and describing letter Just, the object that user is projected in user with the real world and reality environment is not distinguished strictly.For example, if necessary Azimuth information of the object in reality environment is obtained, practical operation step is the orientation detection unit inspection worn by user Survey physical orientation of the user in reality, then the correspondence between the object in user and reality environment based on real world Relationship obtains azimuth information of the object in reality environment.It, can directly will be such for the ease of understanding and describing simplicity Operation is known as detecting the azimuth information of object.In addition it should also be noted that, virtual auditory environment of the present invention can not The individualism dependent on virtual vision environment.
Single object virtual auditory environment
Fig. 1 shows the control audio signal output according to an embodiment of the invention in single object virtual auditory environment Device 10 (hereinafter referred to as " device 10 ").In single object virtual auditory environment, the number of object is only one.As shown in Figure 1, Device 10 includes audio input unit 110, azimuth information acquiring unit 120, determination unit 130, processing unit 140 and audio Output unit 150.
Referring to Fig.1, audio input unit 110 is for receiving and conveying audio signal to processing unit 140.Audio signal can To be the audio signal for example acquired by microphone, or by the audio signal of audio player output.Audio signal can be Such as monophonic, two-channel or stereo audio signal.
In order to determine that orientation of the object in virtual auditory environment, azimuth information acquiring unit 120 obtain the orientation of object Information.The orientation information includes orientation of the object in the single object virtual auditory environment.
The azimuth information of object can provide unit (not shown) by azimuth information and azimuth information is supplied to obtain list Member 120.The function that azimuth information provides unit can be accomplished in several ways, these modes can be used alone, can also It is applied in combination.For example, user can detect user in virtual auditory equipped with orientation detection unit by orientation detection unit Then the azimuth information detected is supplied to azimuth information acquiring unit 120 by the orientation of the object projected in environment.Or Person, user can also receive user to it in virtual auditory environment equipped with user input interface by user input interface The input in the orientation of middle projected object, is then supplied to azimuth information acquiring unit 120 by azimuth information.Orientation detection list Member can be, such as any suitable element or member that can be used in detecting object aspect such as gyroscope, compass, accelerometer Part combines.The input of user input interface may include for example word, symbol, voice or to the click of the icon on interface, The various ways such as dragging.
After azimuth information acquiring unit 120 obtains the azimuth information of object, determination unit 130 is transmitted it to.By determining 130 object-based azimuth information of unit determines one or more HRTF data corresponding with its azimuth information for the object Set, and will determine that information is transmitted to processing unit 140.Determine that information can be in any number of forms, for example, determining that information can be with Including one or more HRTF data acquisition systems itself, or only include the rope of identified one or more HRTF data acquisition systems Draw.For the latter, need to be stored with HRTF data acquisition systems corresponding with each index in processing unit 140.
The above-mentioned one or more HRTF data acquisition systems determined for the object are for characterizing in single object virtual auditory environment One or more sound sources transmission sound wave to the object residing for orientation transmission characteristic.When opposite in single object virtual auditory environment When the sound source of the object is single sound source, determination unit 130 determines one corresponding with the single sound source for the object HRTF data acquisition systems.When in single object virtual auditory environment relative to the sound source of the object be multi-acoustical when, determination unit 130 Multiple HRTF data acquisition systems corresponding with multiple sound source are determined for the object.
HRTF numbers for characterizing the transmission characteristic in orientation residing for the transmission sound wave to object of the sound source in virtual auditory environment It can be acquired by testing measurement method or calculation according to set.In one example, can by test measure or Calculation obtains one-to-one azimuth information between each other and HRTF data acquisition systems in advance.Acquired correspondence can be with It is pre-stored in the internal storage unit (not shown) of device 10, or the external memory except device 10 can also be pre-stored in In.In this example embodiment, determination unit 130 can be based on the azimuth information of the object, and the azimuth information that prestores and HRTF numbers According to the correspondence of set, the one or more HRTF data acquisition systems to match for the Object Selection and its azimuth information.
It should be understood that in the practical measurement of HRTF, it is typically only capable to accomplish the measurement to M confined space orientation, And can not measurement of the limit to all dimensional orientations, therefore, if azimuth information acquiring unit 120 obtain and be transmitted to determine it is single The azimuth information of member 130 is not included within the M confined space orientation, and determination unit 130 can also use Spatial Interpolation Method, Such as linear interpolation method, calculate HRTF data acquisition systems corresponding with the azimuth information by M known HRTF data acquisition systems.
In another example, it can not also obtain and store corresponding between azimuth information and HRTF data acquisition systems in advance Relationship, but by determination unit 130 according to the azimuth information of the object, and what is prestored calculate other correlations required when HRTF Information calculates one or more HRTF data acquisition systems corresponding with its azimuth information in real time for the object.
Processing unit 140 is based on one corresponding with its azimuth information or more that determination unit 130 is that the object determines A HRTF data acquisition systems, respectively to from one or more of audio input unit 110 and single object virtual auditory environment The corresponding one or more audio signals of sound source are handled, and obtain the binaural signals for output, and transmit it to Audio output unit 150.Wherein, it is wrapped in each HRTF data acquisition systems corresponding with its azimuth information determined for the object Include two groups of HRTF data for left and right ear.
Specifically, for each sound source in virtual auditory environment, processing unit 140 is pair corresponding with the sound source first Audio signal carry out Fourier transform, obtain the frequency domain representation of the audio signal.Then, processing unit 140 will correspond to a left side Two groups of HRTF data of auris dextra carry out inverse fourier transform after being multiplied respectively with the frequency domain representation of audio signal, to obtain for defeated The binaural signals gone out.
It is understood that it is above it is corresponding be signal processing mode in a frequency domain, and corresponding in time domain, be then by Convolution is carried out corresponding to the two groups of HRIR data and audio signal of left and right ear, to obtain the binaural signals for output.
Binaural signals are supplied to user by audio output unit 150.
In view of orientation of the object in single object virtual auditory environment may change, therefore, further In embodiment, device 10 is also based on the azimuth information after the variation of object, for its determination and the azimuth information phase after variation Corresponding another or another multiple HRTF data acquisition systems, are that the Object Selection matches with its present orientation in real time to realize HRTF data acquisition systems effect.It is described in detail below for the further embodiment.
When orientation of the object in virtual auditory environment changes, unit can be provided (in figure not by azimuth information Show) it obtains the Orientation differences of the object and Orientation differences information is supplied to azimuth information acquiring unit 120.Azimuth information carries It can be realized in several ways for the function of unit.It is virtually being listened for example, user can be detected by orientation detection unit Feel the Orientation differences of the object projected in environment, and/or user is received to it in virtual auditory ring by user input interface Then Orientation differences information is supplied to azimuth information acquiring unit 120 by the input of the Orientation differences of the object projected in border. For example, orientation detection unit can in real time or periodically be detected the Orientation differences of the object, it can also be in response to Triggering is detected the Orientation differences of the object.On the other hand, can also the object be inputted by user input interface at any time Orientation differences.Orientation differences of the object in single object virtual auditory environment may include rotation in the horizontal direction, Translation, rotation in the vertical direction, translation or its arbitrary combination.
Azimuth information acquiring unit 120 obtains the Orientation differences information that unit (not shown) is provided from azimuth information Afterwards, the Orientation differences information of object-based former azimuth information and this acquisition, calculates the azimuth information after the variation of object, And the azimuth information after variation is supplied to determination unit 130.Azimuth information after 130 object-based variation of determination unit, For the object determine it is corresponding with its azimuth information after changing another or separately multiple HRTF data acquisition systems.
Alternatively, azimuth information, which provides unit, can also directly acquire the orientation after the variation of the object and by the side after variation Position information is supplied to azimuth information acquiring unit 120.In this case, azimuth information acquiring unit 120 is without executing above-mentioned meter Calculate, and the azimuth information after the variation of acquisition need to be only transmitted to determination unit 130, by determination unit 130 according to variation after Azimuth information, for the object determine it is corresponding with its azimuth information after changing another or separately multiple HRTF data acquisition systems.
Determination unit 130 be object determine with change after azimuth information it is corresponding another or separately multiple HRTF data The mode of set is that object determines one or more HRTF corresponding with azimuth information with the determination unit 130 being outlined above The mode of data acquisition system is similar, and for simplicity, therefore not to repeat here.
Determination unit 130 determine with change after azimuth information it is corresponding another or separately multiple HRTF data acquisition systems Afterwards, processing unit 140 is based on one or more HRTF data acquisition systems, to from audio input unit 110 and virtual auditory The corresponding one or more audio signals of one or more of environment sound source are handled, and the binaural sound for output is obtained Signal.Binaural signals are then supplied to user by audio output unit 150.
Processing unit 140 is based on another or another multiple HRTF data acquisition systems handle one or more audio signals Mode and the processing unit 140 that is outlined above based on one or more HRTF data acquisition systems to one or more audio signals The mode handled is similar, and for simplicity, therefore not to repeat here.
Hereinafter, by taking the single object virtual auditory environment for simulating seabeach scene as an example, in conjunction with Fig. 2 to the control audio of Fig. 1 The device of signal output in concrete application scene with being further described.
Fig. 2 shows exemplary single object virtual reality systems 20 according to an embodiment of the invention.As shown in Fig. 2, single The device 10 of control audio signal output of the object virtual reality system 20 including audio/video player 210, Fig. 1 is (hereinafter referred to as " device 10 "), azimuth information unit 220, bi-telephone 230 and virtual reality glasses 240 are provided.
With reference to Fig. 2, audio/video player 210 is for providing audio signal and vision signal.Audio/video player 210 can be with It is any setting with audio and video playing function appropriate such as PC, smart mobile phone, personal digital assistant, MP3 player It is standby.In the seabeach scene of the present embodiment, when bi-telephone 230 in user's wearing and virtual reality glasses 240, audio and video playing Device 210 is to 240 transmission video signal of virtual reality glasses, the virtual vision in blue sea and white sandy beach to be presented to the user Environment, and it is transferred through the audio signal that device 10 is handled to bi-telephone 230, it is listened with providing a user the virtual of sound of sea wave Feel environment.In addition, in the seabeach scene of the present embodiment, it is assumed that the number of sound source is only one namely sea.
When the object that user is projected enters virtual reality scenario, azimuth information provides unit 220 and is existed by obtaining user Residing physical orientation obtains the orientation of object that user is projected in virtual reality scenario in actual environment, and by object Azimuth information be supplied to device 10.By taking azimuth information offer unit 220 is orientation detection unit as an example, in the sea of the present embodiment In the scene of beach, orientation detection unit obtains object in the virtual seabeach scene by detecting the physical orientation residing for user The variation in direction, position and the two, and azimuth information and Orientation differences information are supplied to device 10.Orientation detection unit It can be arranged in any position appropriate for being able to detect that user's physical orientation, for example, being independently worn on the head of user Position, or be integrated in bi-telephone 230 or virtual reality glasses 240.
By the left ear of object towards for sound source (namely sea) direction, it is assumed that object surface towards orientation when Sounnd source direction be 0 degree, And angle increases when object rotates clockwise, then the orientation that orientation detection unit detects at this time is 90 degree.In device 10 Azimuth information acquiring unit (not shown) obtains 90 degree of azimuth informations from orientation detection unit and transmits it to dress Set the determination unit (not shown) in 10.Then, it is determined for object by determination unit corresponding with 90 degree of azimuth informations HRTF data acquisition systems, and by determine information be transmitted to the processing unit (not shown) in device 10.Processing unit is based on Identified HRTF data acquisition systems corresponding with 90 degree of azimuth informations, to by the audio input unit in device 10 (in figure not Showing) audio signal of sound of sea wave that is received from audio/video player 210 handled, obtain the binaural sound for output Signal.Binaural signals are supplied to user by the audio output unit (not shown) in device 10 by bi-telephone 230. Since the audio signal to the sound of sea wave from audio/video player 210 uses HRTF numbers corresponding with 90 degree of azimuth informations It is handled according to set, therefore, the sound of sea wave that user is experienced will embody orientative feature corresponding with 90 degree of azimuth informations, That is, the sound of sea wave that user is experienced is located at its left ear side.
Further, the physical orientation as user in actual environment changes, such as rotates clockwise 90 degree, then The object that user is projected also rotates clockwise 90 degree in the scene of seabeach, the Orientation differences that orientation detection unit detects at this time To rotate clockwise 90 degree, and the azimuth information acquiring unit that the orientation change information is supplied in device 10 (is not shown in figure Go out).After azimuth information acquiring unit obtains the Orientation differences information for rotating clockwise 90 degree, object-based original azimuth information (namely 90 degree of azimuth informations) and this Orientation differences information (namely rotate clockwise 90 degree of Orientation differences information) obtained, The azimuth information (namely 180 degree azimuth information) after variation is calculated, and the azimuth information after variation is transmitted in device 10 Determination unit (not shown).Determination unit is determined and the side after the variation based on the azimuth information after variation for object The position corresponding HRTF data acquisition systems of information, and will determine that information is supplied to the processing unit (not shown) in device 10. Processing unit is based on identified HRTF data acquisition systems, to being regarded from sound by the audio input unit (not shown) in device 10 The audio signal of the sound of sea wave received at frequency player 210 is handled, and the binaural signals for output are obtained.Device 10 In audio output unit (not shown) binaural signals are supplied to user by bi-telephone 230.Due to coming from The audio signal of the sound of sea wave of audio/video player 210 uses opposite with the azimuth information (180 degree azimuth information) after variation The HRTF data acquisition systems answered are handled, and therefore, the sound of sea wave that user is experienced will embody corresponding to 180 degree azimuth information Orientative feature, that is, the sound of sea wave that user is experienced is located at the dead astern of user.
It should be noted that although above-described embodiment is horizontal turn clockwise of the direction with object in virtual reality scenario For 90 degree dynamic, the change in location of object it is not related to, it will be appreciated by those skilled in the art that technical scheme of the present invention is actually It is applicable to more complicated object aspect variation.For example, the Orientation differences of the object can also include translation, or rotation with The combination of translation.In addition, the translation and rotation are not limited to two dimensional surface, can be the arbitrary translation in three dimensions And rotation.
It is further to note that virtual auditory environment is combined with virtual vision environment in above-mentioned seabeach application scenarios It is described merely to technical scheme of the present invention is better described, technical scheme of the present invention can be only applied to virtually listen Feel the application scenarios of environment.
Fig. 3 shows the control audio signal output according to an embodiment of the invention in single object virtual auditory environment Method.This method includes the first obtaining step S310, the first determination step S320 and the first processing step S330.
With reference to Fig. 3, in the first obtaining step S310, the orientation letter of the object in single object virtual auditory environment is obtained Breath.The orientation information includes orientation of the object in single object virtual auditory environment.
In first determines step S320, one or more sound source sides corresponding with its azimuth information are determined for the object Position data acquisition system.The one or more sound bearing data acquisition system characterizes one or more in single object virtual auditory environment respectively The transmission characteristic in the orientation residing for a sound source transmission sound wave to the object.
Determine that one or more sound bearings data acquisition system corresponding with its azimuth information can be by a variety of for object Mode is realized.For example, can be based on the azimuth information of the object, and according to the azimuth information that prestores and sound bearing data set The correspondence of conjunction, the one or more sound bearing data acquisition systems to match for the Object Selection and its azimuth information.Example again It such as, can be based on the azimuth information of the object, for the calculation and object one or more sound source sides corresponding with its azimuth information Position data acquisition system.
Sound bearing data acquisition system can be for example head related transfer function data acquisition system, interaural difference data acquisition system, Interaural intensity difference data acquisition system etc. is any appropriate can to characterize sonic transmissions that sound source is sent out to the transmission characteristic in a certain orientation Data acquisition system.
In the first processing step S330, based on identified one or more sound bearings data acquisition systems, respectively pair with The corresponding one or more audio signals of one or more of single object virtual auditory environment sound source are handled, to obtain Binaural signals for output.
In view of orientation of the object in single object virtual auditory environment may change, therefore, further Embodiment in, be also based on the azimuth information after the variation of the object, for its determination with change after azimuth information it is opposite Another or the another multiple HRTF data acquisition systems answered, are that the Object Selection matches with its present orientation in real time to realize The effect of HRTF data acquisition systems.
In the further embodiment, in addition to above-mentioned first obtaining step, first determine step and the first processing step Further include the second obtaining step, the second determination step and second processing step except rapid.
In the second obtaining step, the azimuth information after the variation of the object in single object virtual auditory environment is obtained. Azimuth information after the variation includes the orientation after variation of the object in single object virtual auditory environment.
Second determine step in, for the object determine it is corresponding with its azimuth information after changing another or separately it is more A sound bearing data acquisition system.This another or another multi-acoustical bearing data set characterize single object virtual auditory environment respectively One or more of sound source transmission sound wave to the object residing for variation after orientation transmission characteristic.
Corresponding with its azimuth information after changing another or another multi-acoustical bearing data collection are determined for the object Conjunction can also be realized in several ways.For example, can be based on the azimuth information after the variation of the object, and according to prestoring The correspondence of azimuth information and sound bearing data acquisition system matches for the Object Selection with the azimuth information after it changes Another or another multi-acoustical bearing data set.In another example can be right for this based on the azimuth information after the variation of the object As calculating corresponding with its azimuth information after changing another or another multi-acoustical bearing data set.
In the second processing step, another or another multi-acoustical bearing data set based on determined by, respectively pair with The corresponding one or more audio signals of one or more of single object virtual auditory environment sound source are handled, to obtain Binaural signals for output.
It is multipair as virtual auditory environment
Fig. 4 shows that the audio signal according to an embodiment of the invention that controlled in the multipair environment as virtual auditory exports Device 40 (hereinafter referred to as " device 40 ").In object virtual auditory environment, the number of object is multiple.As shown in figure 4, device 40 include audio input unit 410, azimuth information acquiring unit 420, determination unit 430, processing unit 440 and audio output Unit 450.
With reference to Fig. 4, audio input unit 410 is for receiving and conveying audio signal to processing unit 440.For it is multipair as Virtual auditory environment, audio signal can be the audio signals for example acquired by microphone, or by audio player output Audio signal.The audio signal typically acquired by microphone can be the audio signal from each object, typically by audio The audio signal of player output can be the audio signal of the non-object sound source in virtual auditory environment.Described herein comes From the audio signal of each object, actually it is meant that referring to each user is acquired by microphone and be transmitted to audio input unit 410 Audio signal, herein be referred to as the audio signal from each object；Similar, described is non-in virtual auditory environment The audio signal of object sound source is actually meant that finger by audio player plays audio file and is transmitted to audio input unit 410 audio signal is referred to as the audio signal of the non-object sound source in virtual auditory environment at this.Audio signal can be Such as monophonic, two-channel or stereo audio signal.
In order to determine that orientation of the object in virtual auditory environment, azimuth information acquiring unit 420 obtain multipair as virtual The azimuth information of each object in acoustic environments.
In one example, each object only receives the audio signal of the non-object sound source in virtual auditory environment. Under this situation, the azimuth information of each object can only include orientation of the object in virtual auditory environment.
In another example, each object receives the audio signal of the non-object sound source in virtual auditory environment, with And the audio signal from other objects；Or only receive the audio signal from other objects.It is described under this two classes situation The azimuth information of each object includes that orientation and the object of the object in virtual auditory environment are opposite with other objects Orientation.
The azimuth information of object can provide unit (not shown) by azimuth information and azimuth information is supplied to obtain list Member 420.The function that azimuth information provides unit can be accomplished in several ways, these modes can be used alone, can also It is applied in combination.For example, each user can respectively be equipped with orientation detection unit, user is detected in void by orientation detection unit The orientation of the object projected in quasi- acoustic environments, is then supplied to azimuth information acquiring unit by the azimuth information detected 420.Orientation detection unit can be such as gyroscope, compass, accelerometer it is any it is appropriate can be used in detect object side Element or the element combination of position.Alternatively, each user can also be equipped with independent user input interface, by User Defined its The relative bearing in the orientation of the object projected in virtual auditory environment and the object and other objects.User inputs boundary The input in face may include such as word, symbol, voice or to various ways the click of the icon on interface, dragging.It is logical The azimuth information for crossing user input interface input is thereupon provided to azimuth information acquiring unit 420.Above-described detection side Formula and User Defined mode both can be used alone, and can also be combined with each other use.
After azimuth information acquiring unit 420 obtains the azimuth information of each object, determination unit 430 is transmitted it to, by Azimuth information of the determination unit 430 based on each object, respectively each object determine one corresponding with its azimuth information Or multiple HRTF data acquisition systems, and will determine that information is transmitted to processing unit 440.Determine that information can be in any number of forms, example Such as, it determines that information may include one or more HRTF data acquisition systems itself, or only includes identified one or more The index of HRTF data acquisition systems.For the latter, need to be stored with HRTF data sets corresponding with each index in processing unit 440 It closes.
The above-mentioned one or more HRTF data acquisition systems determined for each object are multipair as virtual auditory environment for characterizing One or more of sound source transmission sound wave to the object residing for orientation transmission characteristic.When phase in the multipair environment as virtual auditory When sound source for certain an object is single sound source, determination unit 430 determines corresponding with the single sound source for the object One HRTF data acquisition system.When in the multipair environment as virtual auditory relative to the sound source of certain an object be multi-acoustical when, determine Unit 430 is determined for the object and multiple HRTF data acquisition systems corresponding to a sound source, corresponding to multiple sound source.
Determination unit 430 is that each object determines that one or more HRTF data acquisition systems corresponding with its azimuth information can To realize in several ways.For example, determination unit 430 can be based on the azimuth information of each object, and the side that prestores The correspondence of position information and HRTF data acquisition systems determines the one or more to match with its azimuth information for each object HRTF data acquisition systems.Azimuth information and the acquisition modes of the correspondence of HRTF data acquisition systems have hereinbefore been explained, are For the sake of simplicity, therefore not to repeat here.In another example determination unit 430 can be according to each object azimuth information, and prestore Other relevant informations required when HRTF are calculated, one corresponding with its azimuth information or more is calculated in real time for each object A HRTF data acquisition systems.
Processing unit 440 based on determination unit 430 be each object determine one corresponding with its azimuth information or Multiple HRTF data acquisition systems, respectively to from audio input unit 410 with one or more in the multipair environment as virtual auditory The corresponding one or more audio signals of a sound source are handled, and the binaural signals for output are obtained.Then, audio is defeated Go out unit 450 and binaural signals are respectively supplied to corresponding user.
In view of orientation of each object in the multipair environment as virtual auditory and/or can with the relative bearing of other objects It can change, therefore, in a further embodiment, device 40 is also based on the letter of the orientation after the variation of each object Breath determines corresponding with its azimuth information after changing another or another multiple HRTF data acquisition systems for each object, thus Realize be in real time the HRTF data acquisition systems that each Object Selection matches with the orientation after it changes effect.Below for The further embodiment is described in detail.
When orientation of certain an object in virtual auditory environment in multiple objects and/or the relative bearing with other objects When changing, the Orientation differences that unit (not shown) obtains the object can be provided by the azimuth information that the object is equipped with And/or changes with the relative bearing of other objects and Orientation differences information is supplied to azimuth information acquiring unit 420.
In one example, each object only receives the audio signal of the non-object sound source in virtual auditory environment. Under this situation, the Orientation differences information of each object can only include orientation of the object in virtual auditory environment and become Change.
In another example, each object receives the audio signal of the non-object sound source in virtual auditory environment, with And the audio signal from other objects；Or only receive the audio signal from other objects.It is described under this two classes situation The azimuth information of each object includes Orientation differences and the object and other objects of the object in virtual auditory environment Relative bearing changes.
The function that azimuth information provides unit can be accomplished in several ways, these modes can be used alone, and also may be used To be applied in combination.For example, each user can respectively be equipped with orientation detection unit, by orientation detection unit come in real time, week Phase property or the Orientation differences of object that are projected in virtual auditory environment in response to detection trigger user, then will inspection The Orientation differences information measured is supplied to azimuth information acquiring unit 420.Side of the object in the multipair environment as virtual auditory Position variation may include rotation or translation in the horizontal direction, and/or rotation or flat in the vertical direction Move or a combination thereof.Alternatively, each user can also be equipped with independent user input interface, by User Defined, it is virtually being listened The relative bearing of the Orientation differences and the object and other objects of feeling the object projected in environment changes.User inputs boundary The input in face may include such as word, symbol, voice or to various ways the click of the icon on interface, dragging.It is logical The Orientation differences information for crossing user input interface input is thereupon provided to azimuth information acquiring unit 420.Above-described inspection Survey mode and User Defined mode both can be used alone, and can also be combined with each other use.
Above-described detection mode and User Defined mode can be combined with each other, jointly to azimuth information acquiring unit 420 provide Orientation differences information of the object in the multipair environment as virtual auditory.For example, when entering virtual auditory environment, User can be by self-defined its object that is projected of input of user input interface in virtual auditory environment orientation, and/or The relative bearing of the object and other objects, and it is supplied to azimuth information acquiring unit using azimuth information as initial orientation information 420.Then by the orientation detection unit provisioned in each user in real time, periodically or in response to detection trigger user The Orientation differences of institute's projective object in virtual auditory environment, and it is supplied to azimuth information to obtain the Orientation differences information detected Take unit 420.In the process, user can also be at any time by the self-defined regulating object of user input interface in virtual auditory ring The relative bearing in orientation and/or the object and other objects in border.
After azimuth information acquiring unit 420 obtains Orientation differences information of each object in virtual auditory environment, it is based on The Orientation differences information of the former azimuth information and this acquisition of each object, calculates the azimuth information after variation, and will be each Azimuth information after the variation of object is supplied to determination unit 430.Orientation after variation of the determination unit 430 based on each object Information, for each object determine it is corresponding with its azimuth information after changing another or separately multiple HRTF data acquisition systems.
Alternatively, the azimuth information that object is equipped with provide unit can also directly acquire the orientation after the variation of the object and/ Or with the relative bearing after the variation of other objects and the azimuth information after variation is supplied to azimuth information acquiring unit 120. In this case, azimuth information acquiring unit 420 is without executing above-mentioned calculating, and only need to will be after the variation of each object of acquisition Azimuth information be transmitted to determination unit 430, by the azimuth information after the variation of determination unit 430 according to each object, respectively For each object determine it is corresponding with its azimuth information after changing another or separately multiple HRTF data acquisition systems.
Determination unit 430 be each object determine it is corresponding with its azimuth information after changing another or separately it is multiple After HRTF data acquisition systems, processing unit 440 is based on one or more HRTF data acquisition systems, to coming from audio input unit 410 One or more audio signals corresponding with one or more of virtual auditory environment sound source handled, be used for The binaural signals of output.Binaural signals are then respectively supplied to corresponding user by audio output unit 450.
The device according to an embodiment of the invention that audio signal output is controlled in the multipair environment as virtual auditory can be with Positioned at distal end, for example, being integrated in remote server；It can also be located locally end, for example, being integrated in local audio and video playing In equipment or Multi-Party Conference equipment.
It should be noted that although above-described embodiment is defeated with the control audio signal in the multipair environment as virtual auditory The device gone out obtains the azimuth information of each object in multiple objects and is determined for each object corresponding with its azimuth information One or more HRTF data acquisition systems for be described, should it will be appreciated by those skilled in the art that in other embodiments The device of control audio signal output can also only obtain the azimuth information of the partial objects in multiple objects and be respectively the portion Object is divided to determine one or more HRTF data acquisition systems corresponding with its azimuth information.For example, when the control audio signal is defeated When the device gone out is located locally end, it can only obtain the azimuth information of native object and be determined for native object and be believed with its orientation The corresponding one or more HRTF data acquisition systems of manner of breathing, the azimuth information of native object may include native object multipair as void Orientation in quasi- acoustic environments and/or the relative bearing with other objects.The number of the native object can be one or more It is a.
Hereinafter, by taking the multipair environment as virtual auditory for simulating Multi-Party Conference scene as an example, the control in conjunction with Fig. 5 to Fig. 4 The device of audio signal output in concrete application scene with being further described.
Fig. 5 shows according to an embodiment of the invention exemplary multipair as virtual reality system 50.As shown in figure 5, more Object virtual reality system 50 include Multi-Party Conference equipment 510, the control audio signal of Fig. 4 output device 40 (hereinafter referred to as " device 40 "), user input interface 520, bi-telephone 530 and virtual reality glasses 540.In this application scene, with dress Set 40 be located locally hold and only receive native object azimuth information and variation azimuth information for illustrate.
With reference to Fig. 5, Multi-Party Conference equipment 510 provides vision signal and the audio signal from participant, the video Signal is used to simulate the virtual vision environment of Multi-Party Conference scene, and the audio signal is for simulating the virtual of Multi-Party Conference scene Acoustic environments.When bi-telephone 530 in user's wearing and virtual reality glasses 540, Multi-Party Conference equipment 510 is to virtual reality eye 540 transmission video signal of mirror, virtual Multi-Party Conference scene to be presented to the user.In the virtual Multi-Party Conference scene In, the user and other participants can be appeared to be in the object of different direction respectively.On this basis, Multi-Party Conference Equipment 510 is also transferred through the audio signal from other participants that device 40 is handled to bi-telephone 530, is in simulation The sound of the object of different direction.
Assuming that in the Multi-Party Conference scene of the present embodiment, three participants are shared, they are to be located at 40 institute of device respectively Local user A on ground and strange land participant B and C located elsewhere.Local user A wears 530 He of above-mentioned bi-telephone Virtual reality glasses 540.After Multi-Party Conference equipment 510 is activated, local user A is projected as the sheet in virtual reality scenario Ground object a, strange land participant B and C are projected respectively as other object b and c in virtual reality scenario.Local user A at this time The object b and c in virtual reality scenario can be seen by virtual reality glasses 540, it can also be seen that at least partly seeing The object a being projeced into itself in virtual reality scenario.
In the present embodiment, local user A can input the sheet itself projected by the way that user input interface 520 is self-defined The relative bearing in orientation and object a and other object bs and c of the ground object a in Multi-Party Conference scene, and by this azimuth information The azimuth information acquiring unit (not shown) being supplied to as initial orientation information in device 40.
Azimuth information acquiring unit (not shown) in device 40 obtains the azimuth information and transmits it to device Determination unit (not shown) in 40 is that native object a determines two corresponding with the azimuth information by determination unit HRTF data acquisition systems, and will determine that information is transmitted to the processing unit (not shown) in device 40.One of HRTF numbers To the transmission feature of object a, another HRTF data acquisition system characterizes the sonic transmissions sent out as sound source according to set characterization object b The sonic transmissions that object c is sent out as sound source to object a transmission feature.Then, the processing unit in device 40 (does not show in figure Go out) based on identified two HRTF data acquisition systems, respectively to by the audio input unit (not shown) in device 40 from The audio signal from object b and c received at Multi-Party Conference equipment 510 is handled, and the binaural sound for output is obtained Signal.Binaural signals are posed to local use by the audio output unit (not shown) in device 40 by bi-telephone 530 Family A.Since the audio signal from object b and c is respectively by the processing unit in device 40 based on corresponding with azimuth information Different HRTF data acquisition systems are handled, and the sound from object b and c that local user A is experienced will embody and orientation The corresponding orientative feature of information.According to such scheme, user can be according only to sound in the Multi-Party Conference of virtual reality scenario Just tell the orientation of different participants.For example it is assumed that the azimuth information of the self-defined inputs of local user A is that native object a exists Middle, object b is in its left front, and object c is in its right front, then local user A is practical pleasant to the ear in the virtual Multi-Party Conference The sound of the participant B arrived just in its left front, the sound of participant C also just in its right front and user's A settings, or User A is the same by the virtual vision scene that virtual reality glasses 540 can see, to acoustically enhance user Virtual reality experience.
Further, local user A can also adjust native object a by user input interface 520 at any time and in many ways can Discuss the relative bearing in the orientation and object a and other object b and c in scene.
Azimuth information acquiring unit (not shown) in device 40 obtains the azimuth information after variation and is transmitted It is that native object a is determined and the azimuth information after variation by determination unit to the determination unit (not shown) in device 40 Corresponding two HRTF data acquisition systems.Wherein, the sonic transmissions that a HRTF data acquisition systems characterization object b is sent out as sound source To the transmission feature of object a, the sonic transmissions that another HRTF data acquisition system characterizes object c as sound source and sends out are to object a's Transmission feature.Then, the processing unit (not shown) in device 40 is based on identified two HRTF data acquisition systems, to by The sound from object b and c that audio input unit (not shown) in device 40 receives from Multi-Party Conference equipment 510 Frequency signal is handled, and the binaural signals for output are obtained.Audio output unit (not shown) in device 40 will Binaural signals are supplied to local user by bi-telephone 530.Since the audio signal from object b and c is respectively by device Processing unit in 40 is handled based on HRTF data acquisition systems corresponding with the azimuth information after variation, local user A institutes The sound of the object b and c that experience will embody variation corresponding with azimuth information variation.According to such scheme, user is in void The Orientation differences of different participants can be just told in the Multi-Party Conference of quasi- reality scene according only to sound.For example, in user A On the basis of the azimuth information of preceding primary self-defined input, it is assumed that the azimuth information after the variation of the self-defined inputs of user A is, local Object a is constant in middle, and object b is moved to its front, and object c is moved to its front-right, then user A is practical virtual at this Meeting in the sound of participant B heard move to its front, the sound of participant C also moves to its front-right, It is this time set with user A, or the same by the virtual vision scene that virtual reality glasses 540 can see, to In the virtual reality experience for acoustically further enhancing user.
It is understood that for two strange lands the participant B and C, can also be equipped in its respective local side There is device 40, to provide it similar virtual auditory environment.
Being combined to be described with virtual vision environment by virtual auditory environment in above-mentioned Multi-Party Conference application scenarios is only Technical solution in order to better illustrate the present invention, it is to be understood that technical scheme of the present invention can be only applied to virtually The application scenarios of acoustic environments.
Fig. 6 shows that the audio signal according to an embodiment of the invention that controlled in the multipair environment as virtual auditory exports Method.This method includes the first obtaining step S610, the first determination step S620 and the first processing step S630.
With reference to Fig. 6, in the first obtaining step S610, obtain in multiple objects in the multipair environment as virtual auditory extremely The azimuth information of a few object.The azimuth information of at least one object includes the object in the multipair environment as virtual auditory Orientation and/or relative bearing with other objects.
In first determines step S620, one or more corresponding with its azimuth information is determined at least one object A sound bearing data acquisition system.The one or more sound bearing data acquisition system is characterized respectively in the multipair environment as virtual auditory The transmission characteristic in the orientation residing for one or more sound source transmission sound waves at least one object.
Determine that one or more sound bearings data acquisition system corresponding with its azimuth information can be at least one object It realizes in several ways.For example, can be based on the azimuth information of at least one object, and according to the azimuth information to prestore With the correspondence of sound bearing data acquisition system, one or more to match at least one Object Selection and its azimuth information A sound bearing data acquisition system.In another example can be based on the azimuth information of at least one object, at least one object meter Calculate one or more sound bearings data acquisition system corresponding with its azimuth information.
Sound bearing data acquisition system can be for example head related transfer function data acquisition system, interaural difference data acquisition system, Interaural intensity difference data acquisition system etc. is any appropriate can to characterize sonic transmissions that sound source is sent out to the transmission characteristic in a certain orientation Data acquisition system.
In the first processing step S630, based on identified one or more sound bearings data acquisition systems, respectively pair with It is multipair as the corresponding one or more audio signals of one or more of virtual auditory environment sound source are handled, to obtain Binaural signals for output.
In one example, for each object at least one object, other objects are multipair as virtual as this One or more of acoustic environments sound source.
In view of orientation of at least one object in the multipair environment as virtual auditory may change, therefore, In a further embodiment, the azimuth information being also based on after the variation of at least one object is at least one right for this As determining corresponding with its azimuth information after changing another or another multiple HRTF data acquisition systems, to which realization is in real time The effect for the HRTF data acquisition systems that at least one Object Selection matches with its present orientation.
In the further embodiment, in addition to above-mentioned first obtaining step, first determine step and the first processing step Further include the second obtaining step, the second determination step and second processing step except rapid.
In the second obtaining step, the azimuth information after the variation of at least one object is obtained.At least one object Variation after azimuth information include orientation after variation of the object in the virtual auditory environment and/or with other objects Variation after relative bearing.
In second determines step, determined at least one object corresponding another with its azimuth information after changing A or another multi-acoustical bearing data set.This another or another multi-acoustical bearing data set characterize respectively it is multipair as virtual The transmission that one or more of acoustic environments sound source transmits the orientation after the variation residing for sound wave at least one object is special Property.
Corresponding with its azimuth information after changing another or another multi-acoustical side are determined at least one object Position data acquisition system can also be realized in several ways.For example, orientation that can be after the variation based at least one object Information, and according to the correspondence of the azimuth information and sound bearing data acquisition system that prestore, at least one Object Selection with Another or the another multi-acoustical bearing data set that its azimuth information after changing matches.In another example this can be based on extremely Azimuth information after the variation of a few object, it is corresponding with its azimuth information after changing at least one calculation and object Another or another multi-acoustical bearing data set.
In the second processing step, another or another multi-acoustical bearing data set based on determined by, respectively pair with It is multipair as the corresponding one or more audio signals of one or more of virtual auditory environment sound source are handled, to obtain Binaural signals for output.
It can be completely or partially according to each unit in the device of the control audio signal of various embodiments of the present invention output It is realized by software, hardware, firmware or its arbitrary combination on ground.The example of hardware cell include but not limited to system on chip (SOC), Integrated circuit (IC), application-specific integrated circuit (ASIC), field programmable gate array (FPGA), etc..
The device exported according to the control audio signal of various embodiments of the present invention may be implemented as stand-alone device, that is, All units are all realized in a physical machine.Alternatively, according to the output of the control audio signal of various embodiments of the present invention Device can also be a distributed apparatus, that is, certain units and other units be deployed in respectively different physical machine or Position.
According between the device and external device (ED) of the output of the control audio signal of various embodiments of the present invention or according to this hair Can be currently known by means of any between the inside each unit of the device of the control audio signal output of bright each embodiment or The communication technology of exploitation communicates with one another in the future, and such technology includes but not limited to：Communication between process/thread, bluetooth, it is infrared, Computer network wirelessly or non-wirelessly, communication network, etc..
The above description of the present invention is for enabling any those of ordinary skill of this field to realize or using invention.For For those of ordinary skill in the art, various modifications of the invention will be apparent from, and defined herein general former Reason can also be applied to other deformations in the case where not departing from the spirit and scope of the disclosure.Therefore, the present invention is not It is limited to the example and design of this paper, but it is consistent with the widest scope of principle disclosed herein and novel features.

Claims (27)

1. a kind of method controlling audio signal output in virtual auditory environment, including：
First obtaining step obtains the azimuth information of the object in the virtual auditory environment；
First determines step, and one or more sound bearings data set corresponding with the azimuth information is determined for the object It closes, one or more of sound bearing data acquisition systems characterize one or more of virtual auditory environment sound source biography respectively The transmission characteristic in defeated sound wave extremely orientation corresponding with the azimuth information；And
First processing step is based on one or more of sound bearing data acquisition systems, respectively pair with one or more of sound The corresponding one or more audio signals in source are handled, to obtain the binaural signals for output.
2. according to the method described in claim 1, wherein, further including：
Second obtaining step obtains the azimuth information after the variation of the object in the virtual auditory environment；
Second determine step, for the object determine it is corresponding with the azimuth information after the variation another or separately multiple sound Source bearing data set, another or the another multi-acoustical bearing data set are characterized respectively in the virtual auditory environment The transmission characteristic in one or more sound source transmission sound waves extremely orientation corresponding with the azimuth information after the variation；And
Second processing step, based on another or the another multi-acoustical bearing data set, respectively pair with it is one or more The corresponding one or more audio signals of a sound source are handled, to obtain the binaural signals for output.
3. according to the method described in claim 1, wherein,
Described first determines that step includes：Based on the azimuth information, and according to the azimuth information to prestore and sound bearing data The correspondence of set, the one or more of sound bearing data to match with the azimuth information for the Object Selection Set.
4. according to the method described in claim 1, wherein,
Described first determines that step includes：It is that the calculation and object is corresponding with the azimuth information based on the azimuth information One or more of sound bearing data acquisition systems.
5. according to the method described in claim 2, wherein,
Described second determines that step includes：Based on the azimuth information after the variation, and according to the azimuth information and sound source to prestore The correspondence of bearing data set, for the azimuth information after the Object Selection and the variation match it is described another Or another multi-acoustical bearing data set.
6. according to the method described in claim 2, wherein,
Described second determines that step includes：Based on the azimuth information after the variation, after the calculation and object and the variation Azimuth information corresponding another or the another multi-acoustical bearing data set.
7. according to the method described in claim 1, wherein, the object is an object,
First obtaining step includes：The azimuth information of one object is obtained, the azimuth information includes one Orientation of the object in the virtual auditory environment；
Described first determines that step includes：One or more sound sources corresponding with its azimuth information are determined for one object Bearing data set, one or more of sound bearing data acquisition systems characterize respectively one in the virtual auditory environment or Multi-acoustical transmits sound wave to the transmission characteristic in the orientation of one object.
8. according to the method described in claim 2, wherein, the object is an object,
Second obtaining step includes：Obtain the azimuth information after the variation of one object, the orientation after the variation Information includes the orientation after variation of one object in the virtual auditory environment；
Described second determines that step includes：For one object determine it is corresponding with its azimuth information after changing another Or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set characterize respectively described virtually listens Feel the transmission characteristic in the orientation after one or more of environment sound source transmission sound wave to the variation of one object.
9. according to the method described in claim 1, wherein, the object is multiple objects,
First obtaining step includes：Obtain the azimuth information of at least one of the multiple object object, it is described at least The azimuth information of one object includes orientation of the object in the virtual auditory environment and/or the contra with other objects Position；
Described first determines that step includes：One or more corresponding with its azimuth information is determined at least one object Sound bearing data acquisition system, one or more of sound bearing data acquisition systems characterize one in the virtual auditory environment respectively The transmission characteristic in the orientation of a or multi-acoustical transmission sound wave at least one object.
10. according to the method described in claim 2, wherein, the object is multiple objects,
Second obtaining step includes：The azimuth information after the variation of at least one of the multiple object object is obtained, Azimuth information after the variation of at least one object includes the side after variation of the object in the virtual auditory environment Position and/or with the relative bearing after the variation of other objects；
Described second determines that step includes：It is determined at least one object corresponding with its azimuth information after changing another One or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set characterize the void respectively The biography in the orientation after one or more of quasi- acoustic environments sound source transmission sound wave to the variation of at least one object Defeated characteristic.
11. method according to claim 9 or 10, wherein other for each object at least one object Object is one or more of sound sources in the virtual auditory environment.
12. according to the method described in any one of claim 1-10, wherein the sound bearing data acquisition system includes following One：
Head related transfer function data acquisition system；
Interaural difference data acquisition system；
Interaural intensity difference data acquisition system.
13. a kind of device controlling audio signal output in virtual auditory environment, including：
Audio input unit is configured as receiving and one or more of virtual auditory environment sound source corresponding one A or multiple audio signals；
Azimuth information acquiring unit is configured as obtaining the azimuth information of the object in the virtual auditory environment；
Determination unit is configured as determining one or more sound bearings corresponding with the azimuth information for the object Data acquisition system, one or more of sound bearing data acquisition systems characterize one or more of described virtual auditory environment respectively Sound source transmits sound wave to the transmission characteristic in orientation corresponding with the azimuth information；
Processing unit is configured as being based on one or more of sound bearing data acquisition systems, respectively to one or more A audio signal is handled, to obtain binaural signals；
Audio output unit is configured as exporting the binaural signals.
14. device according to claim 13, wherein
The azimuth information acquiring unit is configured to：Obtain the variation of the object in the virtual auditory environment Azimuth information afterwards；
The determination unit is configured to：It is determined for the object corresponding with the azimuth information after the variation another One or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set characterize the void respectively The biography in one or more of quasi- acoustic environments sound source transmission sound wave extremely orientation corresponding with the azimuth information after the variation Defeated characteristic；
The processing unit is configured to：It is right respectively based on another or the another multi-acoustical bearing data set One or more of audio signals are handled, to obtain binaural signals.
15. device according to claim 14, wherein
The azimuth information acquiring unit is configured as：Receive the Orientation differences letter of the object in the virtual auditory environment Breath, and it is based on the Orientation differences information and the azimuth information, determine the azimuth information after the variation of the object.
16. device according to claim 13, wherein
The determination unit is configured as：Based on the azimuth information, and according to the azimuth information to prestore and sound bearing data The correspondence of set, the one or more of sound bearing data to match with the azimuth information for the Object Selection Set.
17. device according to claim 13, wherein
The determination unit is configured as：It is that the calculation and object is corresponding with the azimuth information based on the azimuth information One or more of sound bearing data acquisition systems.
18. device according to claim 14, wherein
The determination unit is configured as：Based on the azimuth information after the variation, and according to the azimuth information and sound source to prestore The correspondence of bearing data set, for the azimuth information after the Object Selection and the variation match it is described another Or another multi-acoustical bearing data set.
19. device according to claim 14, wherein
The determination unit is configured as：Based on the azimuth information after the variation, after the calculation and object and the variation Azimuth information corresponding another or the another multi-acoustical bearing data set.
20. device according to claim 13, wherein the object is an object,
The azimuth information acquiring unit is configured as：The azimuth information of one object is obtained, the azimuth information includes Orientation of one object in the virtual auditory environment；
The determination unit is configured as：One or more sound sources corresponding with its azimuth information are determined for one object Bearing data set, one or more of sound bearing data acquisition systems characterize respectively one in the virtual auditory environment or Multi-acoustical transmits sound wave to the transmission characteristic in the orientation of one object.
21. device according to claim 14, wherein the object is an object,
The azimuth information acquiring unit is configured as：Obtain the azimuth information after the variation of one object, the variation Azimuth information afterwards includes the orientation after variation of one object in the virtual auditory environment；
The determination unit is configured as：For one object determine it is corresponding with its azimuth information after changing another Or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set characterize respectively described virtually listens Feel the transmission characteristic in the orientation after one or more of environment sound source transmission sound wave to the variation of one object.
22. device according to claim 13, wherein the object is multiple objects,
The azimuth information acquiring unit is configured as：The azimuth information of at least one of the multiple object object is obtained, The azimuth information of at least one object include orientation of the object in the virtual auditory environment and/or with other objects Relative bearing；
The determination unit is configured as：One or more corresponding with its azimuth information is determined at least one object Sound bearing data acquisition system, one or more of sound bearing data acquisition systems characterize one in the virtual auditory environment respectively The transmission characteristic in the orientation of a or multi-acoustical transmission sound wave at least one object.
23. device according to claim 14, wherein the object is multiple objects,
The azimuth information acquiring unit is configured as：Obtain the side after the variation of at least one of the multiple object object Position information, the azimuth information after the variation of at least one object includes variation of the object in the virtual auditory environment Rear orientation and/or with the relative bearing after the variation of other objects；
The determination unit is configured as：It is determined at least one object corresponding with its azimuth information after changing another One or another multi-acoustical bearing data set, another or the another multi-acoustical bearing data set characterize the void respectively The biography in the orientation after one or more of quasi- acoustic environments sound source transmission sound wave to the variation of at least one object Defeated characteristic.
24. the device according to claim 22 or 23, wherein for each object at least one object, Its object is one or more of sound sources in the virtual auditory environment.
25. device according to claim 13, wherein further include：
Azimuth information provides unit, and the orientation for being configured as providing the object to the azimuth information acquiring unit is believed Breath.
26. device according to claim 14, wherein further include：
Azimuth information provides unit, is configured as after the variation that the azimuth information acquiring unit provides the object Azimuth information or the object Orientation differences information.
27. according to the device described in any one of claim 13-23, wherein the sound bearing data acquisition system includes following Any one：
Head related transfer function data acquisition system；
Interaural difference data acquisition system；
Interaural intensity difference data acquisition system.
CN201710149313.7A 2017-03-14 2017-03-14 The method and apparatus that audio signal output is controlled in virtual auditory environment Pending CN108574925A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201710149313.7A CN108574925A (en) 2017-03-14 2017-03-14 The method and apparatus that audio signal output is controlled in virtual auditory environment
JP2017113712A JP2018152834A (en) 2017-03-14 2017-06-08 Method and apparatus for controlling audio signal output in virtual auditory environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710149313.7A CN108574925A (en) 2017-03-14 2017-03-14 The method and apparatus that audio signal output is controlled in virtual auditory environment

Publications (1)

Publication Number Publication Date
CN108574925A true CN108574925A (en) 2018-09-25

Family

ID=63578455

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710149313.7A Pending CN108574925A (en) 2017-03-14 2017-03-14 The method and apparatus that audio signal output is controlled in virtual auditory environment

Country Status (2)

Country Link
JP (1) JP2018152834A (en)
CN (1) CN108574925A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109582273A (en) * 2018-11-26 2019-04-05 联想(北京)有限公司 Audio-frequency inputting method, electronic equipment and audio output device
CN114025287A (en) * 2021-10-29 2022-02-08 歌尔光学科技有限公司 Audio output control method, system and related components

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102064781A (en) * 2010-10-29 2011-05-18 华为终端有限公司 Method and device for adjusting audio frequency of terminal and terminal
CN105163242A (en) * 2015-09-01 2015-12-16 深圳东方酷音信息技术有限公司 Multi-angle 3D sound playback method and device
CN105872928A (en) * 2016-04-26 2016-08-17 惠州Tcl移动通信有限公司 Method and system for generating virtual surround sound based on mobile terminal
CN106412751A (en) * 2016-11-14 2017-02-15 惠州Tcl移动通信有限公司 Earphone capable of discriminating direction and realization method thereof
CN106484099A (en) * 2016-08-30 2017-03-08 王杰 Content reproduction apparatus, the processing system with the replay device and method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5944197A (en) * 1982-09-06 1984-03-12 Matsushita Electric Ind Co Ltd Headphone device
JP3111455B2 (en) * 1990-02-14 2000-11-20 ソニー株式会社 Sound signal reproduction device
JPH0879899A (en) * 1994-09-01 1996-03-22 Sanyo Electric Co Ltd Audio signal reproducing device
JPH10230899A (en) * 1997-02-24 1998-09-02 Motoya Takeyama Man-machine interface of aerospace aircraft

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102064781A (en) * 2010-10-29 2011-05-18 华为终端有限公司 Method and device for adjusting audio frequency of terminal and terminal
CN105163242A (en) * 2015-09-01 2015-12-16 深圳东方酷音信息技术有限公司 Multi-angle 3D sound playback method and device
CN105872928A (en) * 2016-04-26 2016-08-17 惠州Tcl移动通信有限公司 Method and system for generating virtual surround sound based on mobile terminal
CN106484099A (en) * 2016-08-30 2017-03-08 王杰 Content reproduction apparatus, the processing system with the replay device and method
CN106412751A (en) * 2016-11-14 2017-02-15 惠州Tcl移动通信有限公司 Earphone capable of discriminating direction and realization method thereof

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109582273A (en) * 2018-11-26 2019-04-05 联想(北京)有限公司 Audio-frequency inputting method, electronic equipment and audio output device
US11202163B2 (en) 2018-11-26 2021-12-14 Lenovo (Beijing) Co., Ltd. Audio output method, electronic device, and audio output apparatus
CN114025287A (en) * 2021-10-29 2022-02-08 歌尔光学科技有限公司 Audio output control method, system and related components
CN114025287B (en) * 2021-10-29 2023-02-17 歌尔科技有限公司 Audio output control method, system and related components

Also Published As

Publication number Publication date
JP2018152834A (en) 2018-09-27

Similar Documents

Publication Publication Date Title
US10911882B2 (en) Methods and systems for generating spatialized audio
CN108616789B (en) Personalized virtual audio playback method based on double-ear real-time measurement
US9906885B2 (en) Methods and systems for inserting virtual sounds into an environment
JP5894634B2 (en) Determination of HRTF for each individual
TWI687106B (en) Wearable electronic device, virtual reality system and control method
US10257630B2 (en) Computer program and method of determining a personalized head-related transfer function and interaural time difference function
US20150326963A1 (en) Real-time Control Of An Acoustic Environment
JP2019536395A (en) System and method for capturing an image of the pinna and using the pinna image to characterize human auditory anatomy
EP3687190B1 (en) Mapping virtual sound sources to physical speakers in extended reality applications
JPH10500809A (en) Binaural signal synthesis, head related transfer function and its use
JPWO2017134973A1 (en) Sound output device, sound output method, program, and sound system
CN110267166B (en) Virtual sound field real-time interaction system based on binaural effect
US11758347B1 (en) Dynamic speech directivity reproduction
Roginska Binaural audio through headphones
WO2016167007A1 (en) Head-related transfer function selection device, head-related transfer function selection method, head-related transfer function selection program, and sound reproduction device
JP5697079B2 (en) Sound reproduction system, sound reproduction device, and sound reproduction method
CN111492342A (en) Audio scene processing
EP2822301B1 (en) Determination of individual HRTFs
CN108574925A (en) The method and apparatus that audio signal output is controlled in virtual auditory environment
US20240031759A1 (en) Information processing device, information processing method, and information processing system
Geronazzo et al. Acoustic selfies for extraction of external ear features in mobile audio augmented reality
US20230401968A1 (en) Training system and method for sound directional discrimination ability
US11589181B1 (en) System and method for realistic rotation of stereo or binaural audio
WO2019174442A1 (en) Adapterization equipment, voice output method, device, storage medium and electronic device
WO2021194487A1 (en) Head-related transfer functions with antropometric measurements

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20181227

Address after: Tokyo, Japan

Applicant after: Toshiba terminal Solutions Ltd

Address before: Tokyo, Japan, Japan

Applicant before: Toshiba Corp

Applicant before: Toshiba terminal Solutions Ltd

WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180925