CN106686520B - The multi-channel audio system of user and the equipment including it can be tracked - Google Patents
The multi-channel audio system of user and the equipment including it can be tracked Download PDFInfo
- Publication number
- CN106686520B CN106686520B CN201710001369.8A CN201710001369A CN106686520B CN 106686520 B CN106686520 B CN 106686520B CN 201710001369 A CN201710001369 A CN 201710001369A CN 106686520 B CN106686520 B CN 106686520B
- Authority
- CN
- China
- Prior art keywords
- loudspeaker
- user
- relative position
- relative
- audio signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
Abstract
Disclose the multi-channel audio system that can track user and the equipment including it.A kind of multi-channel audio system includes multiple loudspeakers, imaging sensor, sound transducer and controller.There is scheduled first relative position between image and sound transducer.Controller includes: loudspeaker position detection unit, and the predetermined audio signal from each loudspeaker for being detected based on sound transducer determines the second relative position of each loudspeaker；User location detection unit, user images for being detected based on imaging sensor determine the third relative position of user；Co-factor propagation unit, for merging the first, second, and third relative position to determine fourth relative position of the user relative to each loudspeaker；Delay compensation computing unit, for calculating the delay compensation value of each loudspeaker based on the 4th relative position；And delay adjusts unit, the phase of the audio signal for adjusting each loudspeaker according to delay compensation value.
Description
Technical field
Present invention relates in general to sound reproductions, more specifically it relates to which a kind of multi-channel audio system, can track use
Family, to persistently provide optimal hearing effect for user.
Background technique
Typical multi-channel audio system, including 2.0 systems, 2.1 systems, 5.1 systems, 6.1 systems, 7.1 systems etc.,
Its multiple sound channel be may be by provide stereophonic effect for user.However, in order to realize optimal hearing effect, for sound
Acoustic system is placed with certain requirement.
Figure 1A shows putting for 2.0 or 2.1 sound systems, and Figure 1B shows putting for 5.1 sound systems.For 2.0 or 2.1
System, as shown in Figure 1A, left channel loudspeaker (or speaker) 101 ', right channel loudspeaker 102 ' and 110 three of user are formed
Side triangle, and left channel loudspeaker 101 ' and 102 ' user oriented 110 of right channel loudspeaker.Because subwoofer
Directive property is not strong, so it, which is put, not to show in figure relatively flexibly.For 5.1 systems, as shown in Figure 1B, in addition to left front
Channel loudspeaker 101, right front channels loudspeaker 102 and 110 three of user are formed except equilateral triangle, center loudspeaker 103
It is placed in 110 front of user, and center loudspeaker 103 is equal to the side length of aforementioned equilateral triangle with a distance from user 110.This
Outside, left subsequent channel loudspeaker 104 and rear right channel loudspeaker 105 are respectively placed in the left back and right back of user 110.Equally,
Subwoofer is omitted in Figure 1B.Other than 2.0 systems, 2.1 systems and 5.1 systems, for 6.1 systems, 7.1 systems
Requirement is put accordingly Deng also having.When putting sound system according to such requirement, the position where user 110 is commonly referred to as
" emperor position ", i.e., most preferably listen phoneme, can obtain optimal audition experience in the position.
As it can be seen that needing to execute complicated position to sound system and put to realize best hearing effect.In addition, even if
Put loudspeaker as requested, emperor position region is also limited, when user is at emperor position region, or when user because
When moving and leaving emperor position, then still audio quality can be caused to be greatly lowered because of the interference between sound.
Application for a patent for invention CN201510112555.X proposes a kind of audio reproduction technology based on the synthesis of sound field wave,
Frequency dividing coding and uniform treatment are carried out to per audio signal all the way according to the frequency of audio and wavelength, and is closed after diffraction is handled
At output, to avoid the interference between different loudspeakers, to realize the improvement of the audio quality except emperor position.Although the party
Case can to a certain extent by weaken interference come improve emperor position region other than hearing effect, but its there are still several
Problem.Firstly, the frequency dividing coding, diffraction and synthesis processing etc. to audio signal inevitably will affect fidelity, cause
Audio is deteriorated.Although it still cannot in addition, the program can improve the audio of the exterior domain of emperor position to a certain extent
It realizes and the comparable hearing effect in emperor position.
Accordingly, it is desired to provide a kind of sound effect system, can neatly be put, and even if user in different location,
Such as the position except emperor position, optimal hearing effect can be also provided for user.
Summary of the invention
One aspect of the present invention is to provide a kind of multi-channel audio system, can track user, from regardless of with
Whether family can for user provide optimal audition impression in emperor position.
Another aspect of the present invention, which also resides in, provides a kind of multi-channel audio system, can adjust prolonging for each loudspeaker
Slow and gain, so as to flexibly be put to each loudspeaker.
According to an exemplary embodiment, a kind of multi-channel audio system may include multiple loudspeakers, imaging sensor, sound
Sensor and controller have scheduled first relative position, institute between described image sensor and the sound transducer
Stating controller includes: loudspeaker position detection unit, be configured to that the sound transducer detects from the multiple
The predetermined audio signal of loudspeaker determines second relative position of the multiple loudspeaker relative to the sound transducer；With
Family position detection unit is configured to the user images that described image sensor detects to determine user relative to the figure
As the third relative position of sensor；Co-factor propagation unit is configured to merge first relative position, the second opposite position
It sets and the third relative position, so that it is determined that fourth relative position of the user relative to each loudspeaker；Delay compensation meter
Unit is calculated, is configured to the 4th relative position to calculate the delay compensation value of the audio signal for each loudspeaker；
And delay adjusts unit, is configured to adjust the phase of the audio signal of each loudspeaker according to the delay compensation value.
In one example, one in the multiple loudspeaker is arranged in described image sensor and the sound transducer
On, to have scheduled second relative position between the sound transducer and one loudspeaker.
In one example, the sound transducer includes microphone array, and the loudspeaker position detection unit is configured to
Direction of each loudspeaker relative to the microphone array is determined based on auditory localization, and is issued based on each loudspeaker
Predetermined audio signal and the microphone array receive the time difference between the audio signal to determine each loudspeaker phase
For the distance of the microphone array.
In one example, the controller further include: gain compensation computing unit is configured to the described 4th opposite position
It sets to calculate the gain compensation value of the audio signal for each loudspeaker；And gain adjustment unit, it is configured to according to
Gain compensation value adjusts the gain of the audio signal of each loudspeaker.
Another exemplary embodiment of the invention provides a kind of equipment including above-mentioned multi-channel audio system.The equipment
It can be television set or automobile etc..
Another exemplary embodiment of the invention provides a kind of method that sound is reappeared using multi-channel audio system, institute
Stating multi-channel audio system includes multiple loudspeakers, which comprises receives what each loudspeaker issued from sound transducer
Predetermined audio signal, and determine based on received predetermined audio signal the second relative position of each loudspeaker；From figure
Picture sensor receives user images, and the third relative position of user is determined based on received user images；To described
Scheduled first opposite position between second relative position, the third relative position and sound transducer and imaging sensor
It sets and is merged, to determine fourth relative position of the user relative to each sensor；And it is based on the 4th relative position
It determines the delay compensation value of the audio signal for each loudspeaker, and adjusts each loudspeaking according to the delay compensation value
The phase of the audio signal of device.
In one example, one in the multiple loudspeaker is arranged in described image sensor and the sound transducer
On, to have scheduled second relative position between the sound transducer and one loudspeaker.
In one example, the second relative position for determining each loudspeaker includes: that each raise is determined based on auditory localization
Direction of the sound device relative to the sound transducer, and predetermined audio signal and sound biography are issued based on each loudspeaker
Sensor receives the time difference between the audio signal to determine distance of each loudspeaker relative to the sound transducer.
In one example, the method also includes: based on the 4th relative position determine be used for each loudspeaker sound
The gain compensation value of frequency signal, and adjust according to the gain compensation value gain of the audio signal of each loudspeaker.
Another exemplary embodiment of the invention provides a kind of computer program product, including computer program instructions, institute
State computer program instructions makes the processor execute the above method when being run by processor.
Detailed description of the invention
Figure 1A shows the schematic placement position figure of 2.0 or 2.1 traditional sound systems.
Figure 1B shows the schematic placement position figure of 5.1 traditional sound systems.
Fig. 2 shows the schematic diagrames of the multi-channel audio system of an exemplary embodiment according to the present invention.
Fig. 3 shows the functional block diagram of the controller of Fig. 2 of an exemplary embodiment according to the present invention.
Fig. 4 shows the flow chart of sound reproduction process according to an embodiment of the invention.
Fig. 5 shows the example of the equipment with sound system according to an embodiment of the invention.
Fig. 6 shows the example of another equipment with sound system according to an embodiment of the invention.
Specific embodiment
Exemplary embodiment of the present invention is described below with reference to accompanying drawings.
Fig. 2 shows the schematic diagrames of the multi-channel audio system 200 of an exemplary embodiment according to the present invention.Although Fig. 2 is only
Two loudspeakers are shown, but the principle of the present invention should be understood that also by following description and may be used on having more
The multi-channel audio system of loudspeaker, such as, but not limited to 2.1 systems, 5.1 systems, 6.1 systems, 7.1 systems etc..
As shown in Fig. 2, multi-channel audio system 200 includes the first loudspeaker 210 and the second loudspeaker 220.Typically, example
Such as in 2.0 systems, one in the first loudspeaker 210 and the second loudspeaker 220 can be left channel loudspeaker, another can
Think right channel loudspeaker.Two loudspeakers 210 and 220 may be coupled to controller 230 to be controlled by.Although Fig. 2 shows
Controller 230 is embodied as individual unit, it will be understood that, controller 230 also can be implemented in multi-channel audio system 200
In any one loudspeaker, including subwoofer (if any), or it can also be arranged in and be connected with loudspeaker
In equipment, power amplifier, computer etc..
With continued reference to Fig. 2, multi-channel audio system 200 further includes sound transducer 201 and imaging sensor 202.Sound
Sensor 201 is preferably microphone array comprising the multiple microphones arranged by predetermined pattern.Imaging sensor 202 can be with
It is monocular cam, binocular camera or more mesh cameras etc..Preferably, imaging sensor 202 can be binocular camera shooting
Head.
In the embodiment shown in Figure 2, sound transducer 201 and imaging sensor 202 have been all disposed within the first loudspeaker
On 210, therefore for convenience, the first loudspeaker 210 can be known as main loudspeaker, by other loudspeakers, such as second
Loudspeaker 220, referred to as secondary loudspeaker.However, it is understood that main loudspeaker can be any one of multi-channel audio system 200
Loudspeaker, including but not limited to subwoofer.Preferably, main loudspeaker can be the loudspeaking for being able to easily see user
Device.It should also be understood that sound transducer 201 and imaging sensor 202 also may be provided in other equipment, such as it is embodied as list
Only module.Sound transducer 201 and imaging sensor 202 are preferably provided in same equipment or module, so that therebetween
With known predetermined phase position.As being understood that by following description, from being convenient for for raising level of integrated system
For the angle used, it is preferred that sound transducer 201 and imaging sensor 202, which are arranged on some loudspeaker,.
Fig. 3 shows the functional block diagram of the controller 230 of Fig. 2 of an exemplary embodiment according to the present invention.As shown in figure 3,
Controller 230 may include loudspeaker position detection unit 231, user location detection unit 232, Co-factor propagation unit 233, delay
Compensation calculation unit 234, gain compensation computing unit 235, delay adjust unit 236 and gain adjustment unit 237.It below will ginseng
The operation of all parts in controller 230 is described according to Fig. 2 and 3.
Loudspeaker position detection unit 231 can be used for determining the position of each loudspeaker of sound system 200, it is specific and
Speech, position of the loudspeaker relative to sound transducer microphone array 201 in other words, including distance and direction.Specifically, such as
Shown in Fig. 2, each loudspeaker 210,220 of sound system 200 can be controlled to issue scheduled audio signal, such as in advance
Determine the audio signal of frequency and waveform, and sound transducer 201, such as microphone array, can receive each loudspeaker hair
The audio signal out.Loudspeaker position detection unit 231 can be handled the audio signal that microphone array receives,
Direction of each loudspeaker of sound source relative to microphone array is determined as based on auditory localization technology.Based on microphone array
The sound localization method of column includes steerable beam formation technology, High-Resolution Spectral Estimation technology, based on sound wave reaching time-difference
Technology etc. is used equally for the direction in the present invention with each loudspeaker of determination relative to microphone array.To be reached based on sound wave
For the technology of time difference, principle is to determine that the sound wave from sound source reaches the time difference of a pair of of microphone, corresponds to one
A hyperboloid can determine multiple hyperboloids using multipair microphone, their intersection is exactly the position that sound source is located at.This hair
Bright people's discovery, especially for far field system, based on the error for the sound source distance that microphone array and sound source location technology determine
It is larger, therefore in an embodiment of the present invention, determine each loudspeaker relative to microphone array merely with auditory localization technology
The direction of column.
In addition, loudspeaker position detection unit 231 is also connect by determining that loudspeaker issues audio signal with microphone array
The time difference of audio signal is received, to determine distance of the loudspeaker away from microphone array.It is electric in loudspeaker and microphone array
The speed of signal can consider much larger than the velocity of sound, therefore can be readily determined time and Mike that loudspeaker issues audio signal
Wind array received to audio signal time, the difference of the two multiplied by acoustic speed, can be obtained loudspeaker away from microphone array it
Between distance.To which loudspeaker position detection unit 231 can determine position of each loudspeaker relative to microphone array 201
It sets, including direction and distance.
It should be noted that when as shown in Fig. 2, microphone array 201 is arranged on one of loudspeaker such as main loudspeaker 210,
Main loudspeaker 210 can be relative to the position of microphone array it is fixed and known, it will therefore be appreciated that being raised in above-mentioned utilization
When sound device position detection unit 231 determines the position of each loudspeaker, it is convenient to omit the step related to main loudspeaker 210.
User location detection unit 232 can be used for detecting the position of user, specifically, user is relative to imaging sensor
202 position, including direction and distance.As shown in Fig. 2, imaging sensor 202, preferably binocular camera, can capture use
The image at family 110.User location detection unit 232 can receive the image data from imaging sensor 202, and utilize image
Identification technology, such as recognition of face, head-shoulder contour identification, human body contour outline identification etc. identify user, and further basis
Image information determines direction and distance where user.In some special circumstances, such as when identifying multiple users, user
The middle position of such as multiple users can be determined as user location by position detection unit 232, or only be identified according to face
Register the position of user；When not identifying any user, user location detection unit 232 can be using default location as use
Family position, etc..These strategies can be determined according to actual use scene.
As described above, when loudspeaker position detection unit 231 has determined each loudspeaker relative to sound transducer 201
Position (hereinafter referred to as the second relative position), and user location detection unit 232 has determined user relative to imaging sensor
After 202 position (hereinafter referred to as third relative position), Co-factor propagation unit 233 can melt these position datas
It closes.Specifically, known to Co-factor propagation unit 233 between sound transducer 201 and imaging sensor 202 relative position (under
Text be known as the first relative position), such as sound transducer 201 and imaging sensor 202 can be implemented in the same module or
Implement on same loudspeaker, therefore first relative position is known and fixed.Therefore, it is based on sound transducer 201
The the second opposite position of the first relative position, each loudspeaker relative to sound transducer 201 between imaging sensor 202
It sets and third relative position of the user relative to imaging sensor 202, Co-factor propagation unit 233 can determine that user is opposite
It is raised in the relative position (hereinafter referred to as the 4th relative position) of each loudspeaker, such as user shown in Fig. 2 110 relative to first
Distance R2 and direction b of the distance R1 and direction a and user 110 of sound device 210 relative to the second loudspeaker 220.In addition, position
Relative distance and direction between each loudspeaker, such as the first loudspeaker 210 and second can also be determined by setting integrated unit 233
The distance between loudspeaker 220 R3 and direction c.In a further exemplary embodiment, Co-factor propagation unit 233 can also be determined only
Distance of the user relative to each loudspeaker, i.e. the 4th relative position only include distance of the user relative to each loudspeaker.
It, can be by delay compensation computing unit 234 and gain compensation computing unit after the 4th relative position has been determined
235 calculate separately the delay compensation value and gain compensation value of the audio signal for each loudspeaker.As shown in Fig. 2, user 110
The distance R1 of the first loudspeaker of distance 210 may be different from the distance R2 of the second loudspeaker of user distance 220, lead to two loudspeakings
The sound wave that device is simultaneously emitted by arrives at user in different times, and the sound wave for arriving at user have due to distance difference it is different
Strength retrogression causes the audition of user to be experienced and is deteriorated.In an embodiment of the present invention, made by delay compensation and gain compensation
The sound wave for obtaining different loudspeakers reaches user and gain having the same simultaneously.
In one example, delay compensation computing unit 234 can obtain distance of each loudspeaker relative to user, determine
Maximum distance therein, then adjust each loudspeaker delay so that all loudspeakers sound wave arrive at user time with
Maximum distance loudspeaker is identical.For example, for the example of Fig. 2, distance R2 of second loudspeaker 220 apart from user 110 is greater than the
Distance R1 of one loudspeaker 210 apart from user 110, then delay compensation computing unit 234 can be calculated according to following formula 1
Delay compensation value for the first loudspeaker 210.
Δ=ROUND (Fs (R2-R1)/v) formula 1
Wherein, v indicates the velocity of sound, and Fs indicates sample frequency, and ROUND is the function for being rounded up to nearest integer.From public affairs
Formula 1 is as can be seen that Δ indicates that the sound wave of the second loudspeaker 220 arrives at sound wave of the time than the first loudspeaker 210 of user 110
Arrive at the sampling number of the time delay of user 110.
Then, delay adjusts unit 236 can adjust the sound of each loudspeaker according to delay compensation value Δ calculated
The phase of frequency signal.For example, for the example of Fig. 2, then by the phase delay delta of the audio signal of the first loudspeaker 210, so that
The audio signal of itself and the second loudspeaker 210 arrives at user simultaneously.For the second loudspeaker 220, then because it has the latest
The sound wave arrival time does not have to adjustment then.
In addition, gain compensation computing unit 235 can obtain distance of each loudspeaker relative to user, determine therein
Then minimum range adjusts the gain of each loudspeaker so that intensity each other one when the sound wave of all loudspeakers arrives at user
The sound wave for causing, and corresponding to the loudspeaker of minimum range arrives at intensity when user.For example, first raises for the example of Fig. 2
Distance R2 of distance R1 of the sound device 210 apart from user 110 less than the second loudspeaker 220 apart from user 110, then gain compensation meter
The gain compensation value for the second loudspeaker 220 can be calculated according to following formula 2 by calculating unit 235.
G=ea(R2-R1)Formula 2
Wherein, a is the aerial attenuation coefficient of sound wave.Then, gain adjustment unit 236 can be according to increasing calculated
Beneficial offset g adjusts the gain of the audio signal of each loudspeaker.For example, for the example of Fig. 2, then by the second loudspeaker
220 gain is increased to g times of former yield value.For the first loudspeaker 210 with the shortest distance, then because being calculated
Gain compensation value g is equal to one, so its gain is constant.
As shown in figure 3, the audio signal R for the audio signal L of the first loudspeaker 210 and for the second loudspeaker 220
After delay adjustment and gain adjustment, it is sent to corresponding loudspeaker and is converted into acoustic signals.Although giving above
The specific example for having gone out delay adjustment and gain adjustment, however, it is understood that those skilled in the art can also use other modes
It is adjusted, as long as the sound wave for issuing each loudspeaker finally arrives at user with the roughly the same time and arrives at use
There is corresponding intensity, the variation of all these adjustment modes should all be fallen within the scope of the invention when family.
In Fig. 2 and embodiment shown in Fig. 3, each raise is identified by the audio signal issued using each loudspeaker
The relative position of sound device, and the position of user is determined by image recognition.This is because the inventors discovered that, although can be with
Each loudspeaker and the position of user are easily determined by image recognition, but in most application scenarios, very
Difficulty captures the position of each loudspeaker and user by a camera, and is mounted on different location, tool when using
When there are two cameras in the different visuals field, the complexity of system, such as the complexity of installation, wiring etc. can be greatly increased.
Therefore, the present inventor determines user location merely with image recognition, and the position of each loudspeaker is determined using voice signal
It sets, then carries out the position of each loudspeaker and user by the known relative position between imaging sensor and sound transducer
Fusion so as to easily solve foregoing problems the solution of the present invention is easily applied.
In addition, the present invention adjusts the delay and gain of each loudspeaker by tracking the position of user, no matter therefore with
At what position, or even when user is in movement, which can track the position of user at family, and be user
Optimal audition experience is provided.In addition, the solution of the present invention be due to that can be adjusted the audio signal of each loudspeaker, institute
It is insensitive with the position to loudspeaker, each loudspeaker is neatly put so as to be adapted to space, reduces or avoids
The damage due to limitation of space to audition experience.
Fig. 4 shows the flow chart of sound reproduction process 300 according to an embodiment of the invention.Below in conjunction with Fig. 1 to Fig. 4
To illustrate sound reproduction process 300.
Firstly, can control each loudspeaker in step S301 to issue scheduled audio signal.In an embodiment
In, for example, each loudspeaker can successively issue identical or different frequency, the audio signal of waveform.Due to electrical signal form
Speed of the audio signal in the circuit of sound system can consider much larger than the velocity of sound, therefore in this step can also be easily
Determine that each loudspeaker issues the time of the audio signal (audio signal of electric form in other words, is converted to acoustic signals).
Meanwhile microphone array 201 can receive or detect the predetermined audio signal that each loudspeaker issues.
In step S302, the loudspeaker position detection unit 231 of controller 230 can use the detection of microphone array 201
To each loudspeaker audio signal for issuing determine the position of each loudspeaker.For example, fixed using previously described sound source
Position technology, can determine direction of each loudspeaker relative to microphone array；It is made a sound using loudspeaker and microphone array
Column detect the time difference between sound, distance of each loudspeaker relative to microphone array can be determined, it is possible thereby to really
Fixed position of each loudspeaker relative to microphone array, i.e., previously described second relative position.
In addition, can use imaging sensor 202, such as binocular camera in step S303, to capture environment picture number
According to.In step s 304, the image data that imaging sensor 202 captures can be by the user location detection unit of controller 230
232 processing, such as identify that user captures the user in image by image recognition technology；And if recognizing user,
Further determine that position of the user relative to imaging sensor 202, i.e., previously described third relative position.When not detecting
When any user, in one example, user location detection unit 232 can not export third station-keeping data, so that control
Device 230 does not carry out any adjusting to each loudspeaker, or in another example, user location detection unit 232 can export
The third station-keeping data of default, so that controller 230 is adjusted each loudspeaker using the default data.Work as inspection
When measuring multiple users, in one example, user location detection unit 232 can detecte the position of each user, then export
Such as the data of the center of these positions are as third relative position, or can export in that intermediate user's
Position data.
It should be understood that since fixation is generally compared in the position of loudspeaker, and user may move, so detection loudspeaking
The execution frequency of step S301, S302 of device position can be less than the execution frequency of step S303, S304 of detection user location.
For example, the relative position of a loudspeaker can be only detected when sound system 200 starts, and with predetermined frequency in system operation
Rate repeatedly detects the relative position of user.In addition, when not detecting the relative position of loudspeaker, sound transducer 201 can be with
It is turned off to save power consumption.
Next, Co-factor propagation unit 233 can be with execution position mixing operation in step S305.Specifically, position
Integrated unit 233 can receive the second relative position and third relative position determined in abovementioned steps, and using known to it
Sound transducer 201 and imaging sensor 202 between relative position (i.e. the first relative position), come determine user relative to
The relative position of each loudspeaker is referred to as the 4th relative position.In one embodiment, Co-factor propagation unit 233 is at least true
Determine distance of the user 110 relative to each loudspeaker.
Based on the 4th relative position, i.e. distance of the user relative to each loudspeaker, delay and gain adjustment can be carried out.
For example, calculating the delay compensation value of each loudspeaker, and calculate each loudspeaker in step S307 in step S306
Then gain compensation value carries out in step S308 according to audio signal of the offset calculated to each loudspeaker corresponding
It adjusts.Description is not repeated herein above in relation to having been described in the description of Fig. 3 in these steps.
It should be understood that in some embodiments, it is also possible to being based only upon user relative to the position of loudspeaker to adjust each raise
The delay of the audio signal of sound device；In some embodiments, it is also possible to both adjustment delay and gain as described above.
Fig. 5 shows the example of the equipment with sound system according to an embodiment of the invention.Equipment can be vehicle
400, such as domestic sedan car, with sound system 410.As shown, sound system 410 may include multiple loudspeakers
401,402,403,404,405 and 406, wherein loudspeaker 401 and 402 can be mounted in the preposition high pitch loudspeaker in front,
Loudspeaker 403 and 404 can be mounted in middle part and set woofer, and loudspeaker 405 and 406 can be mounted in rear portion
Postposition high pitch loudspeaker.It should be understood that the quantity of the loudspeaker of sound system 410, arrangement etc. are not limited to example shown in the drawings.
Sound system 410 may also include for example more mesh of sound transducer such as microphone array 407 and imaging sensor and take the photograph
As head 408 and controller 409, they and previously described sound transducer 201, imaging sensor 202 and controller 230
It is similar, description is not repeated herein.
In the vehicle 400 of Fig. 5, although each loudspeaker 401 to 406 is installed in fixed position, due to vehicle
Interior space is irregular, and the quantity of general vehicle-mounted loudspeaker is more, it is thus possible to be difficult to obtain each loudspeaking by measuring
The accurate location of device.In one embodiment, it can use sound transducer 407, method as described previously determines each like that
Position of the loudspeaker relative to sound transducer 407, due to the position be it is fixed, can only be surveyed after loudspeaker is installed
Amount is primary, and then measurement result is stored in for example vehicle-mounted memory (not shown).
Imaging sensor 408 for determining the position of user as described previously.In one embodiment, when in addition to driving
There is also when other users (user i.e. on passenger-seat) except the person of sailing (user i.e. on driver's seat), will can only drive
Member is identified as user, when driver is not present, then the passenger of other positions can be identified as user, then controller 409
Loudspeaker adjusting appropriate is carried out for the user identified.
In further embodiments, other than identifying user with camera, it also can receive the specified of user's input
Position.For example, user can specify certain interior position as user location by liquid crystal display (not shown), then control
Device 409 can carry out adjusting appropriate to each loudspeaker according to specified position.
By using sound system of the invention, have when user is on different seats, or because adjusting seat position
When mobile, it can obtain optimal audition and enjoy.
Fig. 6 shows the example of another equipment 500 with sound system according to an embodiment of the invention.Equipment 500 can
To be such as television set, with left channel loudspeaker 501, right channel loudspeaker 502, sound transducer 503, imaging sensor
504 and controller 505, they can be similar with element 210 shown in Fig. 2,220,201,202 and 230 respectively, this
In no longer repeat one by one.
Unlike previously described embodiment, in television set 500, the position of two loudspeakers 501,502 is fixed
And it is easier measurement or is determined in design, and sound transducer 503 and imaging sensor 504 are also all disposed within electricity
Depending on the shell of machine, therefore in some embodiments, it is convenient to omit the step of the position of the previously described each loudspeaker of determination,
But the position of user is only identified by imaging sensor, previously described delay and gain then are carried out to each loudspeaker
It adjusts.
In further embodiments, television set 500 may be connected to external loudspeaker by audio interface and (not shown
Out).At this point, then can according to previously described method, to determine position of the external loudspeaker relative to sound transducer 503,
Delay and gain adjustment are carried out to external loudspeaker in conjunction with user location in turn.
The principle of the present invention or embodiment are also implemented as computer program product, are stored in computer-readable
In medium, and including computer program instructions.These computer program instructions can execute front when being run by processor
The method of description.These computer program products can also pass through transmission on Internet or distribution.For example, the computer program can be with
It is audio driven or control program, can be installed on the electronic equipments such as notebook, mobile phone, plate, these equipment itself
Just there is imaging sensor and sound transducer.Therefore, when these electronic equipments are connected to boom box system, then may be used
To execute those of previously mentioned method, to provide optimal audition experience for user.
Exemplary embodiment is combined to describe basic principle of the invention above, it should be noted however that in the application
In the advantages of referring to, advantage, effect etc. be only exemplary rather than limitation, must not believe that these advantages, advantage, effect etc. are the application
Each embodiment must be provided simultaneously with.In addition, detail disclosed above is merely to exemplary act on and convenient for reason
The effect of solution, rather than limit, above-mentioned details does not limit the invention to realize using above-mentioned concrete details.
Device involved in the present invention, device, equipment, system block diagram only as illustrative example and be not intended to
It is required that or hint must be attached in such a way that box illustrates, arrange, configure.As those skilled in the art will appreciate that
, it can be connected by any way, arrange, configure these devices, device, equipment, system.Shown in flow chart provided by the present application
Method be not limited to the declaration order of flow chart, but, multiple steps can execute in a different order or two or
More multi-step may be performed simultaneously, what these variations will be apparent to those skilled in the art.Such as " comprising ", " packet
Containing ", the word of " having " etc. be open vocabulary, refer to " including but not limited to ", and can be used interchangeably with it.It is used here
Vocabulary "or" and "and" refer to vocabulary "and/or", and can be used interchangeably with it, unless it is not such that context, which is explicitly indicated,.This
In used vocabulary " such as " refer to phrase " such as, but not limited to ", and can be used interchangeably with it.
It may also be noted that in the device and method of the application, each component or each step are can to decompose and/or again
Combination nova.These decompose and/or reconfigure the equivalent scheme that should be regarded as the application.
The above description of disclosed aspect is provided so that any person skilled in the art can make or use this
Application.Various modifications in terms of these are readily apparent to those skilled in the art, and are defined herein
General Principle can be applied to other aspect without departing from scope of the present application.Therefore, the application is not intended to be limited to
Aspect shown in this, but according to principle disclosed herein and the consistent widest range of novel feature.
In order to which purpose of illustration and description has been presented for above description.In addition, this description is not intended to the reality of the application
It applies example and is restricted to form disclosed herein.Although already discussed above multiple exemplary aspects and embodiment, this field skill
Its certain modifications, modification, change, addition and sub-portfolio will be recognized in art personnel.
Claims (7)
1. a kind of multi-channel audio system, including multiple loudspeakers (210,220), imaging sensor (202), sound transducer
(201) and controller (230), there is between described image sensor and the sound transducer the scheduled first opposite position
It sets, the sound transducer includes microphone array, and the controller includes:
Loudspeaker position detection unit (231) is configured to auditory localization to determine each loudspeaker relative to the Mike
The direction of wind array, and described make a reservation for is received based on each loudspeaker sending predetermined audio signal and the microphone array
Time difference between audio signal determines distance of each loudspeaker relative to the microphone array, so that it is determined that described more
Second relative position of a loudspeaker relative to the microphone array；
User location detection unit (232) is configured to the user images that described image sensor detects to determine user
Third relative position relative to described image sensor；
Co-factor propagation unit (233) is configured to merge first relative position, second relative position and described
Three relative positions, so that it is determined that fourth relative position of the user relative to each loudspeaker；
Delay compensation computing unit (234) is configured to the 4th relative position to calculate the sound for each loudspeaker
The delay compensation value of frequency signal；And
Delay adjusts unit (236), is configured to adjust the phase of the audio signal of each loudspeaker according to the delay compensation value
Position,
Wherein, described image sensor and the microphone array are arranged on one in the multiple loudspeaker, thus institute
Stating has scheduled second relative position between microphone array and one loudspeaker.
2. multi-channel audio system as described in claim 1, wherein the controller further include:
Gain compensation computing unit (235) is configured to the 4th relative position to calculate the sound for each loudspeaker
The gain compensation value of frequency signal；And
Gain adjustment unit (237) is configured to adjust the increasing of the audio signal of each loudspeaker according to the gain compensation value
Benefit.
3. a kind of equipment including multi-channel audio system described in any one of claim 1 to 2.
4. equipment as claimed in claim 3, wherein the equipment is television set or automobile.
5. a kind of method for reappearing sound using multi-channel audio system, the multi-channel audio system includes multiple loudspeakings
Device, which comprises
Direction of each loudspeaker relative to the microphone array is determined based on auditory localization using microphone array, and
Based on each loudspeaker issue predetermined audio signal and the microphone array receive between the predetermined audio signal when
Between difference determine distance of each loudspeaker relative to the microphone array, so that it is determined that each loudspeaker is relative to the wheat
Second relative position of gram wind array；
User images are received from imaging sensor, and determine user relative to described image based on received user images
The third relative position of sensor；
To second relative position, the third relative position and the microphone array and described image sensor it
Between scheduled first relative position merged, to determine fourth relative position of the user relative to each sensor；And
The delay compensation value of the audio signal for each loudspeaker is determined based on the 4th relative position, and according to described
Delay compensation value adjusts the phase of the audio signal of each loudspeaker,
Wherein, described image sensor and the microphone array are arranged on one in the multiple loudspeaker, thus institute
Stating has scheduled second relative position between microphone array and one loudspeaker.
6. method as claimed in claim 5, further includes:
The gain compensation value of the audio signal for each loudspeaker is determined based on the 4th relative position, and according to described
Gain compensation value adjusts the gain of the audio signal of each loudspeaker.
7. a kind of computer-readable medium, computer program instructions, the computer journey are stored in the computer-readable medium
Sequence instruction makes method described in any one of described processor perform claim requirement 5 to 6 when being run by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710001369.8A CN106686520B (en) | 2017-01-03 | 2017-01-03 | The multi-channel audio system of user and the equipment including it can be tracked |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710001369.8A CN106686520B (en) | 2017-01-03 | 2017-01-03 | The multi-channel audio system of user and the equipment including it can be tracked |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106686520A CN106686520A (en) | 2017-05-17 |
| CN106686520B true CN106686520B (en) | 2019-04-02 |
Family
ID=58848857
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710001369.8A Active CN106686520B (en) | 2017-01-03 | 2017-01-03 | The multi-channel audio system of user and the equipment including it can be tracked |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106686520B (en) |
Families Citing this family (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10334358B2 (en) * | 2017-06-08 | 2019-06-25 | Dts, Inc. | Correcting for a latency of a speaker |
| WO2019041178A1 (en) * | 2017-08-30 | 2019-03-07 | 深圳魔耳智能声学科技有限公司 | Sound playback method and device, and readable storage medium |
| US10421422B2 (en) * | 2017-09-08 | 2019-09-24 | Harman International Industries, Incorporated | Sound tuning based on adjustable seat positioning |
| CN107888857A (en) * | 2017-11-17 | 2018-04-06 | 青岛海信电器股份有限公司 | For the method for adjustment of sound field, device and separate television in separate television |
| CN108055633A (en) * | 2017-12-06 | 2018-05-18 | 维沃移动通信有限公司 | A kind of audio frequency playing method and mobile terminal |
| CN107920307A (en) * | 2017-12-07 | 2018-04-17 | 北海威德电子科技有限公司 | A kind of device and method of sound equipment adjust automatically sound output phase |
| CN108347673A (en) * | 2018-03-30 | 2018-07-31 | 上海与德科技有限公司 | A kind of control method of intelligent sound box, device, storage medium and intelligent sound box |
| BR112020020279A2 (en) | 2018-04-09 | 2021-01-12 | Sony Corporation | APPARATUS AND METHOD OF PROCESSING INFORMATION, AND, PROGRAM. |
| CN108495234B (en) * | 2018-04-19 | 2020-01-07 | 北京微播视界科技有限公司 | Multi-channel audio processing method, apparatus and computer-readable storage medium |
| CN108551610A (en) * | 2018-05-11 | 2018-09-18 | 四川斐讯信息技术有限公司 | A kind of intelligent sound box and its audio effect generating method |
| CN109495800B (en) * | 2018-10-26 | 2021-01-05 | 成都佳发安泰教育科技股份有限公司 | Audio dynamic acquisition system and method |
| TW202021377A (en) * | 2018-11-23 | 2020-06-01 | 廣達電腦股份有限公司 | Environmental detection system and sound control method using the same |
| CN111314821A (en) * | 2018-12-12 | 2020-06-19 | 深圳市冠旭电子股份有限公司 | Intelligent sound box playing method and device and intelligent sound box |
| DE102018222094A1 (en) * | 2018-12-18 | 2020-06-18 | Audi Ag | Automatic sound system |
| CN109862503B (en) * | 2019-01-30 | 2021-02-23 | 北京雷石天地电子技术有限公司 | Method and equipment for automatically adjusting loudspeaker delay |
| CN111757241B (en) * | 2019-03-11 | 2022-01-04 | 深圳市冠旭电子股份有限公司 | Sound effect control method and device, sound box array and wearable device |
| CN110049429A (en) * | 2019-05-10 | 2019-07-23 | 苏州静声泰科技有限公司 | A kind of trailing type dynamic solid sound system for audio-visual equipment |
| CN112118527A (en) * | 2019-06-19 | 2020-12-22 | 华为技术有限公司 | Multimedia information processing method, device and storage medium |
| CN110996238B (en) * | 2019-12-17 | 2022-02-01 | 杨伟锋 | Binaural synchronous signal processing hearing aid system and method |
| CN113709631B (en) * | 2020-05-22 | 2023-01-17 | 宏碁股份有限公司 | Surround sound system and method for applying surround sound technology to electronic contest seats |
| CN113608641B (en) * | 2020-06-18 | 2024-01-16 | 深圳市冠旭电子股份有限公司 | Method and device for adjusting display position of curved screen, intelligent sound box and storage medium |
| CN112153525B (en) * | 2020-08-11 | 2022-09-16 | 广东声音科技有限公司 | Positioning method and system for multi-loudspeaker panoramic sound effect |
| CN112083379B (en) * | 2020-09-09 | 2023-10-20 | 极米科技股份有限公司 | Audio playing method and device based on sound source localization, projection equipment and medium |
| CN112383655B (en) * | 2020-11-02 | 2022-07-12 | Oppo广东移动通信有限公司 | Electronic device, sound enhancement method for electronic device, and storage medium |
| CN112492455B (en) * | 2020-11-27 | 2022-05-27 | 广东美的制冷设备有限公司 | Control method and operation control device of electronic equipment and electronic equipment |
| US11895466B2 (en) | 2020-12-28 | 2024-02-06 | Hansong (Nanjing) Technology Ltd. | Methods and systems for determining parameters of audio devices |
| CN113207061A (en) * | 2020-12-28 | 2021-08-03 | 汉桑（南京）科技有限公司 | Sound parameter determination method and system |
| CN113542984B (en) * | 2021-06-09 | 2022-04-15 | 荣耀终端有限公司 | Stereo sound implementation system, method, electronic device, and storage medium |
| CN113852905A (en) * | 2021-09-24 | 2021-12-28 | 联想(北京)有限公司 | Control method and control device |
| CN115426589A (en) * | 2022-11-03 | 2022-12-02 | 荣耀终端有限公司 | Audio signal adjusting method and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547533A (en) * | 2010-11-05 | 2012-07-04 | 索尼公司 | Acoustic control apparatus and acoustic control method |
| CN102572676A (en) * | 2012-01-16 | 2012-07-11 | 华南理工大学 | Real-time rendering method for virtual auditory environment |
| CN104581543A (en) * | 2013-10-10 | 2015-04-29 | 三星电子株式会社 | Audio system, method of outputting audio, and speaker apparatus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9560445B2 (en) * | 2014-01-18 | 2017-01-31 | Microsoft Technology Licensing, Llc | Enhanced spatial impression for home audio |
-
2017
- 2017-01-03 CN CN201710001369.8A patent/CN106686520B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547533A (en) * | 2010-11-05 | 2012-07-04 | 索尼公司 | Acoustic control apparatus and acoustic control method |
| CN102572676A (en) * | 2012-01-16 | 2012-07-11 | 华南理工大学 | Real-time rendering method for virtual auditory environment |
| CN104581543A (en) * | 2013-10-10 | 2015-04-29 | 三星电子株式会社 | Audio system, method of outputting audio, and speaker apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106686520A (en) | 2017-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106686520B (en) | The multi-channel audio system of user and the equipment including it can be tracked | |
| CN109286875B (en) | Method, apparatus, electronic device and storage medium for directional sound pickup | |
| US8204248B2 (en) | Acoustic localization of a speaker | |
| EP2984852B1 (en) | Method and apparatus for recording spatial audio | |
| EP2320676A1 (en) | Method, communication device and communication system for controlling sound focusing | |
| US9002027B2 (en) | Space-time noise reduction system for use in a vehicle and method of forming same | |
| US20110058676A1 (en) | Systems, methods, apparatus, and computer-readable media for dereverberation of multichannel signal | |
| US20140294210A1 (en) | Systems, methods, and apparatus for directing sound in a vehicle | |
| US8615392B1 (en) | Systems and methods for producing an acoustic field having a target spatial pattern | |
| Ryan et al. | Application of near-field optimum microphone arrays to hands-free mobile telephony | |
| CN115052231B (en) | Digitized vehicle-mounted personal follow-up ultrasonic directional sound field system | |
| WO2020120754A1 (en) | Audio processing device, audio processing method and computer program thereof | |
| CN110830870B (en) | Earphone wearer voice activity detection system based on microphone technology | |
| EP3467818B1 (en) | Locally attenuated sound field forming device, corresponding method and computer program | |
| JP2003264883A (en) | Voice processing apparatus and voice processing method | |
| CN114464184B (en) | Method, apparatus and storage medium for speech recognition | |
| Scharrer et al. | Sound field classification in small microphone arrays using spatial coherences | |
| US11769486B2 (en) | System and method for data augmentation and speech processing in dynamic acoustic environments | |
| US11721353B2 (en) | Spatial audio wind noise detection | |
| US20220262342A1 (en) | System and method for data augmentation and speech processing in dynamic acoustic environments | |
| Shon et al. | Sudden noise source localization system for intelligent automobile application with acoustic sensors | |
| CN112188358A (en) | Audio signal processing apparatus, audio signal processing method, and non-volatile computer-readable recording medium | |
| JP2001337694A (en) | Method for presuming speech source position, method for recognizing speech, and method for emphasizing speech | |
| Saremi | Spatial audio signal processing for speech telecommunication inside vehicles | |
| US11671752B2 (en) | Audio zoom |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |