CN110488225A - Indicating means, device, readable storage medium storing program for executing and the mobile terminal of sound bearing - Google Patents
Indicating means, device, readable storage medium storing program for executing and the mobile terminal of sound bearing Download PDFInfo
- Publication number
- CN110488225A CN110488225A CN201910985754.XA CN201910985754A CN110488225A CN 110488225 A CN110488225 A CN 110488225A CN 201910985754 A CN201910985754 A CN 201910985754A CN 110488225 A CN110488225 A CN 110488225A
- Authority
- CN
- China
- Prior art keywords
- sound
- data
- audio
- target
- mobile terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S1/00—Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith
- G01S1/72—Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith using ultrasonic, sonic or infrasonic waves
- G01S1/76—Systems for determining direction or position line
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
- G01S5/20—Position of source determined by a plurality of spaced direction-finders
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Stereophonic System (AREA)
Abstract
A kind of indicating means of sound bearing, device, readable storage medium storing program for executing and mobile terminal, this method is applied in mobile terminal, the at least one side edge of the mobile terminal is provided with multiple groups light bar, with the user-center in interface, the direction that user receives sound is divided into multiple angular ranges, the light bar and the angular range are in corresponding relationship, and the indicating means includes: the audio data of two-channel in acquisition system, and feature extraction is carried out to the audio data, obtain audio characteristic data；By neural network model judge extract audio characteristic data whether be target sound data；If so, calculating the Sounnd source direction of the target sound according to the audio characteristic data；It determines angular range belonging to the Sounnd source direction, and controls the corresponding target lamp band of the angular range and light.The embodiment of the present invention is supplied to the orientation of user's sound source by way of visual alerts, as preliminary judgement and to remind, to promote user experience.
Description
Technical field
The present invention relates to electronic technology fields, more particularly to the indicating means, device, readable storage of a kind of sound bearing
Medium and mobile terminal.
Background technique
" listen and argue position " is one of the basic training of victory or defeat of concerning, player for FPS (first person shooting) class game player
It needs before not seeing enemy player, judges the position of enemy and we player, ability in advance according to shot, footsteps, vehicle sound etc.
Correct decisions gain the initiative by striking first and win.The Principles of Acoustics of " listen and argue position " in reality, the left and right ear of people is due to auricle knot
Structure and position difference, the information of a certain simple sound source received by the ear of left and right is can different, especially sound wave arrival time meeting
There is nuance, human brain estimates sound bearing by the calculating to dual channel data difference.For this purpose, game engine is often
Offer at least dual channel data allows player that can accomplish " listen and argue position " to simulate acoustic information in reality.
But when audio output device or environment is limited or player's hearing is by limited time, it is difficult to be accomplished to listen sound according to sound
Debate position.Especially in the game of mobile terminal, since mobile phone speaker is generally difficult to carry out two-channel function, and many times player
Loudspeaker can be used, or even lowers game volume due to the mute requirement of environment, player in gaming can not be direct at this time
Accomplish " listen and argue position ", game experiencing is poor.
Although the game of partial movement end provides the prompt of UI to a certain extent as auxiliary for player in response to this
It helps, such as position of the shot in map can be prompted in " peaceful X English " in small map, but this prompt is by game developer
Self-defining, it is widely different in different game or do not have, and effect is prompted not enough obviously to be difficult to rapid identification orientation.
Summary of the invention
In view of the above situation, it is necessary to for being often difficult to distinguish sound source by listening in game sound in the prior art
The problem of direction, provides indicating means, device, readable storage medium storing program for executing and the mobile terminal of a kind of sound bearing.
A kind of indicating means of sound bearing is applied in mobile terminal, and at least one side edge of the mobile terminal is arranged
There is multiple groups light bar, centered on the user's head position in the interface of mobile terminal, the direction that user receives sound is divided
For multiple angular ranges, the multiple groups light bar and the multiple angular range are in corresponding relationship, and the indicating means includes:
The audio data of two-channel in acquisition system, and feature extraction is carried out to the audio data, obtain audio characteristic data；
By neural network model judge extract audio characteristic data whether be target sound data；
If so, calculating the Sounnd source direction of the target sound according to the audio characteristic data；
It determines angular range belonging to the Sounnd source direction, and controls the corresponding target lamp band of the angular range and light.
Further, the indicating means in above sound orientation, wherein the sound extracted by neural network model judgement
The step of whether frequency characteristic is the data of target sound include:
To the audio characteristic data in each sound channel, successively the data of every three sampled points take a mean value, obtain one section
New audio data.
The mean value computation that the new audio data is carried out to left and right acoustic channels in sampled point each time, to be mixed into monophone
Audio data；
The monaural audio data is calculated by neural network model, to determine whether for target sound.
Further, the indicating means in above sound orientation, wherein the sound extracted by neural network model judgement
The step of whether frequency characteristic is the data of target sound further include:
The type of the target sound is calculated by the neural network model；
It is described to control the step of corresponding target lamp band of the angular range is lighted further include:
It controls the target lamp band to be shown according to the corresponding light bar special efficacy of type of the target sound, the light bar special efficacy
Including light color, light according at least one of predeterminated frequency flashing and brightness gradual change.
Further, the indicating means in above sound orientation, wherein described according to audio characteristic data calculating
The step of Sounnd source direction of target sound includes:
The audio characteristic data is calculated by GCC-PHAT algorithm, to obtain the time difference that the target sound reaches user,
And the Sounnd source direction of target sound is calculated according to the time difference.
Further, the indicating means in above sound orientation, wherein the audio data of two-channel in the acquisition system
Step includes:
The voice data stream of the two-channel of typing in acquisition system, the voice data stream is continuously added in cache blocks；
The audio clip data in the cache blocks is obtained as audio data.
Further, the indicating means in above sound orientation, wherein the step of feature extraction is carried out to the audio data
Before further include:
Amplitude modulation by tone is carried out to the audio data according to the current volume of system and preset target volume, so that the audio
The volume of data is the target volume.
Further, the indicating means in above sound orientation, wherein at least a side of the mobile terminal is uniformly arranged
Three groups of light bar, using user's head position as pole, to establish polar coordinate system to for polar axis immediately ahead of user's head, by user
The direction for receiving sound is divided into three angular ranges, is respectively as follows: 120 ° of 30 ° ~ ﹣ of ﹣；30 ° ~ 0 ° of ﹣, 0 ° ~ 30 °, 120 ° ~ 180 °, ﹣
180 ° of 120 ° ~ ﹣；30°~120°.
Further, the indicating means in above sound orientation, wherein described according to audio characteristic data calculating
The step of Sounnd source direction of target sound further include:
The sound source of the target sound is calculated at a distance from user according to the audio characteristic data；
It is described to control the step of corresponding target lamp band of the angular range is lighted further include:
Control the target lamp band brightness be it is described apart from corresponding brightness or the control target lamp band according to the distance
Corresponding frequency scintillation.
The embodiment of the present invention also provides a kind of instruction device of sound bearing, is applied in mobile terminal, the mobile terminal
At least one side edge be provided with multiple groups light bar, centered on the user's head position in the interface of mobile terminal, by user
The direction for receiving sound is divided into multiple angular ranges, and the multiple groups light bar and the multiple angular range are in corresponding relationship, institute
Stating instruction device includes:
Extraction module carries out feature extraction for obtaining the audio data of two-channel in system, and to the audio data, obtains
Audio characteristic data；
Judgment module, for by neural network model judge extract audio characteristic data whether be target sound data；
Computing module, for calculating the Sounnd source direction of the target sound according to the audio characteristic data；
Control module for determining angular range belonging to the Sounnd source direction, and controls the corresponding target of the angular range
Light bar is lighted.
The embodiment of the invention also provides a kind of readable storage medium storing program for executing, are stored thereon with program, state program and are held by processor
The method as described in any of the above-described is realized when row.
The embodiment of the invention also provides a kind of mobile terminals, including memory, processor and storage are on a memory
And the program that can be run on a processor, the processor realize the side as described in above-mentioned any one when executing described program
Method.
The general direction for indicating sound source in the embodiment of the present invention by the light bar of mobile terminal side, when sound output is set
Standby or environment is limited or player itself is difficult to accomplish to listen according to sound when arguing, by way of visual alerts, is supplied to
The rough orientation of user, as preliminary judgement and to remind, to promote game experiencing.
Detailed description of the invention
Fig. 1 is the flow chart of the indicating means of sound bearing in first embodiment of the invention；
Fig. 2 is light bar and the schematic diagram of corresponding angular range that mobile terminal sets two sides setting in first embodiment of the invention；
Fig. 3 is the flow chart of the indicating means of sound bearing in second embodiment of the invention；
Fig. 4 is the structural schematic diagram of caching group in second embodiment of the invention；
Fig. 5 is the structural block diagram of the instruction device of sound bearing in third embodiment of the invention.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, and for explaining only the invention, and is not considered as limiting the invention.
Referring to following description and drawings, it will be clear that these and other aspects of the embodiment of the present invention.In these descriptions
In attached drawing, some particular implementations in the embodiment of the present invention are specifically disclosed, to indicate to implement implementation of the invention
Some modes of the principle of example, but it is to be understood that the scope of embodiments of the invention is not limited.On the contrary, of the invention
Embodiment includes all changes, modification and the equivalent fallen within the scope of the spirit and intension of attached claims.
Referring to Fig. 1, this method is applied to mobile whole for the indicating means of the sound bearing in first embodiment of the invention
In end, mobile terminal such as mobile phone, tablet computer, personal digital assistant etc..At least one side edge of the mobile terminal is arranged
There is multiple groups light bar, centered on the user's head position in the interface of mobile terminal, the direction that user receives sound is divided
For multiple angular ranges, light bar and multiple angular ranges are in corresponding relationship.User herein be mobile terminal in application program (such as
Games) in correspondence user virtual objects.The indicating means includes step S11 ~ S14.
Step S11, the audio data of two-channel in acquisition system, and feature extraction is carried out to the audio data, it obtains
Audio characteristic data.
Dual-channel audio data is often provided in mobile terminal, can be obtained by the audio data being arranged on mobile terminal
Interface obtains the frequency evidence enrolled in system.The audio data of acquisition for mobile terminal is sound for what is enrolled in terminal system
Sound, i.e., when carrying out game, acquisition is game sound；If opening music while game, what is recorded is two
The sound of person's synthesis, the musical sound enrolled in the case of this kind can interfere judgement.First to the audio data of acquisition in the present embodiment
Feature extraction is carried out, the common algorithm for extracting audio frequency characteristics is, for example, MFCC(mel-frequency cepstrum coefficient) method.MFCC is sound
Constituents extraction in frequency signal with identification comes out, and then other interference informations are removed.Using MFCC method by audio
Audio characteristic data is calculated in data.
Step S12, by neural network model judge extract audio characteristic data whether be target sound data.
Audio characteristic data is carried out by neural network model to be calculated, and is judged whether to belong to target according to calculated result
Sound.The present embodiment target sound is the sound in game, for example including footsteps, shot and the vehicle sound etc. in game.
Wherein, the method for distinguishing audio data can be carried out based on neural network model in the prior art.Such as it can be used
DNN model (Deep Neural Networks, deep neural network model) is divided from DNN by the position of different layers, in DNN
The neural net layer in portion can be divided into three layers, input layer, hidden layer and output layer.
When it is implemented, first constructing DNN model, and the DNN model is trained, which can be used
Tensorflow frame trains audio data, so that it is determined that the parameter of DNN model, the DNN model after training is for distinguishing sound
The sound of frequency evidence is target sound or non-targeted sound.DNN model after the audio data input training that finally will acquire
In, the type of the final output audio data, as target sound or non-targeted sound.MFCC can be used, and (mel-frequency falls
Spectral coefficient) audio feature vector data are calculated in audio data by method, and are inputted in the DNN model after training.
Step S13, when determining the data that the audio characteristic data is target sound, according to the audio frequency characteristics number
According to the Sounnd source direction for calculating the target sound.
Wherein, GCC-PHAT algorithm (Generalized Cross Correlation PHAse can be passed through
Transformation, broad sense cross-correlation-phse conversion method) audio characteristic data is calculated, obtain target sound arrival
The time difference of user, and calculate according to the time difference Sounnd source direction of target sound.Steps are as follows for its specific calculating:
The audio characteristic data of left and right acoustic channels is converted to plural form data by step S1, and is become respectively using fast Fourier
It gets in return to two groups of frequency domain datas.
Step S2 calculates its conjugate complex number to one group frequency domain data and is multiplied again with another group of frequency domain data, then answers again
With inverse Fourier transform, to obtain cross-correlation function.
Step S3 calculates the peak value of cross-correlation function, the coordinate position i.e. phase difference of two sound channels where peak value.
Step S4, phase difference obtain TDOA (the time difference tau that sound reaches) divided by sample frequency, count further according to TDOA
Calculate sound source angle α.Its calculation formula is i.e.:
sinα= tau * (c/b)；
Wherein, the time difference tau that sound reaches=Δ n/f, α is incidence angle of the sound source to observer, and b is left and right ear in horizontal blank
Between on distance, c is speed of sound, and Δ n is the phase difference of left and right acoustic channels, and f is audio data samples frequency.Wherein, join below
Number is known quantity:
B value is 20cm, and c value is 343m/s, and f can be 1600hz.
Therefore, the phase difference n of left and right acoustic channels data is obtained using GCC-PHAT algorithm, sound source can be finally calculated
Angle [alpha].
Step S14 determines angular range belonging to the Sounnd source direction, and controls the corresponding target lamp of the angular range
Band is lighted.
Multiple groups light bar is provided in at least side of mobile terminal, every group of light bar at least contains a lamp bead, when electric current is logical
It is out-of-date to light.For example, the two sides of mobile terminal can be respectively set to multiple groups light bar in the present embodiment, the light bar of two sides setting
Position it is opposite, and light bar effect is identical.For example, as shown in Fig. 2, the two sides of mobile terminal are evenly arranged with three groups of lamps respectively
Band, wherein side light bar is that respectively L1, L2 and L3, other side light bar are respectively N1, N2 and N3 from left to right from left to right.With
User's head position position in interface of mobile terminal is pole, to establish polar coordinates to for polar axis immediately ahead of user's head
The direction that user receives sound is divided into three angular ranges: A by system；B1 ~ B4 and C, wherein A indicates angular range are as follows: 30 ° of ﹣
120 ° of ~ ﹣；B1 ~ B4 indicate angular range be 30 ° ~ 0 ° of ﹣, 0 ° ~ 30 °, 120 ° ~ 180 °, 180 ° of 120 ° ~ ﹣ of ﹣；C indicates angular range
It is 30 ° ~ 120 °.The light bar of mobile terminal two sides and the corresponding relationship of three angular ranges are as shown in table 1.
When determining angular range belonging to Sounnd source direction, the corresponding light bar of the angular range is lighted.For example, sound source exists
Direction in the polar coordinate system is the direction at 25 ° of angles, then lights light bar L1 and N1, user can be straight according to the light bar lighted
The general direction of the judgement sound source of sight.
It should be noted that the setting of light bar quantity and angular range in the present embodiment is only as an example, to this
Invention not limits.Can also there are other set-up modes in other embodiments of the invention, for example, the quantity of every side lamp band can
Four or five are set as, corresponding angular range is also provided with four or five, and the corresponding angular range of every group of light bar can
It is configured according to actual needs.In addition to this it is possible to which light bar only is arranged in side.
The general direction for indicating sound source in the present embodiment by the light bar of mobile terminal side, when audio output device or
Environment is limited or player itself is difficult to accomplish to listen according to sound when arguing, by way of visual alerts, is supplied to user
Rough orientation, as preliminary judgement and to remind, to promote game experiencing.
Referring to Fig. 3, for the indicating means of the sound bearing in second embodiment of the invention, including step S21 ~ S29.
Step S21, the voice data stream of the two-channel of typing, the voice data stream is continuously added in acquisition system
In cache blocks.
Step S22 obtains the audio clip data in the cache blocks as audio data, and extracts the audio data
Audio characteristic data.
Step S23, to the audio characteristic data in each sound channel, successively the data of every three sampled points take primary equal
Value, obtains one section of new audio data.
The identification and classification of audio data first have to be cut into one section one using buffer window to Wave data sub-frame processing
The data of section, simple framing can be such that the part at signal both ends is weakened, also need to retain a part of data thus as next
The start-up portion of frame.A caching group as shown in Figure 4 is safeguarded thus, and the data of overlapping region are synchronized to copy to next
Frame is transmitted to the calculating of next stage after a buffer zone is full, and the area that empties the cache.
Audio sampling frequency when mobile terminal is recorded in the present embodiment is 48000Hz, records dual-channel audio flow data.
Using audio buffer group, voice data stream is continuously added in cache blocks, has been filled with 96000 byte when caching at this time, also
I.e. the dual-channel audio data of 500ms when, this cache data be resampled to 16000hz.When it is implemented, by audio piece
Segment data, in each sound channel, successively the data of every three sampled points take a mean value, obtain one section of new audio data, from
And by audio from 48000Hz resampling to 16000Hz.
The new audio data is carried out the mean value computation of left and right acoustic channels, to mix by step S24 in each sampled point
For monaural audio data.
The audio data of acquisition for mobile terminal is dual channel data, and when carrying out audio data identification and classification, needing will be double
Channel data is mixed into the data of monophonic.It passes through be calculated one section of new audio data on the left side sampled each time
The audio data of channel audio data and right channel carries out mean value computation, can be mixed into monaural audio data.
Step S25 is calculated the monaural audio data, by neural network model to determine whether for target
The data of sound.
Step S26 is calculated when the monaural audio data is the data of target sound by the neural network model
The type of the target sound.
Wherein, the method for distinguishing audio data can be carried out based on neural network model in the prior art.Such as it can be used
DNN model (Deep Neural Networks, deep neural network model).When it is implemented, first constructing DNN model, and lead to
All types of audio datas is crossed as training data to be trained the DNN model, which can be used tensorflow
Frame trains audio data, so that it is determined that the parameter of DNN model, the DNN model after training is used to distinguish the sound of audio data
Sound type.The sound type can be determined by trained audio data, for example, by shot in game gathered in advance,
The audio datas such as footsteps, vehicle sound are trained DNN model.By the mixed monaural audio data input instruction of mobile terminal
In DNN model after white silk, the type of the final output audio data.
When it is implemented, neural network model i.e. exportable target sound when handling monaural audio data
Classification information, if target sound is not present in the mono data, output is 0.
Step S27 calculates the Sounnd source direction of the target sound according to the audio characteristic data.
Audio characteristic data can be calculated by GCC-PHAT algorithm, determine that target sound reaches the time of user
Difference, and calculate according to the time difference Sounnd source direction of target sound.
Step S28 determines angular range belonging to the Sounnd source direction.
Step S29 controls the corresponding target lamp band of the angular range and lights, and according to the type pair of the target sound
The light bar special efficacy answered is shown.
In the present embodiment, 3 groups of light bar can be respectively arranged in the both sides of the edge of the mobile terminal, in interface of mobile terminal
User-center, the direction that user receives sound is divided into 3 angular ranges, three light bar of every survey respectively with this 3
A angular range is in one-to-one relationship.Specifically, the corresponding relationship of the light bar of the mobile terminal two sides and three angular ranges
It can be as shown in the table 1 in above-described embodiment.When determining angular range belonging to Sounnd source direction, by the corresponding lamp of the angular range
Band is lighted.For example, direction of the sound source in the polar coordinate system be 60 ° of angles direction, then light bar L2 and N2 are lighted, user according to
The light bar lighted can intuitively judge the general direction of sound source.If being determined to multiple directions simultaneously there are target sound,
The corresponding light bar in multiple direction is lighted simultaneously.
Further, each type sound corresponds to a kind of light bar special efficacy, which includes color, according to predeterminated frequency
At least one of flashing and light intensity gradual change.User according to the light bar special efficacy of the target lamp band be can determine whether be that type sound
The corresponding light bar special efficacy of vehicle sound in sound, such as game is red light, the corresponding light bar special efficacy of footsteps be yellow light and
It is flashed with frequency of once per second, the corresponding light bar special efficacy of shot is that lamplight brightness is gradually reinforced.
I.e. when target sound is vehicle sound, control target lamp band is lighted, and light color is red；When target sound is Che Sheng
When, control target lamp band is lighted, and color is that yellow and light are dodged according to the corresponding frequency of type for target sound
It is bright；When target sound is shot, control control target lamp band is lighted, and light intensity is gradually reinforced.
It should be understood that the above-mentioned light bar special efficacy enumerated only as an example, specific implementation when the corresponding light bar of each type
Special efficacy can be designed and combine according to the actual situation, can also increase other special efficacys.
Further, the duration of the light bar special efficacy can be configured, such as the duration is 2s, i.e. target lamp band point
The bright duration is 2s, and the lamplight pointing in the period is enough to cause user's note that not needing to be constantly in light shape
State, in order to avoid unnecessary electricity is caused to waste.
Further, before the step of extracting the audio characteristic data in audio data in other embodiments of the invention
Also amplitude modulation by tone can be carried out to the dual-channel audio data of acquisition according to system current volume and preset target volume.
Inputting audio data often carry out amplitude modulation by default volume in system, obtain when system sound volume variation
Audio data is different, is not easy to audio identification.Therefore it needs to carry out amplitude modulation to the audio data of acquisition, so that the audio number obtained
According to volume it is constant always, specific implementation step includes:
Step S31 monitors and saves the parameter value of system current volume；
The parameter value of the parameter value of current volume and preset target volume is counted index, the ratio after fetching number by step S32 respectively
Value is the quotient of amplitudes of target volume and current volume；
Step S33, the audio data by the value of each audio signal of audio data multiplied by the quotient of amplitudes, after obtaining amplitude modulation.
When it is implemented, system presets the parameter value Vt an of target volume, mobile phone is monitored and saved by interface
The parameter value of system current volume, is stated that Vn here.Since the loudness of volume is experienced as exponent function relation in human ear, because
And audio amplitude and volume parameters are also exponent function relation, therefore target volume and current volume are counted index, fetching number respectively
Ratio afterwards is the quotient of amplitudes of target volume and existing volume:
scale = exp(Vt)/exp(Vn)。
After obtaining system audio data, by the value of each audio signal multiplied by scale, the data being converted to just become mesh
Mark with phonetic symbols amount.The volume of audio data after conversion is that target volume will finally adjust the data under different size volume
Under width to this target volume.
As the another embodiment of the embodiment of the present invention, target can also be distinguished according to the luminous intensity of target lamp band
The sound source of sound is at a distance from user.In development of games, sound source can be simulated in such a way that gaming audio is successively decreased by volume
Distance, sound source can judge that sound is strong in the terminal at a distance from user according to the intensity for the sound that user receives
Degree can be indicated with the voltage magnitude of two channel audio signals of left and right.Voltage i.e. in acquisition for mobile terminal audio characteristic data
Amplitude, and the light intensity for calculating sound source according to the voltage magnitude at a distance from user, and controlling target lamp band is this apart from corresponding
Luminous intensity.Prestore the corresponding relationship of distance value (or distance range) and intensity of light in the mobile terminal system, when use looks into
Inquiry.
Sound source is distinguished at a distance from user by luminous intensity in the present embodiment, in other embodiments of the invention, also
By other means, such as distance can be distinguished by the flicker frequency of light, the corresponding light of the closer sound of distance dodges
Bright frequency is higher, and the remoter corresponding light flash frequency of sound of distance is lower.
Referring to Fig. 5, being applied to mobile terminal for the instruction device of one of third embodiment of the invention sound bearing
In, at least one side edge of the mobile terminal is provided with multiple groups light bar, and with the user-center in interface, user is received sound
Direction be divided into multiple angular ranges, the multiple groups light bar and the multiple angular range are in corresponding relationship, and the instruction fills
It sets and includes:
Extraction module 10 carries out feature extraction for obtaining the audio data of two-channel in system, and to the audio data, obtains
To audio characteristic data；
Judgment module 20, for by neural network model judge extract audio characteristic data whether be target sound number
According to；
Computing module 30, for calculating the Sounnd source direction of the target sound according to the audio characteristic data；
Control module 40 for determining angular range belonging to the Sounnd source direction, and controls the corresponding mesh of the angular range
Beacon light band is lighted.
Further, the instruction device in above sound orientation, wherein the judgment module 20 includes:
Sampling module, for, in each sound channel, successively the data of every three sampled points to take once to the audio characteristic data
Mean value obtains one section of new audio data.
Mean value computation module, by by the new audio data each sampled point carry out left and right acoustic channels mean value based on
It calculates, to be mixed into monaural audio data；
Judging submodule, for the monaural audio data to be calculated by neural network model, and determine whether for
Target sound.
Further, the instruction device in above sound orientation, wherein the computing module 30 is also used to: by the mind
The type of the target sound is calculated through network model；
The control module 40 is also used to control the target lamp band according to the corresponding light bar special efficacy of type of the target sound
It is shown, the light bar special efficacy includes light color, light according at least one in predeterminated frequency flashing and brightness gradual change
Kind.
The technical effect of the instruction device of sound bearing provided by the embodiment of the present invention, realization principle and generation is with before
It is identical to state embodiment of the method, to briefly describe, Installation practice part does not refer to place, can refer to phase in preceding method embodiment
Answer content.
The present invention also proposes a kind of readable storage medium storing program for executing, is stored thereon with computer program, which is executed by processor
The indicating means of Shi Shixian above-mentioned sound bearing.
The embodiment of the invention also provides a kind of mobile terminals, including memory, processor and storage are on a memory
And the program that can be run on a processor, the processor realize above-mentioned method when executing described program.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.
The more specific example (non-exhaustive list) of computer-readable medium include the following: there are one or more wirings
Electrical connection section (electronic device), portable computer diskette box (magnetic device), random-access memory (ram), read-only memory
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable
Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.It, and in another embodiment, can be under well known in the art for example, if realized with hardware
Any one of column technology or their combination are realized: having a logic gates for realizing logic function to data-signal
Discrete logic, with suitable combinational logic gate circuit specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
Centainly refer to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be any
One or more embodiment or examples in can be combined in any suitable manner.
The embodiments described above only express several embodiments of the present invention, and the description thereof is more specific and detailed, but simultaneously
Limitations on the scope of the patent of the present invention therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to guarantor of the invention
Protect range.Therefore, the scope of protection of the patent of the invention shall be subject to the appended claims.
Claims (10)
1. a kind of indicating means of sound bearing is applied in mobile terminal, which is characterized in that at least side of the mobile terminal
Edge is provided with multiple groups light bar, and centered on the user's head position in the interface of mobile terminal, user is received sound
Direction is divided into multiple angular ranges, and the multiple groups light bar and the multiple angular range are in corresponding relationship, the indicating means
Include:
The audio data of two-channel in acquisition system, and feature extraction is carried out to the audio data, obtain audio characteristic data；
By neural network model judge extract audio characteristic data whether be target sound data；
If so, calculating the Sounnd source direction of the target sound according to the audio characteristic data；
It determines angular range belonging to the Sounnd source direction, and controls the corresponding target lamp band of the angular range and light.
2. the indicating means of sound bearing as described in claim 1, which is characterized in that described to be judged by neural network model
The step of whether audio characteristic data of extraction is the data of target sound include:
To the audio characteristic data in each sound channel, successively the data of every three sampled points take a mean value, obtain one section
New audio data；
The mean value computation that the new audio data is carried out to left and right acoustic channels in sampled point each time, to be mixed into monophonic sound
Frequency evidence；
The monaural audio data is calculated by neural network model, to determine whether for target sound.
3. the indicating means of sound bearing as described in claim 1, which is characterized in that described to be judged by neural network model
The step of whether audio characteristic data of extraction is the data of target sound further include:
The type of the target sound is calculated by the neural network model；
It is described to control the step of corresponding target lamp band of the angular range is lighted further include:
It controls the target lamp band to be shown according to the corresponding light bar special efficacy of type of the target sound, the light bar special efficacy
Including light color, light according at least one of predeterminated frequency flashing and brightness gradual change.
4. the indicating means of sound bearing as described in claim 1, which is characterized in that described according to the audio characteristic data
The step of calculating the Sounnd source direction of the target sound include:
The audio characteristic data is calculated by GCC-PHAT algorithm, to obtain the time difference that the target sound reaches user,
And the Sounnd source direction of target sound is calculated according to the time difference.
5. the indicating means of sound bearing as described in claim 1, which is characterized in that carry out feature to the audio data and mention
Before the step of taking further include:
Amplitude modulation by tone is carried out to the audio data according to the current volume of system and preset target volume, so that the audio
The volume of data is the target volume.
6. the indicating means of sound bearing as described in claim 1, which is characterized in that at least a side of the mobile terminal
Three groups of light bar are uniformly arranged, using user's head position as pole, to establish polar coordinates to for polar axis immediately ahead of user's head
The direction that user receives sound is divided into three angular ranges, is respectively as follows: 120 ° of 30 ° ~ ﹣ of ﹣ by system；30 ° ~ 0 ° of ﹣, 0 ° ~ 30 °,
120 ° ~ 180 °, 180 ° of 120 ° ~ ﹣ of ﹣；30°~120°.
7. the indicating means of sound bearing as described in claim 1, which is characterized in that described according to the audio characteristic data
The step of calculating the Sounnd source direction of the target sound further include:
The sound source of the target sound is calculated at a distance from user according to the audio characteristic data；
It is described to control the step of corresponding target lamp band of the angular range is lighted further include:
Control the target lamp band brightness be it is described apart from corresponding brightness or the control target lamp band according to the distance
Corresponding frequency scintillation.
8. a kind of instruction device of sound bearing is applied in mobile terminal, which is characterized in that at least side of the mobile terminal
Edge is provided with multiple groups light bar, and centered on the user's head position in the interface of mobile terminal, user is received sound
Direction is divided into multiple angular ranges, and the multiple groups light bar and the multiple angular range are in corresponding relationship, the instruction device
Include:
Extraction module carries out feature extraction for obtaining the audio data of two-channel in system, and to the audio data, obtains
Audio characteristic data；
Judgment module, for by neural network model judge extract audio characteristic data whether be target sound data；
Computing module, for calculating the Sounnd source direction of the target sound according to the audio characteristic data；
Control module for determining angular range belonging to the Sounnd source direction, and controls the corresponding target of the angular range
Light bar is lighted.
9. a kind of readable storage medium storing program for executing, is stored thereon with program, which is characterized in that realized such as when described program is executed by processor
Method as claimed in claim 1 to 7.
10. a kind of mobile terminal including memory, processor and stores the journey that can be run on a memory and on a processor
Sequence, which is characterized in that the processor realizes the method as described in claim 1-7 any one when executing described program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985754.XA CN110488225B (en) | 2019-10-17 | 2019-10-17 | Voice direction indicating method and device, readable storage medium and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985754.XA CN110488225B (en) | 2019-10-17 | 2019-10-17 | Voice direction indicating method and device, readable storage medium and mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110488225A true CN110488225A (en) | 2019-11-22 |
| CN110488225B CN110488225B (en) | 2020-02-07 |
Family
ID=68544717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910985754.XA Active CN110488225B (en) | 2019-10-17 | 2019-10-17 | Voice direction indicating method and device, readable storage medium and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110488225B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110972053A (en) * | 2019-11-25 | 2020-04-07 | 腾讯音乐娱乐科技（深圳）有限公司 | Method and related apparatus for constructing a listening scene |
| CN111929645A (en) * | 2020-09-23 | 2020-11-13 | 深圳市友杰智新科技有限公司 | Method and device for positioning sound source of specific human voice and computer equipment |
| CN112415467A (en) * | 2020-11-06 | 2021-02-26 | 中国海洋大学 | Single-vector subsurface buoy target positioning implementation method based on neural network |
| CN114355289A (en) * | 2022-03-19 | 2022-04-15 | 深圳市烽火宏声科技有限公司 | Sound source positioning method, sound source positioning device, storage medium and computer equipment |
| WO2024082800A1 (en) * | 2022-10-18 | 2024-04-25 | 抖音视界有限公司 | Audio processing method and apparatus, and terminal device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140241549A1 (en) * | 2013-02-22 | 2014-08-28 | Texas Instruments Incorporated | Robust Estimation of Sound Source Localization |
| CN105378826A (en) * | 2013-05-31 | 2016-03-02 | 诺基亚技术有限公司 | An audio scene apparatus |
| CN107231586A (en) * | 2016-03-24 | 2017-10-03 | 徐超 | Sound is listened to distinguish the method and device of position |
| CN107450883A (en) * | 2017-07-19 | 2017-12-08 | 维沃移动通信有限公司 | A kind of audio data processing method, device and mobile terminal |
| CN109788130A (en) * | 2018-12-27 | 2019-05-21 | 努比亚技术有限公司 | Terminal and its orientation based reminding method and computer readable storage medium |
| CN109960484A (en) * | 2017-12-26 | 2019-07-02 | 腾讯科技（深圳）有限公司 | A kind of audio volume acquisition methods and device, storage medium, terminal |
-
2019
- 2019-10-17 CN CN201910985754.XA patent/CN110488225B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140241549A1 (en) * | 2013-02-22 | 2014-08-28 | Texas Instruments Incorporated | Robust Estimation of Sound Source Localization |
| CN105378826A (en) * | 2013-05-31 | 2016-03-02 | 诺基亚技术有限公司 | An audio scene apparatus |
| CN107231586A (en) * | 2016-03-24 | 2017-10-03 | 徐超 | Sound is listened to distinguish the method and device of position |
| CN107450883A (en) * | 2017-07-19 | 2017-12-08 | 维沃移动通信有限公司 | A kind of audio data processing method, device and mobile terminal |
| CN109960484A (en) * | 2017-12-26 | 2019-07-02 | 腾讯科技（深圳）有限公司 | A kind of audio volume acquisition methods and device, storage medium, terminal |
| CN109788130A (en) * | 2018-12-27 | 2019-05-21 | 努比亚技术有限公司 | Terminal and its orientation based reminding method and computer readable storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110972053A (en) * | 2019-11-25 | 2020-04-07 | 腾讯音乐娱乐科技（深圳）有限公司 | Method and related apparatus for constructing a listening scene |
| CN111929645A (en) * | 2020-09-23 | 2020-11-13 | 深圳市友杰智新科技有限公司 | Method and device for positioning sound source of specific human voice and computer equipment |
| CN112415467A (en) * | 2020-11-06 | 2021-02-26 | 中国海洋大学 | Single-vector subsurface buoy target positioning implementation method based on neural network |
| CN114355289A (en) * | 2022-03-19 | 2022-04-15 | 深圳市烽火宏声科技有限公司 | Sound source positioning method, sound source positioning device, storage medium and computer equipment |
| CN114355289B (en) * | 2022-03-19 | 2022-06-10 | 深圳市烽火宏声科技有限公司 | Sound source positioning method, sound source positioning device, storage medium and computer equipment |
| WO2024082800A1 (en) * | 2022-10-18 | 2024-04-25 | 抖音视界有限公司 | Audio processing method and apparatus, and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110488225B (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110488225A (en) | Indicating means, device, readable storage medium storing program for executing and the mobile terminal of sound bearing | |
| US10685638B2 (en) | Audio scene apparatus | |
| KR102639491B1 (en) | Personalized, real-time audio processing | |
| KR20220054602A (en) | Systems and methods that support selective listening | |
| CN110972053B (en) | Method and related apparatus for constructing a listening scene | |
| Ericson et al. | Factors that influence intelligibility in multitalker speech displays | |
| CN107450724A (en) | A kind of gesture identification method and system based on dual-channel audio Doppler effect | |
| CN109658942A (en) | A kind of audio data processing method, device and relevant device | |
| Best et al. | Spatial unmasking of birdsong in human listeners: Energetic and informational factors | |
| CN107241672B (en) | Method, device and equipment for obtaining spatial audio directional vector | |
| Cooke et al. | Computational auditory scene analysis: Listening to several things at once | |
| JP2009539133A (en) | Utterance distinction | |
| Griesinger | What is clarity, and how it can be measured? | |
| JP6737342B2 (en) | Signal processing device and signal processing method | |
| JP2023536270A (en) | Systems and Methods for Headphone Equalization and Room Adaptation for Binaural Playback in Augmented Reality | |
| CN106465032B (en) | The apparatus and method for manipulating input audio signal | |
| CN108389584A (en) | Sound analysis method and device | |
| CN101516055A (en) | Method and device capable of simulating three-dimensional echo sound effect in different acoustic environments | |
| Plack | Auditory perception | |
| Topper et al. | Spatio-operational spectral (sos) synthesis | |
| KR20050048686A (en) | Method for simulating a movement by means of an acoustic reproduction device and sound reproduction arrangement therefor | |
| May et al. | Preserving auditory situation awareness in headphone-distracted persons | |
| Ziemer et al. | Psychoacoustics | |
| US11696088B1 (en) | Method and apparatus to generate a six dimensional audio dataset | |
| Hofmann | Limits of Audio-Visual Congruence Using 3D Videos in VR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230320 Address after: 518055 1501, Building 1, Chongwen Park, Nanshan Zhiyuan, No. 3370, Liuxian Avenue, Fuguang Community, Taoyuan Street, Nanshan District, Shenzhen, Guangdong Province Patentee after: Shenzhen Grey Shark Technology Co.,Ltd. Address before: Room 601, Block A, Chuangzhi Building, No. 17, Xinghuo Road, Jiangbei New District, Nanjing, Jiangsu, 210000 Patentee before: Nanjing Thunder Shark Information Technology Co.,Ltd. |