CN115002607B - Sound source position determining method, device and storage medium - Google Patents
Sound source position determining method, device and storage medium Download PDFInfo
- Publication number
- CN115002607B CN115002607B CN202210575064.9A CN202210575064A CN115002607B CN 115002607 B CN115002607 B CN 115002607B CN 202210575064 A CN202210575064 A CN 202210575064A CN 115002607 B CN115002607 B CN 115002607B
- Authority
- CN
- China
- Prior art keywords
- coordinates
- microphone
- sound source
- determining
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000005236 sound signal Effects 0.000 claims abstract description 148
- 230000005540 biological transmission Effects 0.000 claims description 12
- 230000033001 locomotion Effects 0.000 claims description 11
- 101000893549 Homo sapiens Growth/differentiation factor 15 Proteins 0.000 description 14
- 101000692878 Homo sapiens Regulator of MON1-CCZ1 complex Proteins 0.000 description 14
- 102100026436 Regulator of MON1-CCZ1 complex Human genes 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 8
- 102100032912 CD44 antigen Human genes 0.000 description 7
- 101000868273 Homo sapiens CD44 antigen Proteins 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 102000008482 12E7 Antigen Human genes 0.000 description 5
- 108010020567 12E7 Antigen Proteins 0.000 description 5
- 102100037904 CD9 antigen Human genes 0.000 description 5
- 101000738354 Homo sapiens CD9 antigen Proteins 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000015541 sensory perception of touch Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/326—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only for microphones
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
- G01S5/22—Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention discloses a sound source position determining method, a sound source position determining device and a storage medium, wherein the method is applied to head-mounted equipment, the head-mounted equipment comprises at least four microphones, and microphone coordinates of the microphones in a space coordinate system are determined according to equipment coordinates of preset sound source equipment; determining a time difference of receiving time points of the microphone for receiving the audio signals sent by the sound source to be positioned; and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time difference. According to the invention, the position of the sound source making sound can be accurately calculated through the head-mounted equipment, so that a user can clearly know the specific position of the sound source making sound in the real space environment under the condition of wearing the head-mounted equipment.
Description
Technical Field
The present invention relates to the field of virtual reality technologies, and in particular, to a method, an apparatus, and a storage medium for determining a sound source position.
Background
Virtual Reality (VR) technology, which is a Virtual world that simulates and generates a three-dimensional space by using a computer or other intelligent computing devices, provides a simulation of visual, auditory, tactile senses of a user, and enables the user to look like a body calendar. Different head-mounted devices (for example, VR devices) are produced on the market based on VR technology, and a user can experience a virtual reality scene by using the head-mounted devices. When the existing head-mounted equipment presents a pseudo-reality scene, the sound source position in the reality environment cannot be accurately presented in the virtual reality scene, and the sound feeling of a user experiencing the real world in the virtual reality scene is reduced.
Disclosure of Invention
The embodiment of the invention provides a sound source position determining method, a sound source position determining device and a sound source position determining storage medium, and aims to solve the technical problem that the sound source position in a real environment cannot be accurately presented in a virtual reality scene by using existing head-mounted equipment, so that the sound feeling of a user in the real world is reduced.
The embodiment of the invention provides a sound source position determining method which is applied to head-mounted equipment, wherein the head-mounted equipment comprises at least four microphones, and the sound source position determining method comprises the following steps:
determining microphone coordinates of the microphone in a space coordinate system according to equipment coordinates of a preset sound source device, wherein the space coordinate system is established in a space where the head-mounted device is located;
determining a time difference of receiving time points of the microphone for receiving the audio signals sent by the sound source to be positioned;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time difference.
In an embodiment, the preset sound source device includes at least four speakers disposed on the head-mounted device, and the step of determining the microphone coordinates of the microphone in the spatial coordinate system according to the device coordinates of the preset sound source device includes:
Determining device coordinates of each of the speakers;
controlling each loudspeaker to emit a first calibration audio signal;
determining a first calibration time difference of a first calibration receiving time point of each microphone for receiving the first calibration audio signal;
and determining the microphone coordinates according to the first calibration time difference and the equipment coordinates.
In an embodiment, the step of determining the device coordinates of each of the speakers comprises:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
controlling each loudspeaker to emit a second calibration audio signal, and determining a second calibration time difference of a second calibration receiving time point when the microphone receives the second calibration audio signal;
and determining the equipment coordinates of each loudspeaker according to the second calibration time difference and the initial microphone coordinates.
In one embodiment, the step of controlling each of the speakers to emit a first nominal audio signal comprises:
constructing a motion coordinate system based on the center of the headset;
and controlling a loudspeaker positioned on a coordinate axis of the motion coordinate system and a loudspeaker positioned outside a coordinate axis plane to emit the first calibration audio signal.
In an embodiment, the step of determining the sound source coordinates of the sound source to be positioned in the spatial coordinate system according to the microphone coordinates and the time difference includes:
setting a minimum time of flight for the audio signal to reach one of the microphones;
constructing an equation set according to the sound transmission speed, the minimum flight time, the microphone coordinates and the time difference;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the solving result of the equation set.
In an embodiment, after the step of determining the sound source coordinates of the sound source to be positioned in the spatial coordinate system according to the microphone coordinates and the time difference, the method further includes:
determining the relative position coordinates of the sound source to be positioned relative to the head-mounted equipment according to the sound source coordinates and the microphone coordinates;
determining playing parameters corresponding to the audio signals according to the relative position coordinates, and playing the audio signals based on the playing parameters; and/or the number of the groups of groups,
and marking the position of the sound source to be positioned in a real environment picture displayed by the head-mounted equipment according to the relative position coordinates.
In an embodiment, the preset sound source device includes at least one speaker disposed on the head-mounted device, and the step of determining the microphone coordinates of the microphone in the spatial coordinate system according to the device coordinates of the preset sound source device further includes:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
controlling the loudspeaker to emit a third calibration audio signal, and determining a third calibration time difference of a third calibration receiving time point when the microphone receives the third calibration audio signal;
determining equipment coordinates of each loudspeaker according to the third calibration time difference and the initial microphone coordinates, and controlling the loudspeaker to emit a fourth calibration audio signal;
and determining a fourth calibration time difference of a fourth calibration receiving time point of each microphone for receiving the fourth calibration audio signal, and determining the microphone coordinates according to the fourth calibration time difference and the equipment coordinates.
In an embodiment, the preset sound source device includes at least one speaker with known device coordinates, and the step of determining the microphone coordinates of the microphone in the spatial coordinate system according to the device coordinates of the preset sound source device further includes:
Acquiring equipment coordinates of the loudspeaker;
controlling each loudspeaker to send out a calibrated audio signal;
determining a calibration time difference of a calibration receiving time point of each microphone for receiving the calibration audio signal;
and determining the microphone coordinates according to the calibration time difference and the equipment coordinates.
In addition, to achieve the above object, the present invention also provides a head-mounted device including: the sound source position determining device comprises a memory, a processor and a sound source position determining program which is stored in the memory and can run on the processor, wherein the sound source position determining program realizes the steps of the sound source position determining method when being executed by the processor.
In addition, in order to achieve the above object, the present invention also provides a storage medium having stored thereon a sound source position determining program which, when executed by a processor, implements the steps of the sound source position determining method described above.
The technical scheme of the method, the equipment and the storage medium for determining the sound source position provided by the embodiment of the invention has at least the following technical effects or advantages:
the sound source position determining method is applied to the head-mounted equipment, the head-mounted equipment comprises at least four microphones, the microphone coordinates of the microphones in a space coordinate system are determined according to the equipment coordinates of the preset sound source equipment, the time difference of receiving time points of the microphones for receiving the audio signals sent by the sound sources to be positioned is determined, and the sound source coordinates of the sound sources to be positioned in the space coordinate system are determined according to the microphone coordinates and the time difference. According to the invention, the position of the sound source making sound can be accurately calculated through the head-mounted equipment, so that a user can clearly know the specific position of the sound source making sound in the real space environment under the condition of wearing the head-mounted equipment.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating an embodiment of a method for determining a sound source position according to the present invention;
FIG. 3 is a schematic diagram of the positional relationship between the headset and the spatial coordinate system;
FIG. 4 is a schematic diagram of a sound source to be positioned and a headset according to the present invention;
FIG. 5 is a schematic diagram of the positional relationship of a microphone, a speaker and a spatial coordinate system according to the present invention;
fig. 6 is a flowchart illustrating a method for determining a sound source position in step S230 according to the present invention.
Detailed Description
In order that the above-described aspects may be better understood, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware running environment according to an embodiment of the present invention.
It should be noted that fig. 1 may be a schematic structural diagram of a hardware running environment of the headset.
As shown in fig. 1, the headset may include: a processor 1001, such as a CPU, memory 1005, user interface 1003, network interface 1004, communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the headset structure shown in fig. 1 is not limiting on the headset and may include more or fewer components than shown, or certain components in combination, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a sound source position determining program may be included in the memory 1005 as one type of storage medium. The operating system is a program for managing and controlling hardware and software resources of the head-mounted device, a sound source position determining program and other software or running of the program.
In the head-mounted device shown in fig. 1, the user interface 1003 is mainly used for connecting a terminal, and performing data communication with the terminal; the network interface 1004 is mainly used for a background server and is in data communication with the background server; the processor 1001 may be configured to call a sound source position determining program stored in the memory 1005.
In this embodiment, the head-mounted device includes: a memory 1005, a processor 1001, and a sound source position determining program stored on the memory 1005 and executable on the processor, wherein:
when the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are performed:
determining microphone coordinates of a microphone in a space coordinate system according to equipment coordinates of a preset sound source device, wherein the space coordinate system is established in a space where the head-mounted device is located;
determining a time difference of receiving time points of the microphone for receiving the audio signals sent by the sound source to be positioned;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time difference.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
Determining device coordinates of each of the speakers;
controlling each loudspeaker to emit a first calibration audio signal;
determining a first calibration time difference of a first calibration receiving time point of each microphone for receiving the first calibration audio signal;
and determining the microphone coordinates according to the first calibration time difference and the equipment coordinates.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
controlling each loudspeaker to emit a second calibration audio signal, and determining a second calibration time difference of a second calibration receiving time point when the microphone receives the second calibration audio signal;
and determining the equipment coordinates of each loudspeaker according to the second calibration time difference and the initial microphone coordinates.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
constructing a motion coordinate system based on the center of the headset;
and controlling a loudspeaker positioned on a coordinate axis of the motion coordinate system and a loudspeaker positioned outside a coordinate axis plane to emit the first calibration audio signal.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
setting a minimum time of flight for the audio signal to reach one of the microphones;
constructing an equation set according to the sound transmission speed, the minimum flight time, the microphone coordinates and the time difference;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the solving result of the equation set.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
determining the relative position coordinates of the sound source to be positioned relative to the head-mounted equipment according to the sound source coordinates and the microphone coordinates;
determining playing parameters corresponding to the audio signals according to the relative position coordinates, and playing the audio signals based on the playing parameters; and/or the number of the groups of groups,
and marking the position of the sound source to be positioned in a real environment picture displayed by the head-mounted equipment according to the relative position coordinates.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
Controlling the loudspeaker to emit a third calibration audio signal, and determining a third calibration time difference of a third calibration receiving time point when the microphone receives the third calibration audio signal;
determining equipment coordinates of each loudspeaker according to the third calibration time difference and the initial microphone coordinates, and controlling the loudspeaker to emit a fourth calibration audio signal;
and determining a fourth calibration time difference of a fourth calibration receiving time point of each microphone for receiving the fourth calibration audio signal, and determining the microphone coordinates according to the fourth calibration time difference and the equipment coordinates.
When the processor 1001 calls the sound source position determining program stored in the memory 1005, the following operations are also performed:
acquiring equipment coordinates of the loudspeaker;
controlling each loudspeaker to send out a calibrated audio signal;
determining a calibration time difference of a calibration receiving time point of each microphone for receiving the calibration audio signal;
and determining the microphone coordinates according to the calibration time difference and the equipment coordinates.
The embodiments of the present invention provide embodiments of a method for determining a location of a sound source, and it should be noted that although a logic sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown or described herein.
As shown in fig. 2, in an embodiment of the present invention, the method for determining a sound source position is applied to a headset, where the headset includes at least four microphones, each microphone is distributed on the headset, and the position of a sound source emitting an audio signal in a real environment can be located by each microphone, and this embodiment is illustrated by taking four microphones as an example. The sound source position determining method comprises the following steps:
step S210: and determining the microphone coordinates of the microphone in a space coordinate system according to the equipment coordinates of the preset sound source equipment.
The preset sound source device is a device capable of emitting sound signals with different frequencies through control, and the sound signals emitted by the preset sound source device are called as calibrated audio signals in the embodiment. The preset sound source device may or may not be provided on the head-mounted device. The device coordinates of the preset sound source device are coordinates in a large coordinate system, and when the preset sound source device is not provided on the head-mounted device, the device coordinates are known; the device coordinates are unknown when the preset sound source device is set on the head-mounted device. When the device coordinates are known, they can be obtained directly; when the device coordinates are unknown, they can be calculated from the initial microphone coordinates of the microphone in the spatial coordinate system. It will be appreciated that the spatial coordinate system is established in the space in which the headset is located, and may also be referred to as the geodetic coordinate system in the space in which the headset is located, and the position of the headset is calibrated by the spatial coordinate system. As shown in fig. 3, D in fig. 3 represents a head-mounted device, and the initial position coordinates S thereof in the space coordinate system are (X S ，Y S ，Z S ). At least 4 MIC 'S are set at the origin position of the spatial coordinate system for receiving sounds emitted from the head-mounted device, so that the absolute position of the point S on the head-mounted device in space can be calculated by coordinates of at least 4 MIC' S. Wherein at least 4 MIC are dedicatedThe door is used to detect the spatial position and attitude of the headset.
Since the spatial position and posture of the head-mounted device change with the movement of the head of the user after the user wears the head-mounted device, after the device coordinates of the preset sound source device are determined, the microphone coordinates of the microphone in the spatial coordinate system can be calculated according to the device coordinates of the preset sound source device.
Step S220: and determining the time difference of the receiving time points of the microphone for receiving the audio signals sent by the sound source to be positioned.
The audio signal is emitted by a sound source to be positioned in a real environment, wherein the sound source to be positioned is a thing which can emit sound, such as a person, an animal, a device which can play sound, and the like, and the audio signal can be a sound signal of the person, a music audio signal, a whistling sound signal of an automobile, a sound signal of water flow, and the like.
After the user wears the headset and turns on the headset, the headset keeps each microphone on, and each microphone detects audio signals in real-time in a real-time environment. When each microphone receives the same audio signal, the time difference between the receiving time points of the audio signals received by each microphone can be calculated according to the receiving time points of the audio signals received by each microphone, and the time difference can be used for representing the time difference of the same audio signal reaching the flight time of each microphone from the audio source to be positioned. For example, the reception time point of the audio signal received by the microphone 1 is 8 hours 10 minutes 11 seconds, the reception time point of the audio signal received by the microphone 1 is 8 hours 10 minutes 12 seconds, the time difference between the reception time point of the audio signal received by the microphone 1 and the reception time point of the audio signal received by the microphone 2 is 1s, and then the time difference in which the audio signal reaches the time of flight of the microphone 1 and the microphone 2 from the audio source to be localized is 1s.
Step S230: and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the microphone coordinates and the time difference.
After the time difference between the microphone coordinates and the receiving time points is obtained, that is, the time difference between the microphone coordinates and the receiving time points is known, the sound source coordinates of the sound source to be positioned in the space coordinate system are unknown, and the sound source coordinates can be calculated based on the distance calculation principle between any two points in the space coordinate system.
Specifically, step S230 includes:
setting a minimum time of flight for the audio signal to reach one of the microphones;
constructing an equation set according to the sound transmission speed, the minimum flight time, the microphone coordinates and the time difference;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the solving result of the equation set.
As shown in fig. 4, MIC1-MIC4 represents four microphones provided on the head-mounted device, and when the number of microphones is 4, the distances between the respective microphones and the sound source to be positioned are respectively: MIC1P, MIC, P, MIC P and MIC4P. Microphone coordinates of MIC1-MIC4 are known, respectively: MIC1 (x 1, y1, z 1), MIC2 (x 2, y2, z 2), MIC3 (x 3, y3, z 3), MIC4 (x 4, y4, z 4), P represents an unknown point, i.e., a sound source to be localized, and sound source coordinates of the sound source to be localized are unknown, denoted as P (x 0, y0, z 0). Since x0, y0 and z0 are three variables, the time of flight of the audio signal from the source to be localized to each microphone is also unknown, i.e., time of flight, x0, y0, z0 are variables that need to be solved. Specifically, since the time difference between the receiving time points is calculated, that is, a known value, it may be assumed that the time of flight of the audio signal reaching any one of MIC1 to MIC4 is the minimum time of flight, for example, the time of flight of the audio signal reaching MIC1 is the minimum time of flight, and an equation set may be constructed according to the sound transmission speed, the minimum time of flight, the microphone coordinates of MIC1 to MIC4, and the respective time differences, where the equation set is specifically as follows:
MIC1P 2 ＝(V Air-conditioner *T min ) 2 ＝(x1-x0) 2 +(y1-y0) 2 +(z1-z0 )2 ；
MIC2P 2 ＝(V Air-conditioner *(T min +Δ t2 )) 2 ＝(x2-x0) 2 +(y2-y0) 2 +(z2-y0) 2 ；
MIC3P 2 ＝(V Air-conditioner *(T min +Δ t3 )) 2 ＝(x3-x0) 2 +(y3-y0) 2 +(z3-z0) 2 ；
MIC4P 2 ＝(V Air-conditioner *(T min +Δ t4 )) 2 ＝(x4-x0) 2 +(y4-y0) 2 +(z4-z0) 2 。
Wherein, since the transmission speed of the audio signal in the air is the same as the transmission speed of the sound in the air, V Air-conditioner Representing the sound transmission speed, T min Representing minimum time of flight, delta t2 Representing the time difference, delta, between the point in time of receipt of the audio signal received by MIC2 and the point in time of receipt of the audio signal received by MIC1 t3 Representing the time difference, delta, between the point in time of receipt of the audio signal received by MIC3 and the point in time of receipt of the audio signal received by MIC1 t4 Representing the time difference between the point in time of receipt of the audio signal received by MIC4 and the point in time of receipt of the audio signal received by MIC 1.
After the equation set is constructed, T is obtained by solving the equation set min And x0, y0 and z0, to obtain the sound source coordinate P of the sound source to be positioned, assuming x0=1, y0=2 and z0=3, and p= (1, 2, 3).
According to the technical scheme, the position of the sound source making sound is accurately calculated through the head-mounted equipment, so that a user can clearly know the specific position of the sound source making sound in the real space environment under the condition that the user wears the head-mounted equipment.
Optionally, the preset sound source device includes at least four speakers disposed on the headset, that is, at least four speakers are disposed on the headset, and in this embodiment, four speakers are illustrated as S1, S2, S3, and S4, respectively. Step S210 includes the steps of:
Determining device coordinates of each of the speakers;
controlling each loudspeaker to emit a first calibration audio signal;
determining a first calibration time difference of a first calibration receiving time point of each microphone for receiving the first calibration audio signal;
and determining the microphone coordinates according to the first calibration time difference and the equipment coordinates.
Specifically, the device coordinates of each speaker need to be calibrated in advance, so that the known device coordinates of each speaker can be obtained. After the device coordinates of the respective speakers are obtained, sounds are emitted through the respective speakers to determine microphone coordinates of the microphone in the spatial coordinate system. It should be understood that, after each speaker is controlled to emit a sound signal with a different frequency, the sound signal is represented as a first calibration audio signal, and each microphone receives the sound signal with a different frequency emitted by each speaker, a first calibration receiving time point for determining that each microphone receives the first calibration audio signal may be obtained, and a first calibration time difference between the first calibration receiving time points may also be calculated by the first calibration receiving time point. The purpose of controlling each loudspeaker to emit sound signals with different frequencies is to distinguish which loudspeaker emits the corresponding first calibration audio signal through the frequency of the first calibration audio signal, and the frequency of the first calibration audio signal is the sound frequency outside the audible range of human ears, namely, when a user uses the experience head-mounted equipment, the user cannot hear the sound emitted by the loudspeaker, so that noise is brought to the user when the user uses the experience head-mounted equipment, and the use experience is prevented from being influenced.
The first calibration time difference between the first calibration receiving time points and the equipment coordinates of the loudspeaker are obtained, and the microphone coordinates of each microphone in the space coordinate system can be calculated based on the calculation principle of the sound source coordinates of the sound source in the space coordinate system. As shown in fig. 5, S1-S4 represent 4 speakers, and the device coordinates of S1-S4 are known, respectively s1= (a 1, b1, c 1), s2= (a 2, b2, c 2), s3= (a 3, b3, c 3), s4= (a 4, b4, c 4); MIC1-MIC4 represents four microphones provided on the headset, and microphone coordinates of MIC1-MIC4 are unknown, i.e., m1= (i 1, j1, k 1), m2= (i 2, j2, k 2), m3= (i 3, j3, k 3), m4= (i 4, j4, k 4), respectively. The step of determining the microphone coordinates according to the first calibration time difference and the device coordinates comprises: setting a minimum time of flight for the first nominal audio signal to reach one of the microphones; constructing an equation set according to the sound transmission speed, the minimum flight time, the equipment coordinates and the first calibration time difference; and determining the microphone coordinates according to the solving result of the equation set. Assuming that the time of flight of the first nominal audio signal to reach any one of MIC1-MIC4 is the minimum time of flight, for example, the time of flight of the first nominal audio signal to reach MIC1 is the minimum time of flight, a system of equations for calculating the microphone coordinates of MIC1 is constructed as follows:
(V Air-conditioner *T1 min ) 2 ＝(a1-i1) 2 +(b1-j1) 2 +(c1-c1) 2 ；
(V Air-conditioner *(T1 min +Δ t12 )) 2 ＝(a2-i1) 2 +(b2-j1) 2 +(c2-c1) 2 ；
(V Air-conditioner *(T1 min +Δ t13 )) 2 ＝(a3-i1) 2 +(b3-j1) 2 +(c3-c1) 2 ；
(V Air-conditioner *(T1 min +Δ t14 )) 2 ＝(a4-i1) 2 +(b4-j1) 2 +(c4-c1) 2 。
Wherein V is Air-conditioner Representing the sound transmission speed, T1 min Representing minimum time of flight, delta t12 Representing the time difference, delta, between the point in time of receipt of the first nominal audio signal received by MIC2 and the point in time of receipt of the first nominal audio signal received by MIC1 t13 Representing the time difference, delta, between the point in time of receipt of the first nominal audio signal received by MIC3 and the point in time of receipt of the first nominal audio signal received by MIC1 t14 Representing the time difference between the point in time of receipt of the first nominal audio signal received by MIC4 and the point in time of receipt of the first nominal audio signal received by MIC 1.
After an equation set for calculating the microphone coordinates of the MIC1 is constructed, obtaining T1min, i1, j1 and k1 by solving the equation set, and obtaining the microphone coordinates M1 of the MIC 1. In addition, the manner of calculating the microphone coordinates of MIC2-MIC4 is the same as that of MIC1, and will not be described here again. Through the calculation mode, the microphone coordinates of MIC1-MIC4 can be obtained through calculation.
Optionally, the step of determining the device coordinates of each of the speakers includes:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
Controlling each loudspeaker to emit a second calibration audio signal, and determining a second calibration time difference of a second calibration receiving time point when the microphone receives the second calibration audio signal;
and determining the equipment coordinates of each loudspeaker according to the second calibration time difference and the initial microphone coordinates.
Specifically, a spatial coordinate system is constructed with the center of the set position of each microphone as the origin, that is, the initial microphone coordinates of each microphone in the spatial coordinate system are known, that is, the initial microphone coordinates of each microphone in the spatial coordinate system can be obtained. And controlling each loudspeaker to emit sound signals with different frequencies, wherein the sound signals are expressed as second calibration audio signals, and after each microphone receives the sound signals with different frequencies emitted by each loudspeaker, a second calibration receiving time point for determining that each microphone receives the second calibration audio signals can be obtained, and likewise, a second calibration time difference between the second calibration receiving time points can be calculated through the second calibration receiving time points. The purpose of controlling each speaker to emit sound signals with different frequencies is to distinguish which speaker emits the corresponding second calibration audio signal through the frequency of the second calibration audio signal, and the frequency of the second calibration audio signal is the sound frequency outside the audible range of human ears, namely, when the user uses the experience head-mounted equipment, the user cannot hear the sound emitted by the speaker, so that the noise brought to the user when the user uses the experience head-mounted equipment can be avoided, and the use experience is influenced.
After the second calibration time difference and the initial microphone coordinates are obtained, the device coordinates of each speaker may be calculated according to the method for calculating microphone coordinates of the microphone described above. That is, the step of determining the device coordinates of each of the speakers based on the second calibration time difference and the initial microphone coordinates comprises: setting a minimum time of flight for the second calibration to reach one of the microphones; constructing an equation set according to the sound transmission speed, the minimum flight time, the initial microphone coordinates and the second calibration time difference; and determining the equipment coordinates of each loudspeaker according to the solving result of the equation set.
Assuming that the time of flight of the second nominal audio signal to reach any one of MIC1-MIC4 is the minimum time of flight, for example, the time of flight of the second nominal audio signal to reach MIC1 is the minimum time of flight, the device coordinates of S1-S4 are unknown, S '1= (a' 1, b '1, c' 1), S '2= (a' 2, b '2, c' 2), S '3= (a' 3, b '3, c' 3), S '4= (a' 4, b '4, c' 4), respectively; MIC1-MIC4 represents four microphones provided on the headset, and the initial microphone coordinates of MIC1-MIC4 are known, i.e., M '1 = (i' 1, j '1, k' 1), M '2 = (i' 2, j '2, k' 2), M '3 = (i' 3, j '3, k' 3), M '4 = (i' 4, j '4, k' 4), respectively.
The set of equations for calculating the device coordinates S'1 for S1 is constructed as follows:
(V air-conditioner *T1′ min ) 2 ＝(i′1-a′1) 2 +(j′1-b′1) 2 +(,k′1-c′1) 2 ；
(V Air-conditioner *(T1′ min +Δ′ t12 )) 2 ＝(i′2-a′1) 2 +(j′2-b′1) 2 +(,k′2-c′1) 2 ；
(V Air-conditioner *(T1′ min +Δ′ t13 )) 2 ＝(i′3-a′1) 2 +(j′3-b′1) 2 +(,k′3-c′1) 2 ；
(V Air-conditioner *(T1′ min +Δ′ t14 )) 2 ＝(i′4-a′1) 2 +(j′4-b′1) 2 +(,k′4-c′1) 2 。
V Air-conditioner Representing the sound transmission speed, T1' min Representing minimum time of flight, delta' t12 Representing the time difference, delta ', between the point in time of receipt of the second nominal audio signal received by MIC2 and the point in time of receipt of the second nominal audio signal received by MIC 1' t13 Representing the time difference, delta ', between the point in time of receipt of the second nominal audio signal received by MIC3 and the point in time of receipt of the second nominal audio signal received by MIC 1' t14 Representing the time difference between the point in time of receipt of the second nominal audio signal received by MIC4 and the point in time of receipt of the second nominal audio signal received by MIC 1.
After constructing the equation set for calculating the equipment coordinate S '1 of S1, T1' is obtained by solving the equation set ' min A '1, b'1, c '1, the device coordinates S'1 of S1 are obtained. In addition, the manner of calculating the device coordinates of S2 to S4 is the same as that of calculating the device coordinates of S1, and will not be described here again. The equipment coordinates of S1-S4 can be obtained through calculation through the calculation mode.
Optionally, the step of controlling each of the speakers to emit the first calibration audio signal includes:
Constructing a motion coordinate system based on the center of the headset;
and controlling a loudspeaker positioned on a coordinate axis of the motion coordinate system and a loudspeaker positioned outside a coordinate axis plane to emit the first calibration audio signal.
After the device coordinates of S1-S4 are obtained, microphone coordinates of 4 microphones on the headset are calculated from the known device coordinates of S1-S4. And constructing a motion coordinate system oxyz by taking the center of the headset as an origin, so that any three speakers in S1-S4 are positioned on the x axis, the y axis and the z axis of the motion coordinate system oxyz, and the rest speakers are positioned outside the coordinate axis plane. As shown in FIG. 5, S1, S2, S3 are respectively located on the x-axis, the y-axis and the z-axis, S4 is located outside the oxy plane, and if the distance between S1-S4 and the origin 0 is known, the S1-S4 is controlled to send out first calibration audio signals at different frequencies, so that microphone coordinates of MIC1-MIC4 are calculated.
Optionally, the preset sound source device includes at least one speaker disposed on the headset, and since the speaker and each microphone are disposed on the headset, a geometric positional relationship such as a direction of the microphone relative to the speaker is fixed, which belongs to known data. Step S210 further includes the steps of:
Constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
controlling the loudspeaker to emit a third calibration audio signal, and determining a third calibration time difference of a third calibration receiving time point when the microphone receives the third calibration audio signal;
determining equipment coordinates of the loudspeaker according to the third calibration time difference and the initial microphone coordinates;
and acquiring the geometric position relation between the loudspeaker and each microphone on the head-mounted equipment, and determining the microphone coordinates according to the equipment coordinates and the geometric position relation.
Specifically, a spatial coordinate system is constructed with the center of the set position of each microphone as the origin, that is, the initial microphone coordinates of each microphone in the spatial coordinate system are known, that is, the initial microphone coordinates of each microphone in the spatial coordinate system can be obtained. And controlling the loudspeaker to emit sound signals, wherein the sound signals are expressed as third calibration audio signals, after each microphone receives the sound signals emitted by the loudspeaker, a third calibration receiving time point for determining that each microphone receives the third calibration audio signals can be obtained, and the third calibration time difference between the third calibration receiving time points can be calculated through the third calibration receiving time points. The frequency of the third calibration audio signal is the sound frequency outside the audible range of the human ear, namely, when the user uses the experience head-mounted equipment, the user cannot hear the sound emitted by the loudspeaker, so that noise brought to the user when the user uses the experience head-mounted equipment can be avoided, and the use experience is influenced.
After the third calibration time difference and the initial microphone coordinate are obtained, the device coordinate of the speaker may be calculated according to the calculation method of S '1, where the device coordinate of the speaker is the same as the calculation method of S'1, and will not be described herein. After obtaining the device coordinates of the speaker, the geometric positional relationship between the speaker and each microphone on the headset is obtained, i.e. the geometric positional relationship includes the distance between the speaker and the microphone, the direction of the microphone relative to the speaker, and the like. Then, the microphone coordinates of the respective microphone coordinates are calculated from the device coordinates of the speaker and the geometric positional relationship between the speaker and the respective microphones.
Optionally, the preset sound source device includes at least four speakers with known device coordinates, that is, the positions of the four speakers are always fixed, that is, the device coordinates of the four speakers are calibrated in advance, which belongs to known data. Step S210 further includes the steps of:
acquiring equipment coordinates of each loudspeaker;
controlling each loudspeaker to send out a calibrated audio signal;
determining a calibration time difference of a calibration receiving time point of each microphone for receiving the calibration audio signal;
And determining the microphone coordinates according to the calibration time difference and the equipment coordinates.
Specifically, the equipment coordinates of each loudspeaker are obtained to obtain known data, then each loudspeaker is controlled to respectively send out calibration audio signals with different frequencies, each loudspeaker is controlled to respectively send out the calibration audio signals with different frequencies, the sent out calibration audio signals can be distinguished by which loudspeaker is sent out, and the calibration audio signals sent out by each loudspeaker are inaudible to human ears. And then, obtaining the calibration receiving time points of each microphone for receiving the calibration audio signals, calculating the calibration time difference of the calibration receiving time points of each microphone for receiving the calibration audio signals through the calibration receiving time points of each microphone, and further calculating the microphone coordinates according to the calibration time difference and the equipment coordinates. The manner of calculating the microphone coordinates according to the calibration time difference and the device coordinates is the same as the manner of calculating MIC1 (x 1, y1, z 1), MIC2 (x 2, y2, z 2), MIC3 (x 3, y3, z 3), MIC4 (x 4, y4, z 4), and it is also necessary to construct an equation set including at least four variables, and the microphone coordinates of the microphone are obtained by solving the equation set, so that the construction and the solving of the specific equation set are not repeated herein.
Optionally, as shown in fig. 6, step S230 further includes the following steps:
step S240: determining the relative position coordinates of the sound source to be positioned relative to the head-mounted equipment according to the sound source coordinates and the microphone coordinates;
step S250: determining playing parameters corresponding to the audio signals according to the relative position coordinates, and playing the audio signals based on the playing parameters; and/or marking the position of the sound source to be positioned in the real environment picture displayed by the head-mounted equipment according to the relative position coordinates.
The purpose of establishing the spatial coordinate system is to have an absolute reference position for the head-mounted device and the sound source to be positioned, and an absolute reference point after the head-mounted device moves and the position of the sound source to be positioned changes. After the sound source coordinates of the sound source to be positioned in the space coordinate system and the microphone coordinates of each microphone are calculated, the positions of the head-mounted device in the space coordinate system can be determined according to the microphone coordinates of each microphone because each microphone is arranged on the head-mounted device. In addition, because the user is opposite to the sound source to be positioned in the real space environment, the user can hear the audio signal from the sound source to be positioned, in order to enable the user to feel that the azimuth of the audio signal heard when wearing the head-mounted device is identical to the azimuth in the real space environment, the relative position coordinate of the sound source to be positioned relative to the head-mounted device is calculated through the sound source coordinate and the microphone coordinate, and then the playing parameters of the audio signal are obtained according to the relative position coordinate, wherein the playing parameters comprise the azimuth, the frequency, the loudness and the like of the audio signal. And then controlling the head-mounted device to play the audio signal according to the play parameters based on the sound follow-up technology, wherein the sound heard by the user is in the same direction as the sound heard by the user when the user does not wear the head-mounted device in the real space environment when the user uses the head-mounted device to watch the displayed real environment picture, namely the direction of the sound heard by the user when the user wears the quasi-real device is the same as the direction of the sound heard by the user when the user does not wear the head-mounted device in the real world.
In addition, after the camera on the head-mounted device collects the real-time image of the environment, the image is subjected to a series of processing, and finally, the real-world environment picture is displayed on the display of the head-mounted device, and the process can enable the information of the real-world environment seen by the user using the head-mounted device to be identical to the information of the real-world environment seen when the head-mounted device is removed. When the head-mounted device presents a real environment picture for a user, the position of the sound source to be positioned is marked in the real environment picture displayed by the head-mounted device, the sound source to be positioned is presented, the user can clearly see the position of the sound source to be positioned marked in the real environment picture, namely, the specific sound is emitted from the position, and the marked position displays the identifier of the playing sound. When a user wears the head-mounted device, the user can feel the actual source of sound through the head-mounted device without picking off the head-mounted device, namely, the user can clearly know the specific position of the sound source emitting the sound in the real space environment through wearing the head-mounted device, so that the sound feeling of the real world is embodied in the virtual world, and the man-machine interaction experience is greatly enhanced. Notably, when wearing the head-mounted device, the user can play the audio signal of the sound source, and can mark the position of the sound source in the displayed real environment picture at the same time; or when the user wears the head-mounted equipment, the head-mounted equipment plays the audio signal of the sound playing source; or when wearing the head-mounted equipment, the user marks the position of the sound source in the displayed real environment picture.
Further, the present invention also provides a head-mounted device, including: the sound source position determining device comprises a memory, a processor and a sound source position determining program which is stored in the memory and can run on the processor, wherein the sound source position determining program realizes the steps of the sound source position determining method when being executed by the processor.
Further, the present invention also provides a storage medium having stored thereon a sound source position determining program which, when executed by a processor, implements the steps of the sound source position determining method described above.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. A sound source position determining method, characterized by being applied to a head-mounted device, the head-mounted device including at least four microphones, a preset sound source device being provided on the head-mounted device, the sound source position determining method comprising:
according to the equipment coordinates of the preset sound source equipment, determining the microphone coordinates of the microphone in a space coordinate system, wherein the space coordinate system is established in the space where the head-mounted equipment is located;
when the preset sound source device comprises at least four loudspeakers arranged on the head-mounted device and the at least four loudspeakers are unknown, constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system; controlling each loudspeaker to send out a second calibration audio signal, and determining a second calibration time difference of a second calibration receiving time point when the microphone receives the second calibration audio signal, wherein the second calibration audio signal of each loudspeaker is a sound signal with different frequencies; determining equipment coordinates of each loudspeaker according to the second calibration time difference and the initial microphone coordinates;
Determining a time difference of receiving time points of the microphone for receiving the audio signals sent by the sound source to be positioned;
according to the microphone coordinates and the time difference, determining sound source coordinates of the sound source to be positioned in the space coordinate system;
determining the relative position coordinates of the sound source to be positioned relative to the head-mounted equipment according to the sound source coordinates and the microphone coordinates;
and determining playing parameters corresponding to the audio signals according to the relative position coordinates, and playing the audio signals based on the playing parameters, so that the sound heard by the user is in the same direction as the sound heard by the user when the user uses the head-mounted device to watch the displayed real environment picture and the user does not wear the head-mounted device in the real space environment.
2. The method of claim 1, wherein the preset sound source device comprises at least four speakers disposed on the head mounted device, and wherein the step of determining microphone coordinates of the microphone in a spatial coordinate system based on device coordinates of the preset sound source device comprises:
determining device coordinates of each of the speakers;
controlling each loudspeaker to emit a first calibration audio signal;
Determining a first calibration time difference of a first calibration receiving time point of each microphone for receiving the first calibration audio signal;
and determining the microphone coordinates according to the first calibration time difference and the equipment coordinates.
3. The method of claim 2, wherein the step of controlling each of the speakers to emit a first nominal audio signal comprises:
constructing a motion coordinate system based on the center of the headset;
and controlling a loudspeaker positioned on a coordinate axis of the motion coordinate system and a loudspeaker positioned outside a coordinate axis plane to emit the first calibration audio signal.
4. The method of claim 1, wherein the step of determining the sound source coordinates of the sound source to be positioned in the spatial coordinate system based on the microphone coordinates and the time difference comprises:
setting a minimum time of flight for the audio signal to reach one of the microphones;
constructing an equation set according to the sound transmission speed, the minimum flight time, the microphone coordinates and the time difference;
and determining the sound source coordinates of the sound source to be positioned in the space coordinate system according to the solving result of the equation set.
5. The method of claim 1, wherein after the step of determining the relative position coordinates of the sound source to be positioned with respect to the headset based on the sound source coordinates and the microphone coordinates, further comprises:
and marking the position of the sound source to be positioned in a real environment picture displayed by the head-mounted equipment according to the relative position coordinates.
6. The method of claim 1, wherein the preset sound source device includes at least one speaker disposed on the head mounted device, the step of determining microphone coordinates of the microphone in a spatial coordinate system based on device coordinates of the preset sound source device, further comprising:
constructing a space coordinate system based on each microphone, and acquiring initial microphone coordinates of each microphone in the space coordinate system;
controlling the loudspeaker to emit a third calibration audio signal, and determining a third calibration time difference of a third calibration receiving time point when the microphone receives the third calibration audio signal;
determining equipment coordinates of the loudspeaker according to the third calibration time difference and the initial microphone coordinates;
And acquiring the geometric position relation between the loudspeaker and each microphone on the head-mounted equipment, and determining the microphone coordinates according to the equipment coordinates and the geometric position relation.
7. The method of claim 1, wherein the preset sound source device comprises at least four speakers of known device coordinates, the step of determining microphone coordinates of the microphone in a spatial coordinate system based on the device coordinates of the preset sound source device, further comprising:
acquiring equipment coordinates of each loudspeaker;
controlling each loudspeaker to send out a calibrated audio signal;
determining a calibration time difference of a calibration receiving time point of each microphone for receiving the calibration audio signal;
and determining the microphone coordinates according to the calibration time difference and the equipment coordinates.
8. A headset, the headset comprising: a memory, a processor and a sound source position determining program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the sound source position determining method according to any one of claims 1-7.
9. A storage medium having stored thereon a sound source position determining program which, when executed by a processor, implements the steps of the sound source position determining method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575064.9A CN115002607B (en) | 2022-05-25 | 2022-05-25 | Sound source position determining method, device and storage medium |
| PCT/CN2022/103889 WO2023226161A1 (en) | 2022-05-25 | 2022-07-05 | Sound source position determination method, device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575064.9A CN115002607B (en) | 2022-05-25 | 2022-05-25 | Sound source position determining method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115002607A CN115002607A (en) | 2022-09-02 |
| CN115002607B true CN115002607B (en) | 2023-12-22 |
Family
ID=83028316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210575064.9A Active CN115002607B (en) | 2022-05-25 | 2022-05-25 | Sound source position determining method, device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115002607B (en) |
| WO (1) | WO2023226161A1 (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09182044A (en) * | 1995-12-25 | 1997-07-11 | Matsushita Electric Ind Co Ltd | Television conference system |
| KR20110121304A (en) * | 2010-04-30 | 2011-11-07 | 주식회사 에스원 | Apparatus for calculating sound source location and method thereof |
| JP2016114426A (en) * | 2014-12-12 | 2016-06-23 | 株式会社熊谷組 | Sound source direction estimation method |
| CN108429998A (en) * | 2018-03-29 | 2018-08-21 | 广州视源电子科技股份有限公司 | Source of sound localization method and system, sound box system localization method and sound box system |
| CN109164414A (en) * | 2018-09-07 | 2019-01-08 | 深圳市天博智科技有限公司 | Localization method, device and storage medium based on microphone array |
| CN112672251A (en) * | 2020-12-25 | 2021-04-16 | Tcl通力电子（惠州）有限公司 | Control method and system of loudspeaker, storage medium and loudspeaker |
| CN113702909A (en) * | 2021-08-30 | 2021-11-26 | 浙江大学 | Sound source positioning analytic solution calculation method and device based on sound signal arrival time difference |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7522736B2 (en) * | 2004-05-07 | 2009-04-21 | Fuji Xerox Co., Ltd. | Systems and methods for microphone localization |
| CN104422922A (en) * | 2013-08-19 | 2015-03-18 | 中兴通讯股份有限公司 | Method and device for realizing sound source localization by utilizing mobile terminal |
| CN105797366A (en) * | 2016-03-25 | 2016-07-27 | 中国传媒大学 | Head-wearing type interactive audio game terminal based on sound source location |
| EP3477964B1 (en) * | 2017-10-27 | 2021-03-24 | Oticon A/s | A hearing system configured to localize a target sound source |
| CN208334627U (en) * | 2018-05-15 | 2019-01-04 | 上海真曦通信技术有限公司 | A kind of electronic system for auditory localization |
| CN108802689B (en) * | 2018-06-14 | 2020-09-01 | 河北工业大学 | Space microphone positioning method based on sound source array |
| US10674259B2 (en) * | 2018-10-26 | 2020-06-02 | Facebook Technologies, Llc | Virtual microphone |
| CN109788382A (en) * | 2019-01-25 | 2019-05-21 | 深圳大学 | A kind of distribution microphone array pickup system and method |
-
2022
- 2022-05-25 CN CN202210575064.9A patent/CN115002607B/en active Active
- 2022-07-05 WO PCT/CN2022/103889 patent/WO2023226161A1/en unknown
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09182044A (en) * | 1995-12-25 | 1997-07-11 | Matsushita Electric Ind Co Ltd | Television conference system |
| KR20110121304A (en) * | 2010-04-30 | 2011-11-07 | 주식회사 에스원 | Apparatus for calculating sound source location and method thereof |
| JP2016114426A (en) * | 2014-12-12 | 2016-06-23 | 株式会社熊谷組 | Sound source direction estimation method |
| CN108429998A (en) * | 2018-03-29 | 2018-08-21 | 广州视源电子科技股份有限公司 | Source of sound localization method and system, sound box system localization method and sound box system |
| CN109164414A (en) * | 2018-09-07 | 2019-01-08 | 深圳市天博智科技有限公司 | Localization method, device and storage medium based on microphone array |
| CN112672251A (en) * | 2020-12-25 | 2021-04-16 | Tcl通力电子（惠州）有限公司 | Control method and system of loudspeaker, storage medium and loudspeaker |
| CN113702909A (en) * | 2021-08-30 | 2021-11-26 | 浙江大学 | Sound source positioning analytic solution calculation method and device based on sound signal arrival time difference |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023226161A1 (en) | 2023-11-30 |
| CN115002607A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11617050B2 (en) | Systems and methods for sound source virtualization | |
| US9906885B2 (en) | Methods and systems for inserting virtual sounds into an environment | |
| US9781538B2 (en) | Multiuser, geofixed acoustic simulations | |
| US10129681B2 (en) | Calibrating listening devices | |
| US11057730B2 (en) | Methods and apparatuses for implementing a head tracking headset | |
| US20190313201A1 (en) | Systems and methods for sound externalization over headphones | |
| US20200084564A1 (en) | Methods and Systems for Providing a Composite Audio Stream for an Extended Reality World | |
| CN108141696A (en) | The system and method adjusted for space audio | |
| CN107360494A (en) | A kind of 3D sound effect treatment methods, device, system and sound system | |
| US10757528B1 (en) | Methods and systems for simulating spatially-varying acoustics of an extended reality world | |
| WO2017003472A1 (en) | Shoulder-mounted robotic speakers | |
| WO2021169689A1 (en) | Sound effect optimization method and apparatus, electronic device, and storage medium | |
| US20230345198A1 (en) | Sharing Locations where Binaural Sound Externally Localizes | |
| CN116076091A (en) | Spatialization audio with respect to mobile peripheral devices | |
| CN110915240B (en) | Method for providing interactive music composition to user | |
| US11678111B1 (en) | Deep-learning based beam forming synthesis for spatial audio | |
| Mariette | Human factors research in audio augmented reality | |
| CN115002607B (en) | Sound source position determining method, device and storage medium | |
| US10667073B1 (en) | Audio navigation to a point of interest | |
| US10820132B2 (en) | Voice providing device and voice providing method | |
| US20220279303A1 (en) | Spatial audio for wearable devices | |
| CN113810817B (en) | Volume control method and device of wireless earphone and wireless earphone | |
| US20240089687A1 (en) | Spatial audio adjustment for an audio device | |
| WO2024014390A1 (en) | Acoustic signal processing method, information generation method, computer program and acoustic signal processing device | |
| WO2024084949A1 (en) | Acoustic signal processing method, computer program, and acoustic signal processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |