JP5412470B2 - Position measurement system - Google Patents

Description

  The present invention relates to a microphone array / network system for obtaining high-quality sound and a position measurement system using the microphone array / network system.

  Conventionally, in an application system using voice (for example, a voice conference system in which a plurality of microphones are connected, a voice recognition robot system, a system having various voice interfaces, etc.) Various sound processing such as sound source localization, sound source separation, noise removal, and echo cancellation are performed.

  In particular, for the purpose of acquiring high-quality sound, a microphone array mainly used for sound source localization and sound source separation has been widely studied. Here, sound source localization is to specify the direction and position of the sound source from the difference in arrival time of the sound, and sound source separation is to use the result of sound source localization to eliminate the sound source that becomes noise and to specify a specific sound source in a specific direction Is to perform the extraction.

  It is known that sound processing using a microphone array usually improves sound processing performance such as noise processing as the number of microphones increases. In such audio processing, there are many sound source localization methods that use sound source position information (see, for example, Non-Patent Document 1). The more accurate the sound source localization result, the more effective the sound processing. That is, it is necessary to simultaneously increase the accuracy of sound source localization and remove noise for high sound quality by increasing the number of microphones.

  In the case of sound source localization using a conventional large-scale microphone array, as shown in FIG. 10, the position range of the sound source is divided into a mesh shape, and the sound source position is obtained probabilistically for each section. In this calculation, all sound data is collected in one sound processing server such as a workstation, and all sound data is collectively processed to estimate the position of the sound source (see, for example, Non-Patent Document 2). . In such batch processing of all audio data, the signal wiring length between the microphone and the audio processing server for collecting audio, the communication amount, and the calculation amount in the audio processing server are enormous. There is a problem that the number of microphones cannot be increased due to an increase in wiring length, communication amount, computation amount in the voice processing server, and physical restrictions that a large number of A / D converters cannot be arranged in one place of the voice processing server. There is also a problem of noise generation due to a long signal wiring length. Therefore, there has been a problem that it is difficult to increase the number of microphones for pursuing high sound quality.

  As a method for improving such a problem, a sound processing system using a microphone array that divides a plurality of microphones into small arrays and integrates them is known (see, for example, Non-Patent Document 3).

  However, even in the case of such a voice processing system, voice data of all microphones acquired in a small array are integrated into one voice server via the network, so that there is a problem of increase in network communication traffic. In addition, there is a problem that voice processing delay occurs with an increase in communication data volume and communication traffic volume.

  In the future, more microphones will be required to meet demands for sound collection and video conference systems in ubiquitous systems (see, for example, Patent Document 1). However, as described above, in the current microphone array network system, the audio data obtained by the microphone array is merely transferred to the server as it is. There is no system that can reduce the calculation amount of the entire system and the communication amount of the network by exchanging the position information of the sound sources with each other in the microphone array.

  Accordingly, assuming a large-scale microphone array network system, a system architecture that reduces the amount of calculation of the entire system and suppresses the amount of network communication is important.

  As described above, it is required to increase sound source localization accuracy by using a large number of microphone arrays and to effectively perform sound processing such as noise removal while suppressing the communication amount and calculation amount in the sound processing server.

  Recently, a position measurement system using a sound source has been proposed. For example, Patent Document 2 discloses calculating an ultrasonic tag using an ultrasonic tag and a microphone array. Furthermore, Patent Document 3 discloses that sound collection is performed using a microphone array.

JP 2008-113164 A International Publication No. 2008/026463 Pamphlet JP 2008-048342 A

R.O.Schmidt, "Multiple emitter location and signal parameter estimation", In Proceedings of the RADC Spectrum Estimation Workshop, pp.243-248, October 1979. E. Weinstein et al., "Loud: A 1020-node modular microphone array and beamformer for intelligent computing spaces", MIT, MIT / LCS Technical Memo MIT-LCS-TM-642, April 2004. A. Brutti et al., "Classification of Acoustic Maps to Determine Speaker Position and Orientation from a Distributed Microphone Network", In Proceedings of ICASSP, Vol. IV, pp. 493-496, April. 2007.

  However, the position measurement function of GPS systems and WiFi systems installed in many mobile terminals cannot acquire the positional relationship between terminals at a short distance of several tens of centimeters, even if it can acquire a rough position on the map. There was a problem.

  An object of the present invention is to solve the above problems and to provide a position measurement system capable of measuring the position of a terminal with higher accuracy than in the prior art.

A position measurement system according to the present invention includes a control unit that estimates a position of each node using a microphone array network system in which a plurality of nodes including a microphone array are connected to each other via a network and time-synchronized. A position measuring system,
Each of the above nodes
A microphone array configured by arranging a plurality of microphones in an array;
A sound source estimation processing unit that estimates the angle of the arrival direction of the sound signal based on the sound signal from one node received by the microphone array;
A distance estimation unit that estimates a distance from a node that has transmitted the sound signal, based on a difference between a transmission time and a reception time of the sound signal and a speed of the sound signal;
A data communication unit that transmits and receives the estimated angle and distance of the direction of arrival to other nodes by data communication,
The controller is configured to estimate and calculate the position of each node based on the estimated angle and distance of the arrival direction.

  In the position measurement system, the control unit calculates and displays or outputs a coordinate value based on the position of each node.

  In the position measurement system, the control unit executes a predetermined application using the position of each node.

  Further, in the position measurement system, each of the nodes includes a display unit, and the control unit in the predetermined application, based on the calculated position of each node, the direction with respect to each node on the display unit And instructing the node to be transmitted by dragging it toward the node to which the predetermined data is to be transmitted on the display unit, and transmitting the predetermined data to the instructed node It is characterized by.

  Still further, in the position measurement system, the control unit is provided in each of the nodes, one of the plurality of nodes, or a server device different from the plurality of nodes. .

  Therefore, according to the present invention, it is possible to provide a position measurement system capable of measuring the position of the terminal with higher accuracy than in the prior art.

It is explanatory drawing which shows the structure of the sound source localization system which concerns on the 1st Embodiment of this invention. It is explanatory drawing explaining the two-dimensional sound source localization in the sound source localization system of FIG. It is explanatory drawing explaining the three-dimensional sound source localization in the sound source localization system of FIG. 1 is a configuration diagram showing a configuration of a microphone array network system according to Embodiment 1 of the present invention. It is a block diagram which shows the structure of the node provided with the microphone array of FIG. It is a functional diagram which shows the function of the microphone array network system of FIG. FIG. 5 is an explanatory diagram for explaining an experiment of three-dimensional sound source localization accuracy in the microphone array network system of FIG. 4. 6 is a graph showing measurement results showing improvement in three-dimensional sound source localization accuracy in the microphone array network system of FIG. 4. It is a block diagram which shows the structure of the microphone array network system which concerns on Example 2 of this invention. It is explanatory drawing explaining the sound source localization system which concerns on a prior art. It is a block diagram which shows the detailed structure of the node used with the position measurement system which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the network used with the position measuring system of FIG. It is a graph which shows the time synchronization with linear interpolation used with the position measurement system of FIG. FIG. 13 is a first part of a timing chart showing a signal transmission procedure between each tablet and each process executed in each tablet in the position measurement system of FIG. 12. FIG. 13 is a second part of a timing chart showing a signal transmission procedure between each tablet and each process executed in each tablet in the position measurement system of FIG. 12. It is a top view which shows the method of measuring the distance between each tablet from the angle information measured with each tablet of the position measuring system of FIG. It is a top view which shows the concept of the message exchange system using the position measurement system of FIG. It is a display example of a display showing one application of the message exchange system of FIG. It is a top view which shows the concept of the hockey game system using the position measurement system of FIG. It is a display example of the execution screen of the server apparatus of the hockey game system of FIG. 18 is a display example of a tablet when the hockey game system of FIG. 17 is being executed.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, in each following embodiment, the same code | symbol is attached | subjected about the same component.

(First embodiment)
FIG. 1 shows an explanatory diagram of sound source localization according to the present invention. As shown in FIG. 1, six nodes having a microphone array and one voice processing server 20 are connected by a network 10. Six nodes having a microphone array configured by arranging a plurality of microphones in an array form are present on four wall surfaces in the room, and the sound source direction is estimated by the sound collection processor in each node. And the position of the sound source is specified by integrating the result into the voice processing server. Since each node processes data, the amount of network communication can be reduced, and the amount of computation is distributed among the nodes.

  In the following, a detailed description will be given separately for two-dimensional sound source localization and three-dimensional sound source localization. First, the two-dimensional sound source localization method of the present invention will be described with reference to FIG. FIG. 2 illustrates a two-dimensional sound source localization method. As shown in FIG. 2, the nodes 1 to 3 estimate the sound source direction from the collected sound signals collected from the respective microphone arrays. Each node calculates the response intensity of the MUSIC method for each direction, and estimates the direction that takes the maximum value as the sound source direction. In FIG. 2, node 1 calculates the response intensity with respect to directions from −90 ° to 90 °, with the perpendicular direction (front direction) of the arrangement surface of the microphone array being 0 °, and the direction of θ1 = −30 ° Is assumed to be the sound source direction. Similarly, the node 2 and the node 3 also calculate the response intensity for each direction, and estimate the direction that takes the maximum value as the sound source direction.

Then, weighting is performed on the intersection of the sound source direction estimation results of the two nodes, such as node 1 and node 2 or node 1 and node 3. Here, the weight is determined based on the maximum response intensity of the MUSIC method of each node (for example, the product of the maximum response intensity of two nodes). In FIG. 2, the weight scale is expressed by the diameter of the circle at the intersection.
The obtained circles (position and scale) indicating the plurality of weights are sound source position candidates. And a sound source position is estimated by calculating | requiring the gravity center of the obtained several sound source position candidates. In the case of FIG. 2, obtaining the centroids of a plurality of sound source position candidates means obtaining the weighted centroids of circles (positions and scales) indicating a plurality of weights.

  Next, the three-dimensional sound source localization method of the present invention will be described with reference to FIG. FIG. 3 illustrates a three-dimensional sound source localization method. As shown in FIG. 3, the nodes 1 to 3 estimate the sound source direction from the collected sound signals collected from the respective microphone arrays. Each node calculates the response intensity of the MUSIC method with respect to the three-dimensional direction, and estimates the direction that takes the maximum value as the sound source direction. FIG. 3 shows a case where the node 1 calculates the response intensity with respect to the direction of the rotating coordinate system in the direction perpendicular to the arrangement surface of the microphone array (front direction), and estimates the direction with the higher intensity as the sound source direction. . Similarly, the node 2 and the node 3 also calculate the response intensity for each direction, and estimate the direction that takes the maximum value as the sound source direction.

  Then, the node 1 and the node 2, or the node 1 and the node 3, such as the node 1 and the node 3, the weight is obtained for the intersection of the sound source direction estimation results of the two nodes. Often not available. Therefore, an intersection point is virtually obtained on a line segment that connects the straight lines of the sound source direction estimation results of the two nodes at the shortest. Note that the weight is determined based on the maximum response intensity of the MUSIC method of each node (for example, the product of the maximum response intensity of two nodes) as in the case of the two dimensions. In FIG. 3, as in FIG. 2, the weight scale is represented by the diameter of the circle at the intersection.

  The obtained circles (position and scale) indicating the plurality of weights are sound source position candidates. And a sound source position is estimated by calculating | requiring the gravity center of the obtained several sound source position candidates. In the case of FIG. 3, obtaining the centroids of a plurality of sound source position candidates means obtaining the weighted centroids of circles (positions and scales) indicating a plurality of weights.

  An embodiment of the present invention will be described. FIG. 4 is a configuration diagram of the microphone array network system according to the first embodiment. FIG. 4 shows a system configuration in which nodes (1 a, 1 b,..., 1 n) having microphone arrays in which 16 microphones are arranged in an array and one audio processing server 20 are connected via a network 10. As shown in FIG. 5, each node has signal lines of microphones (m11, m12,..., M43, m44) arranged in an array of 16 input / output units (I / O) of the sound collection processing unit 2. The signal collected from the microphone is input to the processor 4 of the sound collection processing unit 2. The processor 4 of the sound collection processing unit 2 performs an MUSIC algorithm process using the input sound collection signal to estimate the sound source direction.

  Then, the processor 4 of the sound collection processing unit 2 transmits the sound source direction estimation result and the maximum response intensity to the sound processing server 20 shown in FIG.

  As described above, the sound localization is performed in a distributed manner in each node, the result is integrated into the sound processing server, the above-described two-dimensional localization and three-dimensional localization processes are performed, and the position of the sound source is estimated.

  FIG. 6 is a functional diagram of the microphone array network system according to the first embodiment.

  The node having the microphone array performs A / D conversion on the signal from the microphone array (step S11), and inputs the sound pickup signal of each microphone (step S13). Using a signal collected from each microphone, a processor mounted on the notebook estimates a sound source direction as a sound collection processing unit (step S15).

  As shown in the graph of FIG. 6, the sound collection processing unit calculates the response intensity of the MUSIC method with respect to directions from −90 ° to 90 ° left and right with the front (perpendicular direction) of the microphone array being 0 °. Then, the direction in which the response intensity is strong is estimated as the sound source direction. The sound collection processing unit is connected to the sound processing server via a network (not shown), and exchanges data of the sound source direction estimation result (A) and the maximum response intensity (B) within the node (step S17). The sound source direction estimation result (A) and the maximum response intensity (B) are sent to the voice processing server.

  The voice processing server receives data sent from each node (step S21). A plurality of sound source position candidates are calculated from the maximum response intensity of each node (step S23). Then, the position of the sound source is estimated based on the sound source direction estimation result (A) and the maximum response intensity (B) (step S25).

  Hereinafter, the three-dimensional sound source localization accuracy will be described. FIG. 7 is a schematic diagram showing the state of the experiment of the three-dimensional sound source localization accuracy. A room with a floor area of 12m x 12m and a height of 3m is assumed. An assumption was made of 16 subarrays in which 16 microphone arrays arranged in an array were arranged at equal intervals on all four sides of the floor (case A of 16 subarrays). In addition, forty-one subarrays are assumed in which 16 microphone arrays are arranged at equal intervals on four sides of the floor and 16 microphone arrays are arranged at equal intervals on the four sides of the ceiling, and nine microphone arrays are arranged at equal intervals on the floor. (41 subarray case B). In addition, 73 sub-arrays are assumed in which 32 microphone arrays are arranged at equal intervals on the floor surface and 32 microphone arrays are arranged at equal intervals on the floor surface, and nine microphone arrays are arranged on the floor surface at equal intervals. (73 subarray case C).

  Using these three cases A to C, the number of nodes and the error variation of the sound source direction estimation of each node were changed, and the results of the three-dimensional position estimation were compared. In the three-dimensional position estimation, each node randomly selects one communication partner and obtains a virtual intersection.

  The measurement results are shown in FIG. The horizontal axis in FIG. 8 indicates the variation (standard deviation) in the direction estimation error, and the vertical axis indicates the position estimation error. From the results of FIG. 8, it can be seen that the accuracy of three-dimensional position estimation can be improved by increasing the number of nodes even if the estimation accuracy of the sound source direction is poor.

  Another embodiment of the present invention will be described. FIG. 9 is a configuration diagram of the microphone array network system according to the second embodiment. FIG. 9 shows a system configuration in which nodes (1a, 1b, 1c) each having a microphone array in which 16 microphones are arranged in an array are connected by a network (11, 12). In the case of the system of the second embodiment, unlike the system configuration of the first embodiment, there is no voice processing server. In addition, as in the first embodiment, each node has signal lines of microphones (m11, m12,..., M43, m44) arranged in an array of 16 as shown in FIG. The signal collected from the microphone is input to the processor 4 of the sound collection processing unit 2. The processor 4 of the sound collection processing unit 2 performs an MUSIC algorithm process using the input sound collection signal to estimate the sound source direction.

  Then, the processor 4 of the sound collection processing unit 2 exchanges the sound source direction estimation results with the adjacent nodes and other nodes. The processor 4 of the sound collection processing unit 2 performs the above-described two-dimensional localization and three-dimensional localization processing from the sound source direction estimation results and the maximum response intensity of a plurality of nodes including its own node, and estimates the position of the sound source.

(Second Embodiment)
FIG. 11 is a block diagram showing a detailed configuration of a node used in the position measurement system according to the second embodiment of the present invention. The position measurement system according to the second embodiment uses the sound source localization system according to the first embodiment to measure the position of the terminal with higher accuracy than in the prior art. The position measurement system according to the present embodiment is constructed by using, for example, a ubiquitous network system (UNS). For example, a small-scale microphone array (sensor node) having 16 microphones is connected by a predetermined network as a whole. A position measurement system is constructed by constructing a large-scale microphone array speech processing system. Here, each sensor node is equipped with a microphone processor, and performs voice processing in a distributed and cooperative manner.

A block diagram of the sensor node is shown in FIG. Each sensor node
(1) an AD conversion circuit 51 connected to a plurality of microphones 1 for collecting sound;
(2) an utterance estimation processing unit (Voice Activity Detection: hereinafter referred to as a VAD processing unit) 52 connected to the AD conversion circuit 51 for detecting a voice signal;
(3) A sound signal or the like including a sound signal or a sound signal AD-converted by the AD conversion circuit 51 (here, the sound signal refers to an audible frequency signal such as 500 Hz or an ultrasonic signal). SRAM (Static Random Access Memory) 54 to be stored in
(4) An SSL processing unit 55 that performs sound source localization processing for estimating the position of a sound source on digital data such as an audio signal output from the SRAM 54 and outputs the result to the SSS processing unit 56; ,
(5) A sound source separation process for extracting a specific sound source is performed on digital data such as an audio signal output from the SRAM 54 and the SSL processing unit 55, and obtained as a result of those processes. SSS processing unit 56 that collects voice data having a high SNR by transmitting and receiving to / from other nodes via network interface circuit 57,
(6) A network interface circuit 57 that is connected to other surrounding sensor nodes Nn (n = 1, 2,..., N) and constitutes a data communication unit that transmits and receives audio data.

  Each sensor node Nn (n = 0, 1, 2,..., N) has the same configuration, but the sensor data N0 of the base station further increases the SNR by aggregating the voice data on the network. Audio data is obtained. The VAD processing unit 52 and the power management unit 53 are used in the sound source localization of the first embodiment, but are not used in principle in the position estimation of the second embodiment. Further, distance estimation to be described later is executed by, for example, the SSL processing unit 55.

  Here, an example of processing in each sensor node will be described below. First, all sensor nodes are in a sleep state at an initial stage, and several sensor nodes that are separated by a certain distance, for example, one sensor node transmits a sound signal for a predetermined time (for example, 3 seconds), and the sound signal The sensor node that detects the sound source starts to estimate the sound source direction by multi-channel input. At the same time, a wake-up message is broadcast to other sensor nodes existing in the vicinity, and the received sensor node immediately starts sound source direction estimation. After completing the sound source direction estimation, each sensor node transmits the estimation result to the base station (the sensor node connected to the server device). The base station estimates the sound source position using the collected direction estimation results of each sensor node, and broadcasts the results to all sensor nodes that have performed sound source direction estimation. Next, each sensor node performs sound source separation using the position estimation result received from the base station. Similar to the sound source localization, sound source separation is executed in two stages within the sensor node and between the sensor nodes. The voice data obtained at each sensor node is again aggregated to the base station via the network. The finally obtained audio signal having a high SNR is transferred from the base station to the server device and used for a predetermined application on the server device.

  12 is a block diagram showing a network configuration (specific example) used in the position measurement system of FIG. 11, and FIG. 13 is a graph showing time synchronization with linear interpolation used in the position measurement system of FIG.

  In FIG. 12, the sensor nodes N0 to N2 including the server device SV are connected by, for example, a UTP cable 60 and communicate using Ethernet (registered trademark) of 10BASE-T. In this embodiment, the sensor nodes N0 to N2 are connected in a straight line topology, and one of the sensor nodes N0 operates as a base station and is connected to a server device SV formed of, for example, a personal computer. For the data link layer of the communication system, a known low power listening method (Low Power Listening) is used to reduce power consumption, and a known tiny diffusion method (Tiny Diffusion) is used for path construction in the network layer. .

  In this embodiment, in order to collect voice data among the sensor nodes N0 to N2, it is necessary to synchronize the time (timer value) in all the sensor nodes on the network. In this embodiment, a synchronization method is used in which linear interpolation is added to a known flooding time synchronization protocol (FTSP). FTSP realizes highly accurate synchronization only by simple communication in one direction. The accuracy of synchronization by FTSP is 1 microsecond or less between adjacent sensor nodes, but the crystal oscillators of each sensor node vary, and as shown in FIG. 13, a time shift occurs with time after synchronization processing. This shift is from several microseconds to several tens of microseconds per second, and this may reduce the performance of sound source separation.

  Therefore, in the proposed system of this embodiment, the time lag between the sensor nodes is stored at the time synchronization by FTSP, and the way of the timer is adjusted by linear interpolation. If the reception time stamp at the first synchronization is the time stamp at the second synchronization, and the timer value on the receiving side is taken, the deviation of the oscillation frequency can be reduced by adjusting how the timer advances only during this period. It can be corrected. Thereby, the time lag after the completion of synchronization can be suppressed within 0.17 microseconds in one second. Even if the time synchronization by FTSP is once per minute, by performing linear interpolation, the time lag between sensor nodes can be suppressed within 10 microseconds, and the performance of sound source separation can be maintained. .

  Relative time (for example, the elapsed time is defined as relative time with the time when the first sensor node was turned on is defined as 0) or absolute time (for example, calendar date / time / minute / second as time) is stored in each sensor node. In addition, time synchronization is performed between the sensor nodes by the method described above. This time synchronization is used to measure an accurate distance between sensor nodes as described later.

  14A and 14B are timing charts showing a signal transmission procedure between the tablets T1 to T4 and processes executed by the tablets T1 to T4 in the position measurement system of FIG. Here, for example, each tablet T1 to T4 having the configuration of FIG. 11 is configured to include the sensor node. In the following description, an example in which the tablet T1 is a master and the tablets T2 to T4 are slaves will be described, but the number of tablets and the master may use any tablet. The sound signal may be an audible sound wave or an ultrasonic wave exceeding the frequency in the audible range. Here, the sound signal includes, for example, an AD conversion circuit 51 and a DA conversion circuit, and generates, for example, an omnidirectional sound signal from one microphone 1 in response to an instruction from the SSL processing unit 55, or an ultrasonic generation element. And an ultrasonic omnidirectional sound signal may be generated in response to an instruction from the SSL processing unit 55. Furthermore, the SSS process may not be executed in FIGS. 14A and 14B.

  14A, first, in step S31, the tablet T1 instructs the tablets T2 to T4 to “prepare to receive the sound signal with the microphone 1 and execute the SSL process in response to the sound signal. After transmitting the “SSL instruction signal”, a sound signal is transmitted for a predetermined time such as 3 seconds after a predetermined time. The SSL instruction signal includes sound signal transmission time information, and each tablet T2 to T4 calculates the difference between the time when the sound signal is received and the transmission time information, that is, the transmission time of the sound signal. The distance between the tablet T1 and its own tablet is calculated and stored in the built-in memory by multiplying the speed of the known sound wave or ultrasonic wave by the calculated transmission time. Each tablet T2 to T4 estimates the direction of arrival of the sound signal by performing sound source localization processing using the MUSIC method described in detail in the first embodiment based on the received sound signal, and calculates the built-in memory. To remember. That is, in the SSL processing of each tablet T2 to T4, the distance from the tablet T1 to its own tablet and the angle with respect to the tablet T1 are estimated and stored.

  Next, in step S32, the tablet T1 transmits, to the tablets T3 and T4, an “SSL instruction signal for preparing to receive with the microphone 1 and instructing to execute an SSL process in response to the sound signal”. Then, after a predetermined time, a sound generation signal instructing generation of a sound signal is transmitted to the tablet T2. Here, the tablet T1 also enters a sound signal standby state. In response to the sound generation signal, the tablet T2 generates a sound signal and transmits it to the tablets T1, T3, and T4. Each tablet T1, T3, T4 estimates the direction of arrival of the sound signal by performing sound source localization processing using the MUSIC method described in detail in the first embodiment based on the received sound signal, and calculates the built-in memory. To remember. That is, in the SSL process of each tablet T1, T3, T4, an angle with respect to the tablet T2 is estimated and stored.

  Further, in step S33, the tablet T1 transmits to the tablets T2 and T4 “SSL instruction signal for preparing to receive by the microphone 1 and instructing to execute SSL processing in response to the sound signal”. Then, after a predetermined time, a sound generation signal instructing to generate a sound signal is transmitted to the tablet T3. Here, the tablet T1 also enters a sound signal standby state. In response to the sound generation signal, the tablet T3 generates a sound signal and transmits it to the tablets T1, T2, and T4. Each tablet T1, T2, T4 estimates the direction of arrival of the sound signal by performing sound source localization processing using the MUSIC method described in detail in the first embodiment based on the received sound signal, and calculates the arrival direction of the sound signal. To remember. That is, in the SSL process of each tablet T1, T2, T4, the angle with respect to the tablet T3 is estimated and stored.

  Still further, in step S34, the tablet T1 transmits to the tablets T2 and T3 "SSL instruction signal for preparing to receive with the microphone 1 and instructing to execute the SSL process in response to the sound signal". Then, after a predetermined time, a sound generation signal instructing generation of a sound signal is transmitted to the tablet T4. Here, the tablet T1 also enters a sound signal standby state. In response to the sound generation signal, the tablet T4 generates a sound signal and transmits it to the tablets T1, T2, T3. Each tablet T1, T2, T3 estimates the direction of arrival of the sound signal by performing sound source localization processing using the MUSIC method described in detail in the first embodiment on the basis of the received sound signal, and calculates the built-in memory. To remember. That is, in the SSL processing of each tablet T1, T2, T3, the angle with respect to the tablet T4 is estimated and stored.

  Next, in step S35 for performing data communication, the tablet T1 transmits an information return instruction signal to the tablet T2. In response to this, the tablet T2 calculates the distance between the tablets T1 and T2 calculated in step S31 and the angles when the tablets T1, T3, and T4 are viewed from the tablet T2 calculated in steps S31 to S34. An information reply signal including the above is returned to the tablet T1. The tablet T1 transmits an information return instruction signal to the tablet T3. In response to this, the tablet T3 calculates the distance between the tablets T1 and T3 calculated in step S31 and the angles when the tablets T1, T2, and T4 are viewed from the tablet T3 calculated in steps S31 to S34. An information reply signal including the above is returned to the tablet T1. Furthermore, the tablet T1 transmits an information return instruction signal to the tablet T4. In response to this, the tablet T4 calculates the distance between the tablets T1 and T4 calculated in step S31 and the angles when the tablets T1, T2, and T3 are viewed from the tablet T4 calculated in steps S31 to S34. An information reply signal including the above is returned to the tablet T1.

In the entire SSL processing of the tablet T1, based on the information collected as described above, the tablet T1 calculates the distance between the tablets as described below with reference to FIG. Based on the angle information of the other tablets T1 to T4 viewed from other tablets, for example, the XY coordinates of the other tablets T2 to T4 when the tablet T1 (A in FIG. 15) is the origin of the XY coordinates. By calculating using the well-known trigonometric function formula,
The coordinate values of all the tablets T1 to T4 can be obtained. The coordinate value may be displayed on a display or output to a printer for printing. Further, for example, a predetermined application described in detail later may be executed using the coordinate value.

  In addition, about the SSL whole process of tablet T1, only the tablet T1 which is a master may perform, and it may carry out with all the tablets T1-T4. That is, at least one tablet or server device (for example, SV in FIG. 12) may be executed. The SSL process and the SSL overall process are executed by the control unit, for example, the SSL processing unit 55.

  15 is a plan view showing a method for measuring the distance between the tablets from the angle information measured by the tablets T1 to T4 (corresponding to A, B, C, and D in FIG. 15) of the position measurement system of FIG. FIG. After all the tablets have acquired the angle information, the server device calculates distance information for everyone. In the calculation of distance information, as shown in FIG. 15, the lengths of all sides are obtained by the sine theorem using the values of 12 angles and the length of any one side. When the length of AB is d, the length of AC is obtained by the following equation.

  Similarly, the lengths of the other sides can be obtained using 12 angles and the length d. If each sensor node can perform the above-described time synchronization, each sensor node can obtain the distance from the difference between the sound generation start time and the arrival time without using the above calculation method. Although the number of nodes in FIG. 15 is four, the present invention is not limited to this, and the distance between nodes can be obtained regardless of the number of nodes when the number of nodes is two or more.

  In the above second embodiment, the two-dimensional position is estimated, but the present invention is not limited to this, and the three-dimensional position may be estimated using a similar mathematical expression.

  Further, the mounting of the sensor node on the mobile terminal will be described below. When the network system is put to practical use, it is conceivable that the sensor node is mounted not only on a wall or ceiling but also on a moving terminal such as a robot. If the position of the person to be recognized can be estimated, an operation of bringing the robot closer to the person to be recognized can be performed in order to collect images with higher resolution and to perform highly accurate speech recognition. In addition, mobile terminals such as smartphones that have been rapidly spreading in recent years can acquire their current positions using the GPS function, but it is difficult to acquire the positional relationship between terminals at a short distance. However, if the sensor node of the network system is mounted on a mobile terminal, it is possible to acquire the positional relationship between the terminals at a short distance that cannot be determined by the GPS function or the like by emitting sound from the terminals and locating each other as a sound source. . In the present embodiment, two types of applications, that is, a message exchange system and a multi-person hockey game system, are implemented using the programming language Java as applications that use the positional relationship between terminals.

  In this embodiment, a tablet personal computer that executes an application and a prototype sensor node are connected. A general-purpose OS is installed as the OS of the tablet personal computer, and a wireless network is configured with two USB 2.0 ports and a wireless LAN function conforming to IEEE 802.1b / g / n. Microphones of prototype sensor nodes are arranged at intervals of 5 cm on the four sides of the tablet personal computer, and a sound source localization module is operated at the sensor node (configured by FPGA) so that the localization result is output to the tablet personal computer. Configured. The position estimation accuracy in this embodiment is about several centimeters, which is significantly higher than that of the prior art.

  16 is a plan view showing a concept of a message exchange system using the position measurement system of FIG. 12, and FIG. 17 is a display example of a display showing one application of the message exchange system of FIG.

  When a message is exchanged with a nearby partner on a mobile terminal, an email or instant messenger must acquire an account in advance and designate the partner every time using an email address or the like. In this embodiment, such a complicated operation is not performed, and an application of a message exchange system that intuitively transmits and receives a message using the other party's angle information such as “who is at which angle” is implemented.

  In the conceptual diagram of FIG. 16, an application of a message exchange system is activated on each tablet, and is connected to a server device that is set up as a base unit via a wireless LAN. In the message exchange system, a sticky note type window on which a message is written is dragged (that is, on the display, which is a display unit having a known pointing device based on the calculated position of each node in the application). Corresponding the direction to each of the above nodes, drag the predetermined data on the display to the node where the data should be transmitted to indicate the node to be transmitted.) Message by skipping to the other node Send. Operations such as adding a message to a sent tag and turning it to the next person can be performed only with the user's own name, server IP address and port number specified at the time of connection.

  In order to use the message exchange system, it is necessary for all the tablets to obtain angle information “in which direction the other tablet is seen from the user” in advance, which will be described with reference to FIGS. 14A and 14B. The angle information of all tablets can be obtained by the communication procedure and process described above.

  In FIG. 17, which shows an example of using an application of the message exchange system, it is usually resident in the task tray, and a sticky note window is displayed on the screen when the icon is double-clicked. Your own sticky notes are displayed in blue, and those sent from other tablets are displayed in yellow. When you drag and drop a sticky note, it will fly off the screen, and the content written on the sticky note will be sent to the partner closest to the direction you were flying. When a message is received, a sticky note on which the received message is written flies off the screen at the angle where the sender is registered. The tag can be deleted by selecting Delete from the right-click menu.

  FIG. 18 is a plan view showing a concept of a hockey game system using the position measurement system of FIG. In the message exchange system described above, only the angle information “who is at which angle” is used. However, by collecting the angle information held by each tablet and obtaining the distance between the tablets, data can be transmitted with a time difference. Is possible. Using this technique, we implemented an application for a multiplayer hockey game system in which players play each other with balls.

  In the conceptual diagram of FIG. 18, the connection method, the acquisition of angle information, and the data transmission / reception method are the same as those of the message exchange system. In the hockey game system, players play against each other by playing balls with a bar. This time, the number of participants in the game is four.

  An execution screen of the server device is shown in FIG. The server device includes an SSL button that collects angle information of each tablet and calculates a distance, a button that starts a game, a pause, a reset, and an End button that ends the connection. The text area displays connection information, distance information calculation completion notification, who sent the ball to whom, and the like. After all the tablets have acquired the angle information, the server calculates the distance information for all members by pressing the SSL button before starting the game. In the calculation of distance information, as shown in FIG. 15, the lengths of all sides are obtained by the sine theorem using the values of 12 angles and the length of any one side. When the length of AB is taken, the length of AC is obtained by the above equation (1). Similarly, the lengths of the other sides can be obtained using 12 angles and the length d. If each sensor node can perform the above-described time synchronization, each sensor node can obtain the distance from the difference between the sound generation start time and the arrival time without using the above calculation method. Although the number of participants was four this time, this method can be used to determine the distance regardless of the number of participants.

  FIG. 19 is a display example of the execution screen of the server device of the hockey game system of FIG. 17, and FIG. 20 is a display example of the tablet when the hockey game system of FIG. 17 is being executed. In FIG. 19, a score is displayed at the top of the screen, and is incremented by 1 when the ball falls to the bottom of the screen and when the opponent misses with the ball that he sent. When you touch the screen, a ball appears a little above the bar, and you play the ball that you put out or the ball sent from your opponent with the bar and send it to your opponent. The bar can be moved up, down, left and right, and you can send a fast ball to your opponent by playing the ball with acceleration. The color of the ball varies from player to player, and you can tell who has sent the ball. The game ends when someone scores 10 or -30.

  INDUSTRIAL APPLICABILITY The present invention is useful for a position measurement system that can measure the position of a terminal with high accuracy using a microphone array and an application using the position measurement system.

1, m11, m12, ..., m43, m44 ... microphones,
1a, 1b, 1c, ..., 1n ... microphone array,
2, 2a, 2b, 2c,..., 2n.
3. Input / output unit (I / O unit),
4 ... Processor,
10, 11, 12 ... network,
20 ... voice processing server,
30, 30a, 30b, 30c ... nodes,
51 ... AD converter circuit,
52 ... VAD processing unit,
53 ... Power management unit,
54 ... SRAM,
55 ... SSL processing unit,
56... SSS processing unit,
57 ... Network interface circuit,
N0 to Nn ... sensor node (node),
SV: server device,
T1-T4 ... Tablet.

Claims (3)

A position measurement system including a controller that estimates a position of each node using a microphone array network system in which a plurality of nodes including a microphone array are connected to each other via a network and time-synchronized.
Each of the above nodes
A microphone array configured by arranging a plurality of microphones in an array;
A sound source estimation processing unit that estimates the angle of the arrival direction of the sound signal based on the sound signal from one node received by the microphone array;
A distance estimation unit that estimates a distance from a node that has transmitted the sound signal, based on a difference between a transmission time and a reception time of the sound signal and a speed of the sound signal;
A data communication unit that transmits and receives the estimated angle and distance of the direction of arrival to other nodes by data communication,
The control unit estimates and calculates the position of each node based on the estimated angle and distance of the arrival direction ,
The control unit executes a predetermined application using the position of each node,
Each of the above nodes has a display unit,
In the predetermined application, the control unit associates a direction with respect to each node on the display unit based on the calculated position of each node, and transmits a predetermined data on the display unit A position measuring system, characterized by instructing a node to be transmitted by dragging toward the node and transmitting the predetermined data to the instructed node .   The position measurement system according to claim 1, wherein the control unit calculates and displays or outputs a coordinate value based on the position of each node. 3. The position measurement according to claim 1, wherein the control unit is provided in each of the nodes, one of the plurality of nodes, or a server device different from the plurality of nodes. system.

Images