JP2014137226A - Mobile object, and system and method for creating acoustic source map - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile object that can identify the position of a sound source, and can collect advance information on an acoustic source available for real voice communication.SOLUTION: In a robot 1000, a robot position identifying section 1040 estimates the position of the robot 1000 using a particle filter including a plurality of particles. By using an incoming direction of a sound estimated from measured values by a microphone array 1052, a sound source position estimating section 1070 accumulates likelihoods with respect to hit positions as raycasts going in the incoming direction from the positions of the particles hit an object that is identified on a geometry map, to estimate the position of an acoustic source.

Description

  The present invention relates to a sound source localization technique in a real environment, and more particularly to a sound source position estimation technique that combines detection of a voice direction by a sound sensor array and a moving object position specifying technique in a real environment.

  Different environments such as home, office, and shopping streets have various noise characteristics depending on location and time, so in applications that target specific sounds such as voice, depending on the type and degree of environmental noise used, There is a problem that the expected performance cannot be obtained.

  For example, in voice communication between a person and a robot, the microphone attached to the robot is usually at a position (1 m or more) away from the robot. Therefore, for example, the signal-to-noise ratio (SNR) is lower than when the distance between the microphone and the mouth is several centimeters as in telephone voice. For this reason, the voices of others nearby and the noise of the environment become interference sounds, making it difficult for the robot to recognize the target speech. Therefore, sound source localization and sound source separation are important for robot applications.

  Regarding the sound source localization, there are those described in Patent Document 1 or Patent Document 2 as conventional techniques assuming an actual environment. The technique described in Patent Document 1 or Patent Document 2 uses a known sound source localization method called the MUSIC method with high resolution.

  In the invention described in Patent Document 1 or Patent Document 2, a current correlation matrix is calculated based on a received signal vector obtained by Fourier-transforming a signal from the microphone array using a microphone array and a past correlation matrix. To do. The correlation matrix thus obtained is subjected to eigenvalue decomposition to obtain the maximum eigenvalue and a noise space that is an eigenvector corresponding to an eigenvalue other than the maximum eigenvalue. Furthermore, the direction of the sound source is estimated by the MUSIC method based on the phase difference of the output of each microphone, the noise space, and the maximum eigenvalue with one microphone as a reference in the microphone array.

  On the other hand, for example, despite several major breakthroughs in the field of robot hearing in recent decades, accurate localization and mapping of acoustic sources is due to the complexity and diversity of the environment. Still a challenging task.

  For example, localization of acoustic sources using mobile platforms has been reported as part of robotic research (see, for example, Non-Patent Documents 1-4).

Japanese Patent Application Laid-Open No. 2008-175733 JP 2011-220701 A Specification

J.-M. Valin, J. Rouat, and F. Michaud, “Enhanced robot audition based on microphone array source separation with post-filter,” in Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004 IEEE / RSJ International Conference on, vol. 3, sept.-2 oct. 2004, pp. 2123-2128 vol.3. E. Martinson and A. C. Schultz, “Auditory evidence grids.” In IROS. IEEE, 2006, pp. 1139-1144. K. Nakadai, H. Okuno, H. Nakajima, Y. Hasegawa, and H. Tsujino, ”An open source software system for robot audition hark and its evalation,” in IEEE-RAS International Conference on Humanoid Robots, 2008, pp. 561-566. Y. Sasaki, S. Thompson, M. Kaneyoshi, and S. Kagami, “Mapgeneration and identification of multiple sound sources from robot in motion,” in Proceedings of IEEE / RSJ International Conference on Intelligent Robots and Systems, IROS 2010, 2010, pp. 437-443.

  When considering application to a system that cannot ignore the influence of noise from surroundings, such as a robot, search for sounds that contain information to be processed from such complex mixed sounds and discard other information. It is necessary.

  Thus, gathering prior knowledge of the acoustic environment at a given location is beneficial for any sound-related task in that environment. Here, the location and characteristics of the sound source are not always stationary.

  On the other hand, some of the acoustic sources can be active for a relatively long period of time, such as a shopping mall television, air conditioner or loudspeaker, and fully localized in the space.

  However, in an environment where such fixed acoustic sources and dynamically changing acoustic sources coexist, how to collect prior information on acoustic sources that can be used for real voice communication and create a database It was not always clear how to solve this problem.

  The present invention has been made to solve such a problem, and is a movable body that can identify the position of a sound source and can collect prior information on an acoustic source that can be used for actual voice communication. Is to provide.

  Another object of the present invention is to provide an acoustic source map creation system and an acoustic source map creation method capable of collecting prior information on acoustic sources that can be used for actual voice communication.

  According to one aspect of the present invention, a moving body, a moving means for driving the moving body, and a geometric map for specifying a geometric position of an object in a movable space of the moving body Storage means for storing the position of the moving object, the position measuring means for measuring the current position of the moving object and outputting the position measurement result, and the distance from the moving object to the object specified by the geometric map in the space with a predetermined probability. Ranging means to obtain distance measurement data according to the distribution, and position estimation means to estimate the position of the moving object by a particle filter including a plurality of particles, each particle is attributed to position and orientation information in space The position estimating means moves each particle based on the position measurement result, and based on the distance measurement data and the probability distribution, the first likelihood for the current attribute of each particle. And a moving body position estimating means for estimating the current position of the moving body as a weighted average of each particle. The sound sensor array and the signal from the sound sensor array Based on arrival direction estimation means for executing processing for specifying the arrival direction of sound arrival in the sound sensor array, and sound source localization means for estimating the position of the acoustic source based on the attribute and arrival direction of each particle And the sound source localization means obtains a second likelihood for the hit position in response to a raycast heading in the direction of arrival from the position of each particle hitting an object specified by the geometric map. Accumulate and estimate the position of the acoustic source based on the accumulated second likelihood.

  Preferably, the distance measuring means is a laser range finder, and the first likelihood is determined from the position of each particle to the direction of the particle in the probability distribution based on the measurement result of the distance from the current position of the moving object to the object. In the case of ray casting, the probability corresponding to the distance to hit the object in the geometric map is integrated at a predetermined angular interval.

  Preferably, the direction-of-arrival estimation means executes a plurality of sound scans, and the sound source localization means determines that at least one of the maximum value or the change amount of the sound power is a predetermined threshold in the second likelihood calculation. Select a sound scan that exceeds the value and execute the accumulation process.

  Preferably, the second likelihood increases with the sound power and is defined by a function having a value scaled to a predetermined range, and the sound source localization means selects a sound scan and the number of hits is predetermined. For the hit positions exceeding the threshold value, the accumulation process is executed so that the second likelihood increases.

  Preferably, the moving body creates an acoustic map indicating the position of the localized acoustic source in the geometric map, and stores the acoustic map in the storage device.

  Preferably, the mobile body is an autonomous mobile robot.

  Preferably, the moving body updates the acoustic map as it moves autonomously.

  Preferably, the position measuring unit is a path length measurement sensor that detects a moving distance and a moving direction of the moving body based on the movement of the moving unit.

  According to another aspect of the present invention, there is provided an acoustic source map creation system for specifying a moving means for driving a moving body and a geometric position of an object in a movable space of the moving body. Storage means for storing the geometric map, position measurement means for measuring the current position of the moving object and outputting a position measurement result, and the distance from the moving object to the object specified by the geometric map in space. , Ranging means for acquiring distance measurement data according to a predetermined probability distribution, and position estimation means for estimating the position of the moving body by a particle filter including a plurality of particles, each particle having a position and orientation in space The position estimation unit moves each particle based on the position measurement result, and based on the distance measurement data and the probability distribution, the current attribute of each particle. Means for calculating a first likelihood for the particle and calculating the weight of the corresponding particle, and a moving object position estimating means for estimating the current position of the moving object as a weighted average of each particle, and is attached to the moving object The sound direction of the sound sensor array, the direction of arrival estimation means for executing the process for specifying the direction of arrival of sound in the sound sensor array based on the signal from the sound sensor array, and the attribute and direction of arrival of each particle. And a sound source localization means for estimating the position of the acoustic source, wherein the sound source localization means hits the target specified by the geometric map by the ray cast from the position of each particle toward the arrival direction. In response, the second likelihood for the hit position is accumulated, the position of the acoustic source is estimated based on the accumulated second likelihood, and the estimated position of the acoustic source is calculated. Further comprising an acoustic source map database storing as HibikiHajime map.

  According to still another aspect of the present invention, there is provided a method for creating an acoustic source map, the step of driving and moving a moving body, and specifying the geometric position of an object in a movable space of the moving body. Storing a geometric map for storage in a storage device, measuring a current position of the moving body, outputting a position measurement result, and a distance from the moving body to an object specified by the geometric map in space Is obtained as distance measurement data according to a predetermined probability distribution, and a step of estimating the position of the moving body by a particle filter including a plurality of particles, each particle having position and orientation information in space. The step of estimating the position of the moving object as an attribute moves each particle based on the position measurement result, and based on the distance measurement data and the probability distribution. Calculating a first likelihood for the current attribute of each particle and calculating a weight of the corresponding particle; and estimating a current position of the moving object as a weighted average of each particle; Based on the signal from the mounted sound sensor array, the step of executing the process for identifying the direction of arrival of the sound in the sound sensor array, and on the basis of the attribute and direction of arrival of each particle, Estimating the position of the sound source, wherein the step of estimating the position of the acoustic source is in response to a raycast heading from the position of each particle hitting an object specified by the geometric map. The second likelihood for the hit position is accumulated, the position of the acoustic source is estimated based on the accumulated second likelihood, and the estimated acoustic source Position further comprising the steps of: storing in a storage device as an acoustic source map.

  According to the present invention, it is possible to identify the position of a sound source and collect prior information on an acoustic source that can be used for actual voice communication.

It is a figure for demonstrating the condition of an acoustic source map. It is another figure for demonstrating the condition of an acoustic source map. It is a figure which shows the structure for acoustic source map preparation among the structures of the robot 1000. FIG. It is a figure which shows the functional block for the sound source localization using a MUSIC algorithm. It is a figure which shows the example of the space which has arrange | positioned the robot 1000 and acoustic source S1 and S2. It is a figure which shows the external appearance of the robot 1000. FIG. It is a figure for demonstrating the concept of the ranging process by a laser range finder. It is a figure for demonstrating the concept of the specific process of the position of the robot. It is a flowchart for demonstrating the specific process of the position of the robot 1000 using a particle filter. It is a diagram for explaining a concept of calculating the particle state vector s ^m [t]. It is a figure which shows the likelihood about each particle based on the distance to a target object. It is a conceptual diagram for demonstrating the procedure which calculates the likelihood of a particle based on the measurement result (ranging data) of a laser range finder. It is a conceptual diagram which shows the procedure of the distance measurement by a laser range finder, and the raycast from a certain particle. It is a figure for demonstrating the concept of the ray cast for pinpointing the position of the sound source from a particle. It is a flowchart for demonstrating the process of the accumulation | accumulation of likelihood about an occupation cell, and an acoustic source mapping. FIG. 6 shows a map obtained for one of the test environments. It is a figure showing the parameter used for specification of an acoustic source position. It is a figure which shows the raw sound source map obtained in the case of flat initial setting. It is a figure which shows the raw sound source map obtained in the case of flat initial setting. It is a figure which shows the raw sound source map obtained in the case of flat initial setting. It is a table which shows the pattern of activation of the acoustic source for each trial.

  Hereinafter, a configuration of a moving object, an acoustic source map creation system, and an acoustic source map creation method according to an embodiment of the present invention will be described with reference to the drawings. In the following embodiments, components and processing steps given the same reference numerals are the same or equivalent, and the description thereof will not be repeated unless necessary.

  In the following description, as a sound sensor, a so-called microphone, more specifically an electret condenser microphone will be described as an example, but other sound sensors may be used as long as they can detect sound as an electric signal. Also good.

  In the present embodiment, a map including a position and characteristics such as power, directivity, frequency, and harmonic distortion of an acoustic source is referred to as an “acoustic source map”.

  In a real environment, a plurality of sounds generated at different locations are mixed and observed. Therefore, a simple method of scanning a space with a sound level meter is insufficient for generating an acoustic source map. Generating a sound environment map that characterizes the location and type of a sound source that may be useful as prior knowledge of the sound environment requires at least localization and separation of the sound source in addition to spatial information (normal map). It is desirable that the sound source is classified.

  Information obtained from the acoustic source map can complement visual data or ranging sensor data, increasing the robot's position, interaction and monitoring capabilities.

  FIG. 1 and FIG. 2 are diagrams for explaining the state of such an acoustic source map.

For example, considering the example in Figure 1, the robot perceives two main sounds:
i) TV sound from the front or the left side ii) User's voice from the right side The robot is an autonomous mobile robot that occupies a predetermined position on the map, and perceives its current posture. Here, in FIG. 1, such an autonomous mobile robot is a humanoid robot configured to be driven by two wheels capable of differential operation.

  As shown in FIG. 2, if an acoustic source map is available to the robot, the robot will have prior knowledge of the fixed acoustic source in the environment, and the data coming from its left is from a fixed source (TV). This means that it is less important than the voice of the user from the right side.

  Then, based on the characteristics of the new sound in the environment (in this case it is human speech), the robot positions itself so that it can hear what the user says (right Can communicate with him / her more effectively.

  An acoustic source map in the environment is similar to a geometric map, but it is not constant and changes very rapidly. Therefore, creating a new source map every time a change occurs in the environment is a tedious task.

  As will be described below, in the present embodiment, while the robot performs its normal task, not only the execution of the task but also the environmental sound can be observed and the change in the acoustic source map can be updated. Provide work.

  FIG. 3 is a diagram showing a configuration for creating an acoustic source map among the configurations of the robot 1000. In FIG. 3, a moving mechanism for moving the robot (for example, a driving mechanism using two wheels that can operate differentially, a sensor for visually detecting an external environment such as a camera) is illustrated. Omitted. In the following example, a human-type autonomous mobile robot driven by two wheels will be described as an example of a “moving body” for creating an acoustic source map. The robot is not limited to the one, and may be a robot having another configuration as long as it can move autonomously in the space.

  Referring to FIG. 3, robot 1000 has a path length measurement sensor 10 for detecting a moving distance, speed, angular velocity, and the like of a driving mechanism that moves the robot by a two-wheel driving mechanism, and a front surface of robot 1000. Is mounted on the rear surface of the robot 1000 and a front surface laser range finder (LRF) 20 for measuring the distance to an object existing ahead by scanning the laser beam. A rear surface laser range finder (LRF) 30 for measuring the distance to the object existing behind, and signals from the path measurement sensor 10, the front surface LRF 20, and the rear surface LRF 30 are output as data via the bus 60, or A sensor input / output board 40 is provided for transmitting commands from the bus 60 to these sensors. That.

  The robot 1000 further functions as a working memory, and includes a memory 50 configured by a RAM (Random Access Memory), a program (not shown) for operating the robot 1000, a geometric map 1102, and an acoustic source map 1104. And a non-volatile storage device 1100 for storing the data. As the nonvolatile storage device 1100, a hard disk may be used as long as it is a randomly accessible storage device, or an SSD (Solid State Drive) or the like may be used. The “geometric map” refers to data that geometrically expresses the position of a wall in the space in which the robot 1000 moves or a fixed / semi-fixed object that exists constantly on the map. The “geometric map” may be two-dimensional information, but is preferably three-dimensional information.

  The robot 1000 further includes a microphone array 1052 for measuring the direction of the sound source (sound arrival direction) and the acoustic power from the direction, and a signal for transmitting the signal from the microphone array 1052 to a signal transmitted to the bus 60. A voice input / output board 1054 and a processor 1010 which is an arithmetic unit for controlling the operation of the robot 1000 and executing creation of a geometric map and creation and update of an acoustic map are provided. Although not shown in FIG. 3, the audio input / output board is also connected to a speaker that reproduces the audio generated by the processor 1010 for communication with the user.

  The processor 1010, based on a program stored in the nonvolatile storage device 1100, a geometric map creation unit 1030 that creates a geometric map based on signals from the path length measurement sensor 10, the front surface LRF 20, and the rear surface LRF 30, A sound source power spectrum acquisition unit 1050 that executes processing for acquiring a sound source power spectrum based on a signal from the microphone array 1052, and a sound source direction (sound arrival direction) is estimated on the robot coordinates based on the sound source power spectrum. A sound source direction estimating unit 1060, a robot position specifying processing unit 1040 for specifying the position of the robot based on signals from the path measurement sensor 10, the front surface LRF 20, and the rear surface LRF 30, and the specified robot position and sound source Sound source position estimation unit 1 for estimating the sound source position from the direction And a 70.

  Here, the estimation of the sound source direction is based on estimation of the time delay of arrival at the microphone array, or estimation of spectral decomposition techniques such as the steered response power (SRP) or the MUSIC algorithm.

  In the case of sound source direction estimation using the phase difference, it is possible to perform sound source direction estimation using the SRP-PHAT method (Steered Response Power-Phase Alignment Transform), SPIRE method (Stepwise Phase Difference Restoration method), etc. It is.

  Such estimation of the sound source direction is also disclosed in the above-mentioned Patent Document 2 and the following documents.

Literature 1: JP 2012-242597 A FIG. 4 is a diagram showing functional blocks for sound source localization using the MUSIC algorithm in such a sound source localization method.

  The sound source localization here means to continuously specify the direction of the sound source, and the position estimation of the sound source is based on the direction of the sound source specified by the sound source localization in a predetermined space. This refers to estimating the position of a sound source.

  As an example, a MUSIC (Multiple Signal Classification) method will be described as an example of a method for estimating the direction of a sound source in order to estimate the position of the sound source. However, other methods may be used as long as the direction of the sound source can be estimated.

The outline of the MUSIC method will be described. First, a multi-channel spectrum X (k, t) is obtained for each frame by fast Fourier transform, and a spatial correlation matrix R _k between channels in a spectral region is obtained for each block. The eigenvalue decomposition of the directional component and the omnidirectional component subspace is decomposed, the eigenvector E _k ⁿ corresponding to the omnidirectional subspace, and the direction vector a _k prepared in advance according to the target search space. Is used to obtain a (narrowband) MUSIC spatial spectrum P (k) for each frequency bin, and a MUSIC spatial spectrum for each frequency bin within a specific frequency band is integrated to obtain a wideband MUSIC spatial spectrum.

  In the following, the broadband MUSIC spatial spectrum is simply referred to as “MUSIC spatial spectrum”, and the time series of the MUSIC spatial spectrum is referred to as “MUSIC spectrogram”.

  In sound source localization, the direction of the sound source is obtained by searching for the peak of the MUSIC spatial spectrum.

  In the following, a case where there is one microphone array will be described as an example, but the number of microphone arrays may be larger.

  Referring to FIG. 4, sound source power spectrum acquisition section 1050 includes microphones 1052.1 to 1052. An A / D converter 1054 that receives p analog sound source signals from the microphone array MC1 including p (p: natural number), performs analog / digital conversion, and outputs p digital sound source signals, respectively, and A / Receiving p digital sound source signals respectively output from the D converter 1054, the correlation matrix and its eigenvalues and eigenvectors required by the MUSIC method are set for each block with a predetermined time, for example, 100 milliseconds as one block. An eigenvector calculation unit 61 for outputting, and a MUSIC processing unit 62 that uses the eigenvector output from the eigenvector calculation unit 61 for each block and outputs a MUSIC space spectrum by the MUSIC method. The sound source direction estimation unit 1060 estimates the direction of the sound source (in this embodiment, two declination angles φ and θ in the three-dimensional polar coordinates) based on the MUSIC spatial spectrum output from the MUSIC processing unit 62. . In the present specification, the “MUSIC response” is obtained by averaging the MUSIC spatial spectrum obtained by the MUSIC algorithm using a predetermined formula.

  Although not particularly limited, in this embodiment, the A / D converter 1054 A / D converts the output of each microphone at a general 16 kHz / 16 bits.

  In addition, the eigenvector calculation unit 61 framing the p digital sound source signals output from the A / D converter 1054 based on the signal from the microphone array MC1 with a frame length of, for example, 4 milliseconds. Unit 80 and p-channel framed sound source signals output from framing processing unit 80 are each subjected to FFT (Fast Fourier Transform), and a predetermined number of frequency regions (hereinafter, each frequency region is referred to as a “bin”). The FFT processing unit 82 that converts the frequency domain number into a number of bins and outputs it, and the bin value of each channel that is output from the FFT processing unit every 4 milliseconds is expressed as 100. A blocking processing unit 84 for blocking every millisecond, and each bin output from the blocking processing unit 84 A correlation matrix calculation unit 86 that calculates and outputs a correlation matrix having a correlation between the two as a factor every predetermined time (every 100 milliseconds), and eigenvalue decomposition of the correlation matrix output from the correlation matrix calculation unit 86, Is output to the MUSIC processing unit 62.

  Usually, 512 to 1024 points are used in FFT (corresponding to 32 to 64 milliseconds at a sampling rate of 16 kHz), but here one frame is set to 4 milliseconds (corresponding to 64 to 128 points in FFT). By reducing the frame length in this way, not only the amount of calculation of FFT is reduced, but also the amount of calculation in later calculation of correlation matrix, eigenvalue decomposition, and calculation of MUSIC response is reduced. As a result, sound source localization can be performed sufficiently in real time even if a relatively weak computer is used without degrading performance.

  The MUSIC processing unit 62 includes a microphone arrangement storage unit 100 for storing a position vector representing the position of each microphone included in the microphone array MC1 using a predetermined coordinate system, and a microphone stored in the microphone arrangement storage unit 100. And a MUSIC spatial spectrum calculation unit 104 that calculates and outputs a MUSIC spatial spectrum by the MUSIC method on the assumption that the number of sound sources is fixed, using the position vector of Eq. The fact that the eigenvalue of the correlation matrix obtained for each block is related to the number of sound sources is described in, for example, Reference 2: Asano et al., “Real-time sound source localization and generation system and its application in automatic speech recognition”, Eurospeech, 2001, Aalborg, Denmark, 2001, 1013-1016 (F. Asano, M. Goto, K. Itou, and H. Asoh, “Real-time sound source localization and separation system and its application on automatic speech recognition,” in Eurospeech 2001, Aalborg, Denmark, 2001, pp. 1013-1016) It is.

  In the present embodiment, not only the two-dimensional azimuth angle of each sound source but also the elevation angle is estimated. Therefore, as the MUSIC algorithm, an algorithm that can calculate in three dimensions is implemented. The set of azimuth and elevation is hereinafter referred to as sound source azimuth (DOA). The algorithm executed by the MUSIC processing unit 62 does not estimate the distance to the sound source. By estimating only the sound source azimuth, the processing time can be significantly reduced.

  The MUSIC processing unit 62 further includes a MUSIC response calculation unit 106 for calculating and outputting a value called a MUSIC response for each direction according to the MUSIC method based on the MUSIC spatial spectrum calculated by the MUSIC spatial spectrum calculation unit 104. .

  The sound source direction estimation unit 1060 includes a buffer 108 for temporarily accumulating a predetermined number of MUSIC response peaks calculated by the MUSIC response calculation unit 106 in time series in the FIFO format. Further, the sound source direction estimation processing unit 110 estimates the direction of the sound source (the two declination angles φ and θ described above) for the MUSIC response of each search point of each block accumulated in the buffer 108.

Here, in the MUSIC method, in the estimation of the narrow band MUSIC space spectrum, it is necessary to give the number of sound sources (NOS) having directivity emitted at that time, but in the following explanation, a fixed number is given and the MUSIC space is given. In the following description, it is assumed that only peaks that exceed a specific threshold on the spectrum are regarded as directional sound sources.
(MUSIC method)
Hereinafter, the MUSIC method for calculating the above-described three-dimensional orientation will be briefly summarized.

  For example, the Fourier transform Xm (k, t) of M microphone inputs is modeled as in equation (M1).

However, the vector s (k, t) consists of N sound source spectra S _n (k, t) (n = 1,..., N).

That is, s (k, t) = [S ₁ (k, t),..., S _N (k, t)] ^T. Here, k and t indicate frequency and time frame indexes, respectively. Vector n (k, t) indicates background noise. The matrix A _k is a conversion function matrix, and its (m, n) element is a conversion function of a direct path from the nth sound source to the mth microphone. The n-th column vectors of A _k is referred to as a position vector of the n-th sound source (Steering Vector).

First, a spatial correlation matrix R _k defined by Expression (M2) is obtained, and E _k composed of a diagonal matrix Λ _k of eigenvalues and eigenvectors is obtained by eigenvalue decomposition of Rk shown in Expression (M3).

The eigenvector can be divided as E _k = [E _ks | E _kn ]. E _ks and E _kn indicate eigenvectors corresponding to the dominant N eigenvalues and other eigenvectors, respectively.

  The MUSIC spatial spectrum is obtained by equations (M4) and (M5). r is a distance, and θ and φ are an azimuth angle and an elevation angle, respectively. Equation (M5) is a normalized position vector at the scanned point (r, θ, φ).

The MUSIC response (corresponding to power) is obtained by averaging the MUSIC spatial spectrum as shown in Equation (M6).

In Expression (M6), k _L and k _H are indices of the lower and upper boundaries of the frequency band, respectively, and K = k _H −k _L +1. The direction of the sound arriving at the microphone array can be obtained by searching for the peak of the MUSIC response.

  As will be described later, the acoustic map creation system according to the present embodiment uses a conventional sound source localization algorithm at different locations by a robot, and combines the results from all these different locations to generate an acoustic map. create.

  As described above, the MUSIC method can be used as an algorithm for estimating the direction of arrival of sound, while other methods such as the steered response power method can be used.

  Thus, although not particularly limited, a case where the direction of arrival of sound is estimated based on the steered response power method will be described below as an example of another algorithm.

  Such a steered response power method is disclosed in the following document.

Reference 3: M. Brandstein and H. Silverman, “A robust method for speech signal time-delay estimation in reverberant rooms,” in IEEE Conference on Acoustics, Speech, and Signal Processing, ICASSP 1997, 1997, pp. 375-378.
Reference 4: A. Badali, J.-M. Valin, F. Michaud, and P. Aarabi, “Evaluating realtime audio localization algorithms for artificial audition on mobile robots,” in Proceedings of IEEE / RSJ International Conference on Intelligent Robots and Systems , IROS 2009, 2009, pp. 2033-2038.
“Staired response power” with phase transformation (SRP-PHAT) is an efficient sound source localization algorithm suitable for mobile robot applications. To illustrate this framework in more detail, we will outline a steered response power approach to sound source localization.

  The goal of sound source localization is to use audio measurements to evaluate the position of the acoustic source in the search area.

Assume that the measured signals from the Q microphones in the array at sampling time k are v ₁ (k),..., V _Q (k).

  Since the spatial arrangement of the microphone array is known in advance accurately, it is possible to focus the array using spatial beamforming and evaluate the sound from a certain spatial location.

  The beamforming output is expressed as:

Here, [x, y, z] are the coordinates of the focal position with reference to the microphone array.
The steered response power (SRP) is obtained by calculating the output power against the T sample time as follows:

Here, the following coordinates represent a set of N positions in the search space:

The location corresponding to the peak of the steered response power gives the position of the acoustic source. There are several ways to obtain the beamforming output, calculate power, and select a set of acoustic source locations.

  In the following, the steered response power acquired at time k, including the power from N locations, will be referred to as the kth “audio scan”.

  Therefore, performing sound source localization as the robot moves means combining audio scans obtained from different locations at different times.

  In particular, these scans are calculated in a search space in coordinates (robot coordinates) relative to the microphone array (the focused position needs to be known in the coordinates relative to the array).

  Therefore, it is necessary to convert them from a robot coordinate system to a global coordinate system for creating an acoustic source map by combining them.

  Since these localization results are acquired at different times, it is possible to create an acoustic source map by distinguishing between a fixed acoustic source and a moving acoustic source.

  In the present embodiment, as described with reference to FIGS. 1 and 2, we are interested in mapping environmental noise of a fixed acoustic source (). That is, having an acoustic source map of a fixed acoustic source is good prior knowledge for detecting what is moving.

  In the above description, the sound source is localized by the microphone array attached to the robot. For example, in the space, the microphone array is installed at a different position from the robot. An acoustic source map may be created together with data from such a fixed-position microphone array.

(Acquisition of audio scan by SRP-PHAT method)
In the present embodiment, the SRP-PHAT processing is performed by, for example, short-time Fourier transform on an observation signal sampled at 48 kHz (analysis window is 25 milliseconds in length, and window shift is 10 milliseconds). After applying the transform (STFT), it is done in the frequency domain.

  The steered response power is then calculated in the frequency band [1000, 6000] Hz using 5 STFT frames for power averaging.

  Therefore, an audio scan is newly executed every 50 milliseconds.

  The feature of the sound source localization algorithm is that the evaluation of its direction is accurate, but the evaluation of the range where the sound source exists is inaccurate. Therefore, polar coordinates [ρ, θ, φ] are often used to describe the search area, as in the MUSIC method.

  In this embodiment, it is assumed that the far field (ρ is larger than the aperture of the array) is assumed, and only the azimuth is scanned.

  Further, in the following, for the sake of simplicity of explanation, only the two-dimensional direction is considered as the direction of the acoustic source. Of course, based on the following description, it is possible to extend the estimation of the orientation of the acoustic source to three dimensions.

  That is, the k-th scan is a set of N angles qn (k) (= θn) in the range [0, 2π] and the power Jn (k) associated therewith (these angles Obtained at the reference coordinates).

(Construction of geometric map)
In the following, a process of constructing a geometric map will be described as a premise of the process of specifying the position of the sound source in the space based on the estimation result of the direction of the sound source as described above.

  That is, for example, a map (design map) created as a space design drawing in advance, such as a wall position for a space in which the robot 1000 moves, is given, and the robot 1000 is stored in the nonvolatile storage device 1100. It shall be held.

  However, in a space where the robot actually moves, there are objects that are fixedly or semi-fixedly installed (for example, the television in FIG. 1), and a geometric map including such objects is present. Must be constructed as a specific premise of the position of the sound source.

  Mapping technology based on simultaneous positioning and mapping (SLAM) for creating such a geometric map has been well studied, completed and applied in practice. Is.

  In this embodiment, the technique is used for generating a grid map.

  For example, in order to create such a geometric map in advance, a human can control a robot with a joystick and collect path length measurement and laser sensor information under the environment of the space to be measured. .

  Then, using the iterative closest point (ICP) based SLAM, the robot trajectory is corrected using the 3D Toolkit library framework, and the laser range finder scan is aligned with the map.

  With the resulting aligned scan, an occupancy grid map is created.

  Such an occupation grid map is disclosed in, for example, the following documents.

Reference 5: A. Elfes, “Using occupancy grids for mobile robot perception and navigation,” Computer, vol. 22, no. 6, pp. 46-57, June 1989.
Therefore, in the present embodiment, “matching the scan of the laser range finder to the map” means matching the direction and position of the scan of the laser range finder and the positioning position in the design drawing. The “occupied grid” is a fixed object (a fixed object such as a wall or a semi-fixed object such as a television set) on a geometric map, and is the time to be measured. A grid that is occupied by what is considered fixed relative to the spacing.

  As will be described later, in the present embodiment, the position of the acoustic source in the space is specified by combining the direction of the acoustic source and information related to the “occupied grid”.

(Robot position identification processing approach)
Hereinafter, processing performed by the robot position specifying processing unit 1040 will be described in detail.

  FIG. 5 is a diagram illustrating an example of a space in which the robot 1000 and the sound sources S1 and S2 are arranged as an experiment.

  In FIG. 5, the robot 1000 has only a configuration for moving mechanism and sound source localization for experiments.

  FIG. 6 is a view showing the appearance of such a robot 1000.

  A front laser range finder (LRF) 20 and a rear laser range finder (LRF) 30 are provided on the front and rear surfaces of the robot 1000. Moreover, the path | route measurement sensor 10 is attached to the wheel which is a moving mechanism.

  FIG. 7 is a diagram for explaining the concept of distance measurement processing by the laser range finder.

  The laser range finder is a device that irradiates laser light from the robot 1000 at an angle b and measures the distance z to the target object at the angle b. By scanning while changing the angle b, the distance to the object within a certain angle range can be acquired.

  As shown in FIG. 7, it is possible to acquire the distance to the object existing at 360 degrees around the robot by the front surface LRF 20 and the rear surface LRF 30 in each one scan.

  FIG. 8 is a diagram for explaining the concept of the process for specifying the position of the robot 1000.

  As shown in FIG. 8A, when only the path sensor 10 is used for specifying the position of the robot 1000, the robot 1000 actually moves even if the start point and the end point coincide with each other. The position of the robot specified inside will shift.

  On the other hand, as shown in FIG. 8B described below, when the path length sensor 10 and the distance measurement data of the front surface LRF 20 and the rear surface LRF 30 are combined, even at a position specified inside the robot, the starting point And the end point match.

  Thus, in order to combine the distance sensor 10 and the distance measurement data of the front surface LRF 20 and the rear surface LRF 30, as described below, in this embodiment, the weighted M pieces are used in evaluating the robot position. Use a particle filter approach with multiple particles.

Each of the m particles includes, as its attributes, the candidate position {xm (k), ym (k)} and candidate direction {qm (k)} of the robot 1000 and the particle weight {wm (k)}. State vectors s ^m [t] = {xm (k), ym (k), qm (k), wm (k)} are associated.

  While the robot 1000 moves, each particle also moves based on path length measurement and a stochastic motion model (which describes uncertainty in the robot motion).

  Further, in the process of calculating the weight for the particle, the particle filter estimates the posterior probability in consideration of the likelihood based on the measurement by the laser range finder.

  FIG. 9 is a flowchart for explaining the process of specifying the position of the robot 1000 using the particle filter.

  Referring to FIG. 9, when the process is started, first, the robot position specification processing unit 1040 initializes the attribute of each particle (S100).

Subsequently, the robot position specifying processing unit 1040 increments the variable m sequentially from 1 to the maximum value mmax of m, and calculates the particle state vector s ^m [t] from the path measurement result, the motion model, and the previous time. Calculation is performed based on the state vector s ^m [t−1] (loop of S102, S104, S106, and S108).

FIG. 10 is a diagram for explaining the concept of calculating the particle state vector s ^m [t].

As shown in FIG. 10, since the path sensor 10 measures the speed v_r of the right wheel and the speed v_l of the left wheel, the robot position specifying processing unit 1040 causes the speed V of the robot 1000 and Each speed R can be calculated. Thus, from the previous time the state vector s ^m [t-1], leading to calculation of a state vector of the particles after the lapse of Δt time s ^m [t]. Each particle is different not only in position but also in direction.

Returning to FIG. 9, subsequently, the robot 1000 is controlled by the processor 1010 to initialize the variable m for counting particles to 1 (S108), and initialize the measurement angle and the variable k (S110). ) After that, distance measurement to the object is performed by the laser range finders 20 and 30 every predetermined angle Δθ from the angle θ ₀ (S112).

  Subsequently, the robot position specification processing unit 1040 calculates the likelihood for each particle based on the distance to the target object at the time t in each direction at time t (S114).

  FIG. 11 is a diagram showing the likelihood of each particle based on the distance to such an object.

  The likelihood based on the distance measurement of such a laser range finder is not intended for particles, but is described in the following documents.

Reference 6: S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics (Intelligent Robotics and Autonomous Agents), The MIT Press, 2005.
As shown in FIG. 11A, when the distance to the object is z _t ^* by the laser range finder, such likelihood (posterior probability distribution) is obtained by superimposing the following probability distributions. .

i) Gaussian distribution centered on the measurement distance z _t ^* in consideration of measurement noise ii) Probability that an unexpected object is detected when the robot moves, and a distribution that decreases exponentially with distance iii) Probability that ranging data becomes maximum when there is no object within the maximum distance measurement range iv) Probability distribution corresponding to white noise due to random factors However, in the following description, for simplicity, The likelihood will be described on the assumption that it is expressed as shown in FIG.

  FIG. 12 is a conceptual diagram for explaining a procedure for calculating the likelihood of particles based on the measurement result (ranging data) of the laser range finder.

  Consider the case where the distance from the actual position of the robot 1000 to the object at the angle θk is measured as z * from the actual position of the robot 1000 as shown in FIG.

  At this time, with respect to the particle 1, it is calculated that the distance to the object at the angle θk is Z1 * in the geometric map, assuming that a ray is emitted from its position and orientation. Further, it is assumed that a light ray is emitted from the direction and that the distance to the object at the angle θk is calculated to be Z2 * in the geometric map.

  Here, the process of irradiating light rays virtually from the particles to the object also corresponds to the fact that irradiating light rays at a predetermined angle in the actual scanning operation of laser light is called “ray casting”. This is called “Raycast”.

  At this time, based on the probability distribution shown in FIG. 11, the robot position specifying processing unit 1040 calculates the likelihood of the particle 1 as L1, and the likelihood of the particle 2 as L2, as shown in FIG. Calculate (S114).

  If the angle θk does not reach the maximum value (S116), the robot position specifying unit 1040 updates the angle θk by Δθ (S118), and measures the distance to the object at the angle θk with the laser range finder. Based on the calculation of the distance to the object at the angle θk for each particle, the likelihood is calculated and integrated by the probability distribution shown in FIG. 11 (S114).

  FIG. 13 is a conceptual diagram showing a procedure of distance measurement by a laser range finder and ray casting from a certain particle.

  As shown in FIG. 13, while scanning at a different angle, the result of distance measurement from the robot 1000 using the laser range finder, and the distance to the object calculated on the geometric map by ray casting from each particle Thus, the likelihood of particles at each angle is calculated, and the likelihoods obtained for the angles are integrated.

  Returning to FIG. 9 again, by repeating such processing until the angle θk reaches the maximum value, the robot position specifying processing unit 1040 determines the weight of each particle in proportion to the likelihood (S120). ).

  The processes from step S110 to step S120 as described above are repeated until the maximum number mmax of particles is reached while incrementing the value of the variable m (loop from S110 to S124).

  Further, the robot position specification processing unit 1040 estimates the probability density distribution of the robot position from the current state represented by the particle and its weight (S126). Further, the robot position specification processing unit 1040 estimates the current state (position and orientation) of the robot using the estimated probability density distribution (S128).

  Then, the robot position specification processing unit 1040 performs resampling processing on the particle filter using the estimated probability density distribution (S130). That is, for example, particles are restored and extracted with a probability proportional to the probability density distribution. That is, the extraction frequency of particles with a large probability is increased, and particles with a small probability are extinguished to obtain a particle set at time t.

  Thereafter, the process returns to step S102.

  As a result of the above processing, after the ray casting, the particles more accurately reflecting the state of the robot have a higher likelihood score and are assigned more weight.

The geometric map creation unit 1030 may be configured to update the geometric map based on the particle dispersion state and the laser scan matching.
(Accumulation of likelihood and sound source mapping for occupied cells)
In the following, a description will be given of a process for specifying the position of the sound source in space in conjunction with the process of estimating the direction of the sound source and specifying the position of the robot 1000 using a particle filter as shown in FIG.

  During autonomous movement, the k-th audio scan {qn (k), Jn (k)} is expressed in robot coordinates, but in the process executed in this embodiment, in the geometric map in global coordinates. To evaluate the position of the sound source, an audio scan is combined with information about the robot's posture in the global coordinate system.

  In other words, from the posture (position and direction) of the robot represented by each particle, ray casting is performed in a plurality of directions specified as the direction of the sound source, and a process of finding an occupied cell on the geometric map hit in these directions is performed. . In other words, this corresponds to assuming that the acoustic source is represented (visible) as an occupied cell in the geometric map.

  FIG. 14 is a diagram for explaining the concept of ray casting for specifying the position of a sound source from such particles.

  As described with reference to FIG. 9, the knowledge about the robot posture is included in the particle distribution.

  When the kth audio scan is performed, raycast is performed from each of M (= mmax) particles in order to propagate the knowledge of the audio direction qn (k) from this particle, as shown in FIG. Is done.

  When such a raycast hits the occupied cell {i, j}, the sound source position estimating unit 1070 increases the likelihood cij (k) for the occupied cell by the particle weight wm.

  Therefore, the likelihood that a certain occupied cell is a sound source position is calculated by such ray casting.

  FIG. 15 is a flowchart for explaining processing of accumulating likelihoods and acoustic source mapping for occupied cells.

  FIG. 16 is a diagram showing a map obtained for one of the test environments. In the following, FIG. 16 is used as a specific example for explaining FIG. FIG. 16 (a) shows a geometric map of the corridor with acoustic source and robot trajectory, and FIG. 16 (b) is a partial enlargement showing the likelihood Cij (k) for the occupied cell obtained after ray casting. The figure is shown. A black X represents an occupied cell).

  Referring to FIG. 15, sound source position estimating section 1070 initializes variable k (S200), and then obtains sound in the robot coordinate system acquired from sound source direction estimating section 1060 as a result of the kth audio scan. Direction of arrival qn (k) and power Jn (k) are acquired (S202).

  The likelihood Cij (k) for the above-mentioned occupied cell is generated to have a significant value only for “useful” scans.

  Therefore, as shown in S204, likelihood accumulation processing is performed for scans in which the scan has a prominent peak and the amount of change in power is greater than a predetermined value. In other words, a scan having the same power as that in all directions is discarded and no accumulation process is performed.

  Therefore, in S204, the kth audio scan is selected in the following case.

Here, εmax and εrange are predetermined threshold values.

  When the audio scan satisfies the above-described conditions (Y in S204), the sound source position estimation unit 1070 initializes the variable n (S206), initializes the variable m and the variable pmax (S208), m For the second particle, ray casting is performed from the position [xm (k), ym (k)] to the direction [qm (k) + qn (k)] (S210).

  The sound source position estimation unit 1070 is used when the raycast hits an occupied cell in the geometric map, or reaches a non-search area (gray area in FIG. 4A) in the map, or When the maximum range d is reached, the raycast is terminated (S212).

  For the cell {i, j}, the hit rate for the k-th scan is represented as Cij (k).

  After the count of hits is initially set to zero, when the cell {i, j} hits a raycast from the m-th particle of the k-th scan, the sound source position estimation unit 1070 selects Cij (k ) Is updated as follows.

Here, wm is the weight of the mth particle.

  The hit cell is added to the list of hit cells, and the list size is incremented by pmax (S214).

  The sound source position estimation unit 1070 repeats such processing for mmax particles (loop from S210 to S218).

  In the sound source position estimation unit 1070, after ray casting from each of the mmax particles, cij (k) represents the probability that the cell {i, j} has hit.

  Thus, the audio direction qn (k) is associated with several cells and the probability for each such cell will be evaluated.

  FIG. 16B shows a cell hit by scanning and its hit probability.

  Returning to FIG. 15, during raycast, cells {i, j} are assigned a likelihood that is a function of audio scan power as follows.

  In the sound source position estimation unit 1070, the variable p is initialized (S220).

  And for the selected scan, the likelihood function is given as:

Here, α is a value for controlling the sharpness of the likelihood peak, and has the following relationship.

The parameter εnorm controls the range of likelihood for low power scans.

  The likelihood is scaled to the following range:

When cij (k) <εproba (Y in S224), log likelihood is assigned to the cell {i, j} as follows (S228).

In other cases (N in S224), the log likelihood is as follows (S226).

Note that the initial value of the log likelihood is as follows:

During accumulation, the logarithmic likelihood is thresholded so as to be in the range of [−LRmax, LRmax].

  The sound source position estimation unit 1070 repeats such processing for all the cells in the hit cell list (loop from S222 to S232).

  Subsequently, the sound source position estimation unit 1070 increments the value of n if the variable n (a variable indicating which sound source is) is not greater than nmax−1 (N in S234) (S238). The processing from step S208 to S234 is repeated.

  If the variable n has reached the maximum number nmax of sound sources (Y in S234), the sound source position estimation unit 1070 continues when the variable k indicating the audio scan is not larger than the maximum value kmax-1 ( N in S236), the value of k is incremented (S205), and the processing from step S202 to S234 is repeated.

  Further, when the processing for all audio scans is completed (Y in S236), the sound source position estimating unit 1070 finds cells having higher log likelihoods, divides these cells into clusters (S240), and sets each cluster. By calculating the weighted average position in these weights (the logarithmic likelihood Lij (k) of the selected cell), sound source localization in the geometric map is performed (S242).

  It is to be noted that the position estimation of the acoustic source and the update of the acoustic source map are periodically performed along with the movement of the robot 1000, or irregularly performed when a predetermined event occurs. It is possible to update the acoustic source map to the latest state.

  Further, the sound source position estimation unit 1070 may count the number of audio scans through which the cell {i, j} is foreseen as Kij (k). In that case, this count number is used to exclude occupied cells that are rarely foreseen.

When there is no prior information regarding sound source localization, it is assumed that p _{ij, init} = 0.5 for all occupied cells {i, j} (flat initial setting).
(Experimental result)
Hereinafter, an experimental result of the acoustic source map using the robot 1000 as described above will be described.

  The experiment was performed on the corridor of the geometric map of the environment as shown in FIG. In FIG. 16, the dimension of the cell in this map is 5 cm × 5 cm.

  Since the data of the geometric map of such an environment is held, the robot 1000 can move autonomously in the hallway.

  At the beginning of the experiment, the robot 1000 is given information for creating a geometric map for each set of positions that can be looped to cover all parts of the hallway.

  In the following, “one trial” corresponds to the robot moving in one loop of the hallway.

  An acoustic source map was generated during these trials, as described below.

  Several (up to four) acoustic sources S1-S4 were installed in the environment (these locations are located on the scanning surface of the laser range finder).

  These acoustic sources are loudspeakers that reproduce the recorded sound, and the sound from each acoustic source installed in the environment is the sound S1 of the air conditioning equipment (when measured at a distance of 5 cm, 78 Sound with a sound pressure of 5 dbA), sound S2 of a desktop computer fan (77.5 dbA), sound S3 of a server rack (77 dbA), and sound S4 of a popular song (73 dbA).

  The sound pressure in the quiet corridor was about 42 dBA.

  FIG. 17 is a diagram illustrating parameters used for specifying the acoustic source position.

  In FIG. 17, audio scan selection parameters (εmax and εrange), likelihood function shape (εnorm, α) in audio scan, likelihood accumulation (Lmax, Lmin, εproba, LRmax), and raycast The set value of the maximum distance d is shown. The maximum distance d used for ray casting was set to 3 meters. This is because the sound emitted from the acoustic source becomes less than the noise level of the space when it is more than 3 meters away. For ray casting processing, longer distances result in more variable results and increased computational load.

  In the trial in the experiment, the activation pattern of each acoustic source is changed.

  FIG. 21 is a table showing the activation pattern of the acoustic source for each trial.

  Also, among the trials that were performed, people were walking in the environment and talking in four trials.

  The log likelihood of the occupied cell is updated when audio scanning is possible during autonomous movement.

The latest raw acoustic source map is generated and made available after each audio scan. FIGS. 18-20 are diagrams showing raw acoustic source maps obtained in the case of a flat initial setting.

  18 (a) (b), FIG. 19 (c) (d), and FIG. 20 (e) (f), respectively, in the case of a flat initial setting, a plurality of times under the following conditions: It is a figure which shows the raw acoustic source map obtained at the end of trial.

  (o) is ground truth and (x) is the detected source. FIG. 18 (a) shows trial 1 without an active source and FIG. 18 (b) shows trial 2 with active sources S1, S2 and S3.

  FIG. 19 (c) shows trial 3 with active sources S1, S2 and S3, and FIG. 19 (d) shows trial 4 with sources S1, S2 and S3 active while people walk around and talk. .

  FIG. 20 (e) shows trial 5 in which the sources S1 and S3 are active, and FIG. 20 (f) shows trial 6 in which the sources S1, S2, S3 and S4 are active.

  For these raw acoustic source maps, cells with positive likelihood are clustered.

  The location of the cluster centroid is shown as a black x. The actual position of the acoustic source is given as a black circle.

  For each of the clusters, the sum of the number of cells and the log likelihood is shown in FIG.

  In FIG. 21, S * represents that it was erroneously detected where there was no source.

  As shown in FIG. 21, the approach of the present embodiment can successfully evaluate the position of the acoustic source.

  For the flat initial setting, the average localization error was 12.88 cm.

  Considering that the size of the cell is 5cm x 5cm and the loudspeaker is not a point sound source, but may be spread over multiple cells, the error that falls within the range of several cells was accurately localized. It shows that.

  The highest log likelihood sum obtained from an erroneously detected sound source is 687 (FIG. 20 (f)), and the smallest log likelihood sum 2116 obtained for an active sound source (FIG. 19 (d)). Smaller than)).

  Moreover, this low value corresponds to trial 4 when two people are moving while speaking in the environment during mapping.

  In the above description, the robot 1000 that is a moving body has a mechanism that moves autonomously, i) a process that specifies its own position, ii) a process that specifies the direction of the acoustic source, and iii) It has been described that the process of specifying the position of the acoustic source, iv) the process of holding or updating the geometric map, and v) the process of holding or updating the acoustic source map are all performed in the robot 1000 alone.

  However, when considered as an acoustic source map creation system, part or all of the processes from i) to v) may be executed as a process of another computer that can communicate with the robot 1000 by wireless communication or the like.

  In addition, i) in the process of specifying its own position, the movement of each particle in the particle filter has been described as being calculated by the measured value of the path length measurement sensor. A configuration may be used in which a sensor group disposed in the environment is used to calculate a measured value of the position of the robot.

  In addition, the robot has been described as using a laser range finder as a device for measuring the distance to an external object, but the distance measurement that can measure the distance to the object and model the measurement result as a predetermined probability distribution If it is a sensor, you may use another sensor.

  As described above, according to the moving body of the present embodiment and the method of creating the acoustic source map using the moving body, the acoustic source that has the function of specifying the position of the sound source and can be used for actual voice communication is used. Prior information can be collected.

  In particular, by combining a geometric map and an audio scan, it is possible to create an acoustic source map more accurately without increasing the calculation load. This is because with this approach, only the occupied cells of the geometric map are considered in the raycast. In ray casting for this occupied cell, the knowledge about the robot posture (reflected in the particle attributes) is used to update the likelihood of the occupied cell.

  Thus, for example, such an acoustic source map will assist the robot in acquiring better knowledge about environmental noise.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10 path length measurement sensor, 20 front laser range finder, 30 rear laser range finder, 40 sensor input / output board, 50 memory, 60 bus, 61 eigenvector calculation unit, 62 MUSIC processing unit, 86 correlation matrix calculation unit, 88 eigenvalue decomposition unit, 104 MUSIC spatial spectrum calculation unit, 106 MUSIC response calculation unit, 110 sound source direction estimation processing unit, MC1 microphone array, 1000 robot, 1010 processor, 1030 geometric map creation unit, 1040 robot location specification processing unit, 1050 sound source power spectrum acquisition unit , 1052 Microphone array, 1054 Sound input / output board, 1060 Sound source direction estimation unit, 1070 Sound source position estimation unit, 1100 Non-volatile storage device, 1102 Geometric map, 1104 Acoustic source map

Claims (10)

A moving object,
Moving means for driving the moving body;
Storage means for storing a geometric map for specifying a geometric position of an object in a movable space of the moving body;
Position measuring means for measuring the current position of the moving body and outputting a position measurement result;
Ranging means for obtaining a distance from the moving body to an object specified by the geometric map in the space as ranging data according to a predetermined probability distribution;
Position estimation means for estimating the position of the moving body by a particle filter including a plurality of particles, each of the particles has position and orientation information in the space as attributes,
The position estimating means includes
Based on the position measurement result, each of the particles is moved, and based on the distance measurement data and the probability distribution, a first likelihood for the current attribute of each of the particles is calculated, and a weight of the corresponding particle is calculated. Means to
As a weighted average of each of the particles, including moving body position estimating means for estimating a current position of the moving body,
A sound sensor array;
A direction-of-arrival estimation means for executing processing for specifying a direction of arrival of sound in the sound sensor array based on a signal from the sound sensor array;
Sound source localization means for estimating the position of the acoustic source based on the attribute of each particle and the direction of arrival;
The sound source localization means obtains a second likelihood for the hit position in response to a ray cast from the position of each particle in the arrival direction hitting an object specified by the geometric map. A moving object that accumulates and estimates the position of the acoustic source based on the accumulated second likelihood. The distance measuring means is a laser range finder,
The first likelihood is determined when the geometric distribution is obtained by ray casting from the position of each particle to the direction of the particle in the probability distribution based on the measurement result of the distance from the current position of the moving object to the object. The moving body according to claim 1, wherein probabilities corresponding to distances until the object is hit in a map are integrated at predetermined angular intervals. The direction-of-arrival estimation means performs a plurality of sound scans,
In the second likelihood calculation, the sound source localization means selects a sound scan in which at least one of a maximum value or a change amount of sound power exceeds a predetermined threshold value, and executes the accumulation process. The moving body according to claim 1 or 2. The second likelihood increases with the sound power and is defined by a function having a value scaled to a predetermined range;
The sound source localization means performs the accumulation process so that the second likelihood increases for the hit position where the sound scan is selected and the number of hits exceeds a predetermined threshold. 3. The moving body according to 3.   The moving body according to claim 1, wherein the moving body creates an acoustic map indicating the position of the localized acoustic source in the geometric map and stores the acoustic map in a storage device.   The mobile body according to claim 5, wherein the mobile body is an autonomous mobile robot.   The said moving body is a moving body of Claim 6 which updates the said acoustic map as it moves autonomously.   The moving body according to any one of claims 5 to 7, wherein the position measuring unit is a path length measurement sensor that detects a moving distance and a moving direction of the moving body based on the movement of the moving unit. An acoustic source map creation system,
Moving means for driving the moving body;
Storage means for storing a geometric map for specifying a geometric position of an object in a movable space of the moving body;
Position measuring means for measuring the current position of the moving body and outputting a position measurement result;
Ranging means for obtaining a distance from the moving body to an object specified by the geometric map in the space as ranging data according to a predetermined probability distribution;
Position estimation means for estimating the position of the moving body by a particle filter including a plurality of particles, each of the particles has position and orientation information in the space as attributes,
The position estimating means includes
Based on the position measurement result, each of the particles is moved, and based on the distance measurement data and the probability distribution, a first likelihood for the current attribute of each of the particles is calculated, and a weight of the corresponding particle is calculated. Means to
As a weighted average of each of the particles, including moving body position estimating means for estimating a current position of the moving body,
A sound sensor array mounted on the moving body;
A direction-of-arrival estimation means for executing processing for specifying a direction of arrival of sound in the sound sensor array based on a signal from the sound sensor array;
Sound source localization means for estimating the position of an acoustic source based on the attribute of each particle and the direction of arrival;
The sound source localization means obtains a second likelihood for the hit position in response to a ray cast from the position of each particle in the arrival direction hitting an object specified by the geometric map. Accumulating and estimating the position of the acoustic source based on the accumulated second likelihood,
An acoustic source map creation system further comprising: an acoustic source map database that stores the estimated position of the acoustic source as an acoustic source map. An acoustic source map creation method comprising:
Driving and moving the moving body;
Storing a geometric map for specifying a geometric position of an object in a movable space of the moving body in a storage device;
Measuring the current position of the moving body and outputting a position measurement result;
Obtaining a distance from the moving object to an object specified by the geometric map in the space as distance measurement data according to a predetermined probability distribution;
A step of estimating the position of the moving body by a particle filter including a plurality of particles, each of the particles has position and orientation information in the space as attributes,
The step of estimating the position of the moving body includes:
Based on the position measurement result, each of the particles is moved, and based on the distance measurement data and the probability distribution, a first likelihood for the current attribute of each of the particles is calculated, and a weight of the corresponding particle is calculated. And steps to
Estimating the current position of the moving body as a weighted average of each of the particles,
Executing a process for specifying a direction of arrival of sound in the sound sensor array based on a signal from the sound sensor array mounted on the moving body;
Further comprising estimating a position of an acoustic source based on each particle attribute and the direction of arrival;
The step of estimating the position of the acoustic source includes a step in which a ray cast toward the arrival direction from each particle position hits an object identified by the geometric map, and Accumulating the likelihood of 2 and estimating the position of the acoustic source based on the accumulated second likelihood,
Storing the estimated position of the acoustic source in a storage device as an acoustic source map.