JP2005354223A - Sound source information processing apparatus, sound source information processing method, and sound source information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound source information processing apparatus with high practicality, capable of carrying out a processing with higher surety and performing processing matching the purpose by taking into consideration a surrounding environment, and to provide a sound source information processing method and a sound source information processing program. <P>SOLUTION: The sound source information processing apparatus includes a plurality of sound collection means (101-1 to 101-N) for collecting sound signals of a plurality of sound sources; a sound source estimating means 104 for estimating the direction of at least one sound source on the basis of the sound signals collected by a plurality of the sound collection means; a sound source information extracting means 108 for extracting the sound source information on a target sound source among the sound sources in the estimated direction; a sound source state information acquiring means 109 for acquiring the sound source state information on at least either of one or more sound source and surrounding states for the purpose of selecting processing contents applied to the extracted sound source information; and a processing means 110 for applying prescribed processing to the extracted sound source information on the basis of the sound source state information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a sound source information processing apparatus, a sound source information processing method, and a sound source information processing program for appropriately processing sound source information estimated using input signals from a plurality of sound collecting means in accordance with purposes and situations It is about.

  Conventionally, it is a method of collecting only a target sound in an environment with ambient noise, and various techniques of a microphone array using a plurality of microphones have been proposed (for example, see Non-Patent Document 1). These are mainly intended to determine the target audio from the signals input from multiple microphones, localize the sound source direction, and apply the filter processing that matches the directivity to the localization direction. The sound is collected. Here, in the case where a plurality of sound sources are considered in this way, the problem of extracting information on a target sound source is referred to as a sound source separation problem. Depending on the purpose, there may be cases where only the localization of the sound source direction or the extraction of audio information from the sound source is indicated.

  In such a sound source separation problem, there are several problems that greatly affect sound source separation in an actual environment. Such problems include the relationship between the number of sound sources and the number of microphones, noise and reverberation, and movement of sound sources. In order to deal with these problems, there are, for example, a means for obtaining directivity in a desired direction in a short time and a means for obtaining directivity even when the sound source direction changes with time (for example, Patent Document 1). In addition, there is a method dealing with a method of detecting the direction of a target sound source and suppressing noise stably with a small amount of computation (see, for example, Patent Document 2).

Japanese Patent Laid-Open No. 2002-186084 Japanese Patent Laid-Open No. 10-207490 Yutaka Kaneda "Microphone array technology for speech recognition under noise" Journal of the Acoustical Society of Japan, Vol. 53, No. 11 (1997), pp. 872-876

  In this way, effective methods and methods have been proposed for some items that are problematic in the actual environment, but in reality they have been proposed due to multiple causes of problems. There are cases where it deviates from the conditions assumed in the method. Especially when an adaptive signal processing method is used as a solution to these problems, it is common to formulate solving optimization problems under certain constraints. Then, the estimation itself may not work. In such a case, it is quite possible that unexpected performance and no use at all will result in not being able to obtain sufficient performance even if the sound source separation technology as described above is used. is there. In other words, it is considered that it is more appropriate to correct the information about the estimated sound source depending on the purpose and the surrounding environment, and the conventional technology did not always pay attention to such a point. There's a problem.

  In other words, as described above, in an environment where a sound source information processing system for processing a sound source separation problem is actually used, the method of estimating a sound source itself cannot sufficiently cope with the situation, or the system is expected depending on the surrounding environment. It is possible that the operation is specified. Therefore, at present, a sound source information processing system that is excellent in practicality in consideration of the surrounding environment has not yet been established.

  The present invention has been made in view of the above, and has a highly practical sound source information processing apparatus, sound source information processing method, and sound source information capable of performing more reliable processing in consideration of the surrounding environment. The purpose is to obtain a processing program.

  In order to solve the above-described problems and achieve the object, a sound source information processing apparatus according to the present invention is picked up by a plurality of sound collecting means for collecting sound signals of a plurality of sound sources and a plurality of sound collecting means. Sound source estimation means for estimating the direction of at least one sound source based on the obtained sound signal, sound source information extraction means for extracting sound source information relating to the target sound source from among the sound sources in the estimated direction, and processing performed on the extracted sound source information Sound source status information acquisition means for acquiring sound source status information related to at least one of one or a plurality of sound sources and surrounding conditions for selecting content, and for the sound source information extracted based on the sound source status information And a processing means for performing a predetermined process.

  In addition, a sound source information processing method according to the present invention includes a sound collection step for collecting sound signals of a plurality of sound sources, and a sound source for estimating a direction of at least one sound source based on the sound signals collected in the sound collection step. An estimation step, a sound source information extraction step for extracting sound source information relating to the output target from among the sound sources in the estimated direction, and one or a plurality of sound sources and surroundings for selecting processing contents to be performed on the extracted sound source information A sound source status information acquisition step for acquiring sound source status information related to at least one of the situations, and a processing step for performing predetermined processing on the extracted sound source information based on the sound source status information .

  The sound source information processing program according to the present invention includes a sound collection step for collecting sound signals of a plurality of sound sources, and a sound source for estimating the direction of at least one sound source based on the sound signals collected in the sound collection step. An estimation step, a sound source information extraction step for extracting sound source information relating to the output target from among the sound sources in the estimated direction, and one or a plurality of sound sources and surroundings for selecting processing contents to be performed on the extracted sound source information Causing a computer to execute a sound source status information acquisition step of acquiring sound source status information relating to at least one of the situations, and a processing step of performing a predetermined process on the extracted sound source information based on the sound source status information Features.

  According to this invention, appropriate processing is performed on the sound source information to be output estimated by the sound source estimation means based on a predetermined criterion. Specifically, predetermined sound source status information is acquired by the sound source status information acquisition means, and predetermined processing to be performed on the estimated sound source information (sound source information to be output) based on the sound source status information is performed. Select and apply this process. Therefore, the sound source information output in the present invention is information that has been appropriately processed in accordance with the purpose and the surrounding situation.

  The sound source status information is information that is not directly included in the final output, but is extracted in order to know the sound source and its surrounding information, that is, to be used for the determination of selecting processing in the processing means. . The sound source status information may be information obtained from the sound source information, or may be information other than audio information. Based on such sound source status information, the sound source information to be output is subjected to appropriate processing in accordance with the purpose and surrounding circumstances.

  According to the present invention, since sound source information to be output is appropriately processed in accordance with the purpose and the surrounding situation, it is possible to perform a more reliable process in accordance with the surrounding situation. Sound source information processing apparatus, sound source information processing method, and sound source information processing program can be obtained.

  Hereinafter, a sound source information processing apparatus, a sound source information processing method, and a sound source information processing program according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following description, In the range which does not deviate from the summary of this invention, it can change suitably.

  First, the basic concept of the present invention will be described. A sound source information processing apparatus according to the present invention includes a plurality of sound collecting means for collecting sound signals of a plurality of sound sources, and a direction of at least one sound source based on the sound signals collected by the plurality of sound collecting means. Sound source estimation means for estimation, sound source information extraction means for extracting sound source information related to the target sound source from among the sound sources in the estimated direction, and one or a plurality of sound sources for selecting processing contents to be performed on the extracted sound source information And sound source status information acquisition means for acquiring sound source status information regarding at least one of the surrounding situations, and processing means for performing predetermined processing on the extracted sound source information based on the sound source status information. It is a feature.

  Here, the sound source estimation means has, as a general function, for example, estimating the direction of one or a plurality of sound sources and emphasizing and outputting the sound in the estimated direction. This sound source estimation means can be realized by a microphone array, for example.

  The sound source information extracted by the sound source information extraction means is information related to the output from the sound source information processing apparatus, and is the target information output by the sound source information processing apparatus. For example, when sound is collected by two microphones and the sound source information processing apparatus outputs sound only in the direction in which the sound comes, the sound emphasized with respect to the estimated direction of arrival of the sound corresponds to the sound source information. When the processing device is applied to a robot or a response system, when the sound source information processing device outputs only from which direction the speaker is speaking, the angle of the estimated voice arrival direction corresponds.

  The sound source status information for selecting the processing content to be applied to the extracted sound source information acquired by the sound source status information acquisition means is not directly included in the final output from the sound source information processing apparatus, but the sound source and its This information is extracted in order to know the surrounding information, that is, to be used for determination by the processing means. For example, when a plurality of sound sources are estimated by the sound source estimation means and one of the estimated sounds is output from the sound source information processing apparatus, the respective sound volumes of the estimated plurality of sound sources are It can be used as situation information. This volume can be easily obtained from the estimated sound source estimated by the sound source estimating means.

  Also, for example, when a plurality of sound sources are estimated by the sound source estimation means and one of the estimated sounds is output from the sound source information processing apparatus, each of the estimated plurality of sound sources Can be extracted and used as sound source status information. Extraction of information about whether or not it seems to be a human voice can be realized by an existing technique such as estimating a zero crossing of a waveform or a harmonic structure.

  Further, for example, when a plurality of sound sources are estimated by sound source estimation means and one of the estimated sounds is output from the sound source information processing apparatus, each of the estimated plurality of sounds is Authentication is performed, and the result of speaker authentication can be used as sound source status information. For the authentication, an existing technique such as using a HMM (Hidden Markov Modeling) score for each speaker can be used.

  The processing means determines and selects what kind of processing is to be performed on the sound source information extracted by the sound source information extraction means based on the sound source status information acquired by the sound source status information acquisition means, and according to the result The sound source information extracted by the sound source information extracting means is subjected to a predetermined process. That is, the sound source information extracted by the sound source information extraction means is used as an input, and predetermined processing is performed on this input. The sound source information that has been subjected to the predetermined processing in this processing means is the final output content. Here, no processing may be performed on the input, or a form of predetermined processing may be used. In that case, the information input to the processing means is output as it is.

  In addition, for example, the apparatus includes holding means for holding processing contents to be performed on the sound source information extracted based on the sound source status information, and the processing means calls a predetermined processing procedure based on the sound source status information from the holding means. Processing can be executed. In this holding means, for example, when the input is above a certain threshold value, this processing is performed, and when the input is less than the threshold value, other processing is performed. ? "→" What is the age? "→ ..., etc., the information of the hierarchical structure of the judgment conditions is held as a table, and by using this table, the input and output in the processing means are Association can be performed.

  As the determination by such processing means, when using the sound volumes of the plurality of sound sources estimated as described above as sound source status information, for example, paying attention to the sound volumes of the plurality of sound sources, the output is scheduled. If the volume of the estimated sound source is higher than the volume of the other sound sources by a certain value or more, the estimated sound source information is output as it is as an advantageous environment for sound source estimation, otherwise the sound source estimation is difficult. It is possible to perform processing such as outputting only sound source information in the defined initial value direction. By performing such processing, if the sound source estimation is easy, the estimation information is used as it is, and if the sound source estimation is difficult, the entire system can operate reliably even if there is a sound source estimation error. Can be.

  Further, in the case where the estimated human voice like the sound as described above is used as the sound source status information, for example, the following processing can be performed on the input. It is assumed that human voice-like information can be stored for each candidate for a certain period of time, and that there is only one candidate whose human voice-like degree exceeds a predetermined threshold and continues beyond a predetermined time threshold. If not, the candidate is output even if the first candidate for sound source estimation by the sound source estimating means is not the voice in that direction.

  On the other hand, when there are more than a certain number of sound sources whose human voiceness exceeds a predetermined threshold, only the sound source information in the initial value direction is output. If neither of the above applies, the first candidate for sound source estimation of the sound source estimation means is output. By performing such processing, it is possible to capture human speech as much as possible even when the sound source estimation means makes a mistake in sound source estimation. When there are a plurality of sound sources that are likely to be human voices, the sound source information is obtained by performing a specific operation (for example, outputting only sound source information in a predetermined initial value direction) without forcibly estimating one sound. It becomes possible to give a certain regularity to the behavior of the entire processing apparatus, and it becomes easy to understand the behavior.

  Further, as described above, when the result of speaker authentication is used as the sound source state information, for example, the following processing can be performed on the input. For example, when a sound source information processing device is used in a dialogue device, if it is authenticated that the sound source estimated is a speaker who is currently speaking, the estimated sound source information is output and the sound source estimation is performed. If it is determined that the selected sound source is not the speaker who is currently speaking, the sound is not output by returning silence or the like. By performing such processing, even when the sound source is erroneously estimated with respect to the surrounding noise, it is possible to prevent reaction to other than the target sound source.

  In addition, when used as a system that outputs sound in the estimated sound source direction, the output volume can be changed depending on the estimated sound source direction. For example, the sound volume of the sound source whose sound source direction is estimated to be in front is increased and output, and the sound volume of the output is reduced as the estimated sound source direction shifts in the horizontal direction. If the output from the sound source information processing device is just the sound from the estimated sound source, it will not be clear from which direction the sound came from. However, if the system is turned left and right by using the above configuration, The volume changes even with the same sound source. Thereby, for example, when this system is applied to a robot, a situation close to human hearing can be realized, and information on the direction of the sound source can be perceived by a subsequent system.

  Further, when used as a system for outputting sound in the estimated sound source direction, it is possible to perform sound enhancement in a certain direction and output it. That is, when the processing means determines that the sound in a certain direction is collected and output, the processing means obtains the sound in the estimated direction by performing delay sum processing on the inputs from the plurality of microphones. Output it. For example, if the sound source estimation means estimates that there is a sound source in the direction of 10 degrees and tries to output sound in that direction, it determines that the output is not appropriate from the sound source status information about the sound source and outputs sound in the direction of 90 degrees Suppose that it is determined. In this case, the processing means performs delay sum processing or the like on the sound from the microphone, and emphasizes and outputs the sound in the corresponding direction. As a result, output suitable for the purpose can be performed even under a situation where an estimation error is likely to occur in the sound source estimation in the sound source estimation means.

(First embodiment)
Next, the sound source information processing apparatus, the sound source information processing method, and the sound source information processing program according to the present invention will be described in more detail based on specific embodiments.

  FIG. 1 is a block diagram showing a configuration example of a sound source information processing system to which the present invention is applied. A sound source information processing system 100 to which the present invention shown in FIG. 1 is applied includes N microphones (N ≧ 2) 101-1 to 101-N, an input processing unit 102, a microphone array processing unit 104, and a sound source information processing. And a signal processing unit 103 having a unit 105. Here, the sound source information processing unit 105 extracts the output information output from the sound source information processing system 100 out of the sound sources estimated by the microphone array processing unit 104, that is, the sound source for extracting the target sound source information output by the sound source information processing system 100. Information extraction means (function) 108, sound source status information acquisition means (function) 109 for acquiring sound source status information for selecting processing contents to be applied to the extracted sound source information, and sound source status information are extracted. And processing means (function) 110 for performing predetermined processing on the sound source information.

  In the sound source information processing system 100 configured as described above, first, sound is collected by N microphones 101-1 to 101-N and input to the input processing unit 102. The microphone is not particularly limited, and a commonly used microphone can be used as long as sound can be reliably collected. The N microphones may all be the same type of microphone, and if the frequency characteristics and the like are measured, the difference in sound collection characteristics can be absorbed by subsequent signal processing. For example, different types of microphones can be used.

  These sounds collected by the microphones 101-1 to 101 -N are converted into a format that can be processed by a digital signal by an input circuit 102 by an amplifier circuit, an A / D converter, or the like. The configuration of the input processing unit 102 is not particularly limited as long as the sound collected by the microphone can be converted into a form that can be processed with a digital signal. The configuration of the input processing unit 102 is not limited to an existing electronic circuit or microphone array. It can be realized by various means such as an input system. Further, transmission of signals between the microphones 101-1 to 101-N and the input processing unit 102 may be wired or wireless. This is not particularly limited as long as a signal can be transmitted to the input processing unit 102, and can be realized by an existing signal transmission technique.

  The audio signal converted into a format that can be processed by the input processing unit 102 is input to the signal processing unit 103. The signal processing unit 103 performs the following signal processing. First, microphone array processing is performed in the microphone array processing unit 104. In the microphone array processing, depending on the purpose of use of the sound source information processing system 100, sound source localization is performed, sound in the localization direction is further emphasized, or sound in a specified direction is emphasized to suppress sound from other directions. You can do it. That is, the microphone array processing unit 104 performs signal processing for obtaining a feature amount required by the sound source information processing system 100, and in particular, performs processing for obtaining sound source information from audio signals collected by a plurality of microphones. Is called.

  In response to the information from the microphone array processing unit 104, the sound source information processing unit 105 extracts sound source information related to the target sound source output from the sound source information processing system from the sound source information from the microphone array processing unit 104 by the sound source information extraction function. The sound source status information for selecting the processing content to be applied to the extracted sound source information by the sound source status information acquisition function is acquired, and the extracted sound source information is obtained based on the sound source status information by the processing function. Then, a predetermined process is performed, and information 106 according to the output target of the sound source information processing system is output 106 in a predetermined format.

  The signal processing unit 103 including the microphone array processing unit 104 and the sound source information processing unit 105 is not particularly limited as long as the above-described processing can be performed, and an existing signal processing chip, electronic circuit, or the like is used. The digital signal processing can be performed in a computer. Further, it can be realized by a process in a computer.

  In the input processing unit 102, the signal processing unit 103, the microphone array processing unit 104, and the sound source information processing unit 105 described above, names are individually given in the function of the present invention. It may be distributed in such a relationship that information can be exchanged on the network. Further, in order to adjust the individual processing units named in the function, the entire processing may be managed by a system higher than the sound source information processing system. In the present specification, in the sound source information processing system 100, the signal processing unit 103 including the microphone array processing unit 104 and the sound source information processing unit 105 is referred to as a sound source information processing apparatus.

  Next, a specific embodiment when the above-described sound source information processing system 100 is applied to the automobile 201 will be described. FIG. 2 is a diagram for explaining an embodiment in which a sound source information processing system to which the present invention is applied is applied in an automobile. A driver seat 202a and a passenger seat 203a in the automobile 201 include a driver seat user 202 and a passenger seat user 203, respectively. Further, a sound source information processing system 204 in which a microphone is integrated is disposed on the ceiling portion in the automobile 201. The sound source information processing system 204 can perform predetermined processing by communicating information with the in-vehicle information system 205. Information transmission between the sound source information processing system 204 and the in-vehicle information system 205 may be performed wirelessly or by wire. The in-vehicle information system 205 can also obtain various types of information from the operation system 206 of the automobile, whereby the in-vehicle information system 205 can know the driving situation of the automobile 201, and further, this driving situation information can be obtained. It can also be transmitted to the sound source information processing system 204. The details of the sound source information processing system 204 have the same configuration as that of the sound source information processing system 100 shown in FIG. 1 described above.

  Here, it is assumed that the users 202 and 203 and the in-vehicle information system 205 can exchange information by voice via the sound source information processing system 204. The voice commands of the users 202 and 203 may be affected by ambient noise or utterances from speakers other than those in conversation with the in-vehicle information system 205. And if the voice affected by such influence is transmitted as it is, the in-vehicle information system 205 cannot obtain sufficient performance, and there is a possibility that it will not be used at all as an unexpected operation or result. is there. In other words, the in-vehicle information system 205 may not function properly depending on the surrounding environment.

  Therefore, if the sound source information processing system 204 can accurately process the utterance from which the current voice command is uttered, the utterance from an appropriate speaker can be collected, and the result is sent to the in-vehicle information system 205. It becomes possible to send. Information on such a sound source and its use will be described later with an example.

  Here, the flow of processing in the sound source information processing system 204 will be described using the flowchart shown in FIG. First, when the dialogue between the users 202 and 203 and the in-vehicle information system 205 is started (S301), the sound source information processing system 204 collects in-vehicle sound by N (two or more) predetermined microphones ( S302). The collected audio signal is input to the input processing unit 102. The input processing unit 102 takes in an audio signal from the microphone and converts it into a format that allows digital signal processing (S303). The amplification of the received signal in the input processing unit 102 is calibrated so that when the sound source information processing system 204 is configured, any microphone can obtain the same sound volume when there is a constant sound volume in a certain direction. And Alternatively, the system or the user may arbitrarily calibrate by a predetermined operation. Since the calibration method itself is not directly related to the present invention, the details are omitted.

  Next, the audio signal converted into a format that can be processed by the input processing unit 102 is input to the signal processing unit 103. From the sound signal input to the signal processing unit 103 in this manner, the sound source is estimated according to the purpose of the sound source information processing system 204 by the signal processing in the microphone array processing unit 104 (S304).

  The microphone array processing unit 104 can determine the position of the sound source or pick up the sound in the target direction based on the phase difference or volume of the received signal. The signal processing in the microphone array processing unit 104, that is, the microphone array processing is not limited in the present invention as long as sound source information suitable for the purpose can be obtained as described in the following embodiments. It can be realized by a technique such as a delay-and-sum array or an adaptive array as mentioned in the publicly known document of Non-Patent Document 1. The details are omitted here. Similarly, regarding the arrangement and orientation of the N microphones, the present invention is only required to be able to cope with the purpose. Here, a description will be given assuming that two microphones are subjected to microphone array processing according to Patent Document 2. In this case, two adaptive beamformers are configured for two microphones, and one is controlled so that one suppresses the target sound source and the other suppresses the noise source.

  The sound source information estimated by the microphone array processing unit 104 is input to the sound source information processing unit 105. The sound source information processing unit 105 extracts the sound source information of the target sound source to be output from the sound source information processing system 100 from the sound sources estimated by the microphone array processing unit 104, and selects the processing content to be applied to the extracted sound source information Sound source status information is acquired. As the sound source status information, various kinds of information such as information on a sound source estimated by the microphone array processing unit 104, information on another sound source, or information other than the information on the sound source can be used. Then, based on the sound source status information, it is determined whether or not a predetermined process needs to be performed on the estimated sound source information and what kind of process is performed. Details of this determination and details of processing will be described later.

  When it is determined that it is necessary to perform predetermined processing on the extracted sound source information in response to the determination result (Yes in S305), the sound source information processing unit 105 performs a predetermined rule on the sound source information. Processing is performed (S306). Then, the sound source information subjected to the predetermined processing is converted into a predetermined output format as final output information from the sound source information processing system 204 (S307), and is output to the in-vehicle information system 205 (S308). A series of processing ends. Here, the output format from the sound source information processing system 204 can be appropriately changed depending on the purpose.

  On the other hand, if it is determined that it is not necessary to perform predetermined processing on the extracted sound source information (No at S305), the extracted sound source information is used as the final output information from the sound source information processing system 204 as the predetermined output information. It converts into an output format (S307), it outputs with respect to the vehicle-mounted information system 205 (S308), and a series of processes are complete | finished.

  In the description of the present embodiment, the output from the sound source information processing system 204 is assumed to be the sound in the direction estimated as the target sound source. The above processing can be performed at regular time intervals or at arbitrary time intervals. The output in the embodiment described here is assumed to be continuous. That is, a sound source for each unit time is estimated, and during the unit time, sound in the target sound source direction is picked up and processed and used as an output of the sound source information processing system 204.

  Next, determination of whether or not a predetermined process needs to be performed on the extracted sound source information and the process will be described. In the following, the output of the microphone array processing unit 104 will be described with reference to FIGS. 4 and 5 in the case where the user's utterance in the vehicle is localized and the sound is collected by directing the direction. In addition, it is assumed that one of the two beamformers collects the target speech by suppressing the noise direction, and the other collects the noise by suppressing the direction of the target sound source. Then, measure the intensity of the noise that is not the target sound source, set a predetermined threshold value to determine whether or not the surrounding noise other than the target sound source is strong, and add the predetermined sound source information (target sound source information) to the predetermined sound source. It is determined whether or not processing is to be performed. Here, FIG. 4 is a table in which procedure information for associating the determination condition when the noise intensity is the sound source state information and the processing content for the sound source information. This table is held in, for example, the information holding unit 107 in the sound source information processing unit 105, and this information is appropriately called by the sound source information processing unit 105 to perform predetermined processing. FIG. 5 is a flowchart showing a determination as to whether or not a predetermined process needs to be performed on the extracted sound source information, and the process flow.

  First, as the sound source status information indicating whether or not the sound source information processing unit 105 needs to perform predetermined processing, the intensity of noise that is not the target sound source is acquired. Then, it is determined whether or not the noise intensity is lower than a predetermined threshold (S501). Here, when it is determined that the noise intensity is lower than the predetermined threshold (Yes in S501), it is determined that the extracted sound source information is uttered from the user and the predetermined process is unnecessary (S502). The extracted sound source information becomes the final output from the sound source information processing system 204 as it is. Therefore, the extracted sound source information is directly converted into a predetermined output format (S307) and output to the in-vehicle information system 205 (S308), and a series of processing ends. As a result, the in-vehicle information system that receives the output can pick up the user's utterance at any location in the car, use the voice for voice recognition, use for hands-free phone calls, etc. Can do.

  On the other hand, if it is determined that the strength of the noise is higher than the predetermined threshold (No in S501), the surrounding noise is strong and the estimation accuracy such as the sound source localization of the target speech is reduced. Particularly in automobiles, it is expected that it is difficult to use voice information when traveling at high speed or when the weather outside is bad. If the target sound source information estimated in such a case is output as it is, it is difficult to estimate, and an output of an erroneous estimation result is output from the sound source information processing system, which adversely affects the processing in the in-vehicle information system 205 at the subsequent stage. May occur. Specifically, a passing sound with an oncoming vehicle running at high speed is erroneously estimated as a user's utterance and sent to the in-vehicle information system, and the in-vehicle information system interprets the voice and interprets a command not intended by the user. It is possible that

  Therefore, as a result of determining the noise intensity, if it is determined that the noise intensity is higher than a predetermined threshold (No in S501), it is determined that a predetermined process is required for the extracted sound source information (S503). Then, the sound source information of the estimated target voice is ignored, the beamformer of the target voice is directed only in the driver seat direction, and only the voice in the driver seat direction is collected and extracted (S504). Then, the sound source information subjected to the above-described processing is converted into a predetermined output format as final output information from the sound source information processing system 204 (S307) and output to the in-vehicle information system 205 (S308). A series of processing ends.

  By performing such processing, the possibility that the in-vehicle information system 205 malfunctions due to noise from directions other than the driver's seat direction is greatly reduced. Of course, there is a possibility that the user's voice comes in from other than the direction of the driver's seat, but there is not much need for voice input from other than the driver's seat in situations where there is a strong noise, such as high speed driving as described above. Unthinkable. In addition, by applying predetermined processing to the sound source information in a noisy situation, the possibility of malfunction of the in-vehicle information system due to a sound source estimation error is reduced, and the sound from the driver's seat is at least reliably collected. The effect of doing can be obtained. As described above, by performing predetermined processing on the sound source information to be output, it is possible to obtain an effect that more practical processing can be performed according to the surrounding situation.

  Further, when outputting sound in the estimated sound source direction (or sound in a specific direction subjected to predetermined processing) from the sound source information processing system 204, there may be a case where a time difference due to processing must be taken into consideration. For example, it is assumed that it takes 1 second to estimate a sound source and that it takes 2 seconds to extract and determine information related to the sound source, assuming that sound enters at time 0. If it is determined that the voice in a certain direction is emphasized and output, it is reflected only from the voice after (1 second + 2 seconds). Therefore, in such a case, a ring buffer is arranged in the signal processing unit 103 so that information from N microphones can be saved in an appropriate format (1 second + 2 seconds) or more. By performing the processing on the voice that has been processed, an output corresponding to the present invention can be obtained.

  FIG. 6 is a diagram illustrating a hardware configuration of the sound source information processing system 204 according to the first embodiment. The sound source information processing system 204 has, as a hardware configuration, a microphone array processing program for executing microphone array processing in the sound source information processing system 204, the above-described sound source information extraction function, sound source state information acquisition function, and processing function in the sound source information processing unit 105. ROM 252 storing a program for executing the processing related to the above, a CPU 251 for controlling each part of the sound source information processing system 204 according to the program in the ROM 252 and executing a buffering time changing process, and a work area are formed, Each unit is connected to a RAM 253 that stores various data necessary for controlling the sound source information processing system 204, a communication I / F 257 that communicates by connecting to a network, a microphone 258 that collects a sound signal from the sound source. With bus 262 It is provided.

  Programs such as the microphone array processing program in the sound source information processing system 204 described above are files in an installable or executable format, such as a CD-ROM, a magnetic disk such as a floppy (registered trademark) disk (FD), a DVD. For example, the program may be provided by being recorded on a computer-readable recording medium.

  In this case, a program such as a microphone array processing program is loaded on the main storage device by being read from the recording medium and executed by the sound source information processing system 204, and each unit described in the software configuration is stored on the main storage device. To be generated.

  Further, a program such as the microphone array processing program of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

  As described above, the present invention has been described using the first embodiment, but various changes or improvements can be added to the above embodiment.

(Second Embodiment)
Next, a second embodiment of the sound source information processing system to which the present invention is applied will be described. For easy understanding, the configuration of the sound source information processing system of this embodiment and the flow of its processing are the same as those of the first embodiment, and the detailed description thereof is omitted here with reference to the above description. . Therefore, differences from the first embodiment will be described below.

  The second embodiment is different from the first embodiment in that a history of past sound source information is used as the sound source state information described above. That is, in the second embodiment, the sound source information processing unit 105 shown in FIG. 1 includes a storage area in which past sound source information can be stored, and the past sound source information recorded in the storage area is used. A predetermined process is performed on the sound source information extracted in this manner. Hereinafter, the information holding means 107 described above is used as this storage area.

  When the sound source information processing system according to the present embodiment performs processing according to the flow shown in FIG. 3, the sound source information is extracted and the information is stored in the information holding means 107 for a certain number or a certain time. In the present embodiment, as an example, the estimated direction of the target sound source among various sound source information is stored. FIG. 7 is a characteristic diagram showing a history of information on the estimated direction of the target sound source stored in the information holding unit 107. In FIG. 7, a history of information on the sound source direction estimated for each time (star symbol in FIG. 7) is stored for a certain time period from a certain past time to the present time. Then, using the estimated history of the direction of the target sound source as the sound source status information, predetermined processing can be performed on the sound source information extracted at regular time intervals or at arbitrary timing.

  Then, in the sound source information processing system 204 according to the present embodiment, function approximation is performed on the stored information on the past sound source direction as shown by the solid line in FIG. 7, and the sound source is based on this function. It is determined whether or not a predetermined process needs to be performed on the information. Specifically, when the estimated direction (angle) of the target sound source is within a certain range from the estimated value by the approximation function, the estimated direction (angle) of the target sound source is used as it is. Used as the final output information.

  On the other hand, if the estimated direction (angle) of the target sound source is more than a certain value from the estimated value by the approximation function, the direction of the target sound source that will be the final output information uses the value of the approximation function. It is also possible to perform processing such as picking up and outputting the sound in the direction with the beam form, or picking up and outputting the sound in the direction estimated by the approximation function every time.

  As the function approximation, a moving average, a method that applies a linear function or a polynomial function locally, a model that assumes a model for movement of a sound source, and that is estimated from the model can be used. By using function approximation in this way, the estimated direction of the target sound source is not greatly disturbed by disturbances or the like. That is, when a unique estimation result is generated as compared with the past estimation result, the estimation result can be output without considering the past result. Therefore, by applying the predetermined processing to the sound source information to be output using the past sound source information as the above-described sound source status information as described above, more practical processing according to the surrounding situation is performed. The effect that it can be obtained.

(Third embodiment)
Next, a third embodiment of the present invention will be described. For easy understanding, the configuration of the sound source information processing system of this embodiment and the flow of its processing are the same as those of the first embodiment, and the detailed description thereof is omitted here with reference to the above description. . Therefore, differences from the first embodiment will be described below.

  The third embodiment differs from the first embodiment in that an information input device other than a microphone is used. That is, in the configuration shown in FIG. 1, one or a plurality of information input devices are further used like the microphone. In this embodiment, in the form shown in FIG. 2, it is assumed that each seat is provided with an interpersonal sensor for determining whether or not there is a user in each seat in the automobile. For example, as shown in FIG. 8, a pressure sensor 202b that senses that the user 202 is seated in the driver's seat 202a is used as a method for determining whether or not there is a user in each seat in the vehicle using such a personal sensor. Can be used. Further, it may be determined whether or not there is a user in each seat by photographing the inside of the car with a camera and using image recognition technology. Thus, the method for determining whether or not there is a user in each seat in the automobile is not particularly limited, and various methods can be used. In this way, information other than the voice input picked up by the microphone, that is, information on whether or not there is a user at each seat in the automobile can be obtained.

  Hereinafter, with reference to the flowchart shown in FIG. 9, processing for determining whether or not it is necessary to perform predetermined processing on the sound source information using the interpersonal sensor 202 b according to the flow shown in FIG. 3. explain.

  First, when a dialogue between the users 202 and 203 and the in-vehicle information system 205 is started (S901), information on the target sound source, particularly its estimated direction, is obtained as in the first embodiment (S902). Next, it is determined whether or not information indicating whether or not there is a user at each seat in the automobile is obtained from the interpersonal sensor 202b (S903). If the information is not obtained (No in S903), the sound source information processing system 204 assumes that there is no utterance from the user even if the estimated sound source information (estimated direction) is obtained in the above step 902. (S905). Alternatively, the order of step 902 and step 903 may be switched, and the sound source information of the target voice may be obtained after determining that there is information from the interpersonal sensor 202b.

  When information indicating whether or not there is a user at each seat in the automobile is obtained from the personal sensor 202b (Yes in S903), that is, information on the estimated target sound source (estimated direction) and information from the personal sensor 202b If both are obtained, the angle of the sound source direction allowed for each seat of the responding interpersonal sensor is compared with the estimated direction (S904), and the estimated direction of the target sound source has reacted. It is determined whether or not the seat corresponds to the seat of the personal sensor 202b (S906). As a result, if the estimated direction of the target sound source does not correspond to the seat of the reacted human sensor 202b (No in S906), it is determined that the estimated sound source was an estimation error or was a sound other than the user, As described above, sound source information is output assuming that there is no utterance from the user (S905).

  On the other hand, when the estimated direction of the target sound source corresponds to the seat of the responding interpersonal sensor 202b (Yes in S906), it is determined that the speech is from the user of the corresponding seat, and the beamform is applied in the corresponding direction. Sound collection and output the sound source information (S907).

  In the sound source information processing system 204 according to this embodiment as described above, by using input information from an information input device other than a microphone as described above, predetermined processing is performed on sound source information obtained by microphone array processing. Appropriate output information can be output. As a result, even when an error occurs in the microphone array processing result due to disturbance or the like, it is possible to prevent an abnormal output from adversely affecting the processing in the subsequent vehicle information system 205. Therefore, by performing predetermined processing on the sound source information to be output as described above, it is possible to obtain an effect that more practical processing can be performed according to the surrounding situation.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. For easy understanding, the configuration of the sound source information processing system of this embodiment and the flow of its processing are the same as those of the first embodiment, and the detailed description thereof is omitted here with reference to the above description. . Therefore, differences from the first embodiment will be described below.

  The fourth embodiment differs from the first embodiment in that the contents of either or both of the estimated target sound source and the sound source information of the sound source (non-target sound source) corresponding to noise that is not the target sound source are estimated. Then, based on the contents, predetermined processing is performed on the extracted sound source information. Here, for easy understanding, an example of estimation from the contents of both sound source information will be described with reference to FIGS. Here, FIG. 10 is a table summarizing procedure information for associating the determination condition when the content of the sound source information of the target sound source and the non-target sound source is the sound source status information and the processing content for the sound source information. This table is held in, for example, the information holding unit 107 described above, and is called as appropriate to perform predetermined processing. FIG. 11 is a flowchart showing a determination as to whether or not a predetermined process needs to be performed on the extracted sound source information and the flow of the process.

  It is assumed that sound source information of a target sound source and sound source information obtained from a beamform whose direction is a blind spot are obtained by microphone array processing according to the processing flow of FIG. Here, “the direction is the blind spot” means “the sound is collected with the directivity removed from the specific direction”. As a result, it is possible to pick up sound other than that specific direction. That is, the sound source information of the target sound and the sound source information of the non-target sound source corresponding to noise can be obtained.

  Then, the contents of the target speech and the non-target sound source (noise) are estimated from the sound source information of both. Here, since the target voice is the voice of the user in the car and is a human voice, the content of the collected sound source information is estimated by focusing on whether the collected voice is a human voice or not. To do. The estimation of the content of the sound source information can be realized by an existing technique, for example, using a zero crossing or a spectrum of a speech waveform. However, since such a determination method itself is not directly related to the present invention, a detailed explanation is given here. Omitted.

  On the other hand, with respect to the non-target sound source (noise), the state of generation of the non-target sound source (noise), that is, whether the non-target sound source (noise) is stationary or non-stationary is focused. Estimate the contents. The estimation of the content of the non-target sound source (noise) can be realized by existing techniques, for example, using a temporal change in volume, but such a determination method as to whether or not it is stationary noise itself is not directly related to the present invention. Details are omitted because they are not relevant.

  When the contents of the target speech and the non-target sound source (noise) are estimated as described above, predetermined processing as described below is performed on the extracted sound source information based on the contents. Is done.

  First, the content of the target sound source is estimated (S1101), and it is determined whether or not the estimated target sound source is a human utterance (S1102). If the estimated target sound source is a human utterance (Yes at S1102), then the contents of the non-target sound source are estimated, and it is determined whether or not the estimated non-target sound source is stationary noise (S1103). . Here, when the estimated non-target sound source is stationary noise (Yes in S1103), the situation is convenient for sound source information extraction, and there is little possibility of an estimation error. Therefore, in this case, the sound source information as estimated is used to pick up the sound only in the seat direction (S1104), and this is handled as the output of the sound source information processing system 204.

  On the other hand, as a result of determining whether or not the estimated non-target sound source is stationary noise, if it is determined that the estimated non-target sound source is not stationary noise (No in S1103), an estimation error occurs in sound source information extraction. This is likely to occur. For example, a situation may occur in which sudden noise is estimated as a person's utterance and a person's utterance is estimated as an unsteady sound. Therefore, in such a case, in order to prevent a malfunction in the subsequent stage, processing is performed in which the estimated direction of the original target sound source is corrected and the sound is collected while paying attention only to the driver's seat direction (S1105). Treated as output of system 204. By performing such processing, it is possible to effectively reduce the possibility that sudden noise is erroneously estimated as human speech and output from the sound source information processing system.

  Further, the content of the target sound source is estimated (S1101), and even when it is determined that the estimated target sound source is not a human utterance (No in S1102), the content of the non-target sound source is estimated and estimated. It is determined whether or not the non-target sound source is stationary noise (S1106). Here, when the estimated non-target sound source is stationary noise (Yes in S1106), it is clearly indicated that it is stationary noise and handled as an output of the sound source information processing system (S1107). As a result, in the latter stage system, it is possible to use a commonly used noise suppression method by estimating the content of the noise from the obtained stationary noise, and further improve the accuracy of processing performed by itself such as speech recognition and speech enhancement. be able to.

  On the other hand, as a result of determining whether or not the estimated non-target sound source is stationary noise, if it is determined that the estimated non-target sound source is not stationary noise (No in S1106), two situations can be considered. . That is, the first situation is a case where there is actually no utterance of the user and there is non-stationary noise. The second situation is when there is an error in sound source localization, noise is estimated as the target sound source, and the user's utterance is estimated as non-stationary noise. Therefore, in this case, the estimated content is checked (S1108). In other words, the estimated direction of the target sound source and the non-target sound source is switched, the first non-target sound source direction is regarded as the target sound source direction, and it is determined again whether or not it is a human utterance. It is determined whether or not the voice is uttered (S1109).

  If it is determined in this case that the non-target sound source is a human utterance (Yes in S1109), the sound from the non-target sound source direction is regarded as the utterance from the user, and this is handled as the output of the sound source information processing system 204. (S1110). On the other hand, if it is not estimated that the voice is a human voice in this determination (No in S1109), it is handled that there is no voice from the user (S1111).

  In the sound source information processing system 204 according to the present embodiment as described above, the content of the sound source is estimated as described above, and this is used as sound source status information, whereby predetermined processing is performed on the sound source information obtained by the microphone array processing. To output appropriate output information. Therefore, by performing predetermined processing on the sound source information to be output as described above, it is possible to obtain an effect that more practical processing can be performed according to the surrounding situation.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. For easy understanding, the configuration of the sound source information processing system of this embodiment and the flow of its processing are the same as those of the first embodiment, and the detailed description thereof is omitted here with reference to the above description. . Therefore, differences from the first embodiment will be described below.

  The fifth embodiment is different from the first embodiment in that the certainty factor of the estimated sound source information is used as the sound source state information described above. Further, in the present embodiment, the output from the sound source information processing system 204 is where the utterance of the user comes from in the car and its direction, or there is no utterance of the user (no useful information), Alternatively, it is assumed that the user's confirmation is requested via the in-vehicle information system 205, and roughly has three kinds of meanings. In this way, when the in-vehicle information system 205 uses the anthropomorphic agent to interact with the user, the anthropomorphic agent can appropriately process the uttered user.

  When the sound source information processing system 204 estimates that the user uttered in the car, the sound source direction is obtained. FIG. 12 shows the definition of the sound source direction in the vehicle in this case. Here, a case of an automobile provided with the right handle 215 will be described.

  When the direction of the imaginary line extending through the center in the width direction of the vehicle and extending rearward of the vehicle is 0 degree, the driver seat direction is the positive direction and the passenger seat direction is the negative direction. The range of the sound source direction from +15 degrees to +45 degrees corresponds to the driver seat 202a, and the range from −15 degrees to −45 degrees corresponds to the passenger seat 203a. The range of the sound source direction from −15 degrees to +15 degrees corresponds to the rear seat 220. The conversion from the sound source direction information to the corresponding seat information is stored in the information holding means 107 by using the sound source information processing unit 105 as the input of the sound source direction information estimated by the microphone array processing unit 104 by the sound source state information acquisition function. This can be done by referring to a table (not shown) of predetermined mapping information. The sound source information processing unit 105 can use the converted sound source status information by its processing function. The actual information here refers to the corresponding seat information from the sound source direction information, such as the range of the sound source direction from how many times corresponds to the driver's seat, and the range of the sound source direction corresponds to the passenger seat. Is procedural information that enables conversion of

  The real information can be rewritten from outside the sound source information processing system 204. By making the real information rewriteable from the outside of the sound source information processing system 204 as described above, it is possible to accurately determine the processing for the sound source information even when the surrounding environment changes. For example, when the sound source information processing system 204 that has been used in a right-hand drive car is brought to a left-hand drive car, the processing contents for the sound source information can be rewritten without rewriting the mapping information corresponding to the angle. , Processing suitable for the purpose can be performed. That is, the sound source information processing system 204 excellent in convenience when the use environment changes can be realized.

  For example, when the sound source information processing system 204 is used for a robot, it is assumed that the range of the sound source direction from the number of times to the number of times is the direction in which the robot's face is facing. By doing this, if the direction of the face is facing, it will respond reliably even if the certainty of the sound source estimation is not high, and if the certainty is not high, the certainty of the sound source estimation must be greater than or equal to a predetermined threshold By applying a process that ignores the sound in that direction, it is possible to make it stronger against disturbance. In addition, by simply rewriting the mapping information corresponding to the angle as described above, the concept of “the direction in which the robot is facing” can be changed as appropriate, and the degree of freedom of use of the robot can be increased. it can.

  It should be noted that the definition of the angle here is only an example for explaining the present embodiment, and it can be changed as appropriate depending on the form of the automobile, changing the positive / negative direction according to the left and right steering wheel. is there. That is, it is only necessary that some meaning can be given according to the sound source direction and it can be associated with the sound source information.

  In the present embodiment, the sound source direction is estimated and at the same time, the certainty of the estimation is estimated. The certainty factor is estimated together with the sound source direction in the microphone array processing unit 104. This certainty factor is the residual after convergence of the adaptive filter used for sound source localization, the error with the model used for estimation, the value correlated with the volume, the measure of human voiceness, the temporal change of the sound source direction and its Although it can be realized by existing technology as being set from features related to sound source information extraction such as variance, the reliability calculation method itself is not directly related to the present invention, so detailed description is omitted here. To do.

  The certainty factor in the present embodiment is obtained from these combinations, and the certainty factor assumes three discrete values of high, medium, and low. Moreover, it is good also as what discretized with the threshold value what was obtained with the function continuously. In this embodiment, the certainty factor is described as a discrete value, but the present invention can be applied without problems to those using continuous values.

  A process of performing a predetermined process on the extracted sound source information using the information on the sound source direction obtained in this way and the certainty of estimation of the sound source direction will be described with reference to FIG. First, the case where the certainty factor of the estimation of the sound source direction is high (high level) will be described. When the certainty of the estimation of the sound source direction is high and the estimated sound source direction is in the range of −15 to +15 degrees, the sound source seat is determined as the rear seat 220 based on the above-described real information, and the rear The sound source information estimated by the microphone array processing unit 104 is output together with the information of the sound source direction by determining that there is a utterance from the user of the seat 220.

  Further, when the certainty of the estimation of the sound source direction is high and the estimated sound source direction is outside the range of −15 to +15 degrees, the seat of the sound source is the driver seat 202a according to the above-described real information. Alternatively, it is determined as the passenger seat 203a, and it is determined that there is a voice from the user at the driver seat or the passenger seat, and the sound source information estimated by the microphone array processing unit 104 is output together with the information of the sound source direction (angle). In particular, even when the estimated sound source direction is a direction away from the seat such as +45 to +90 degrees, it is determined that the user has moved, and there is a voice from the user at the driver seat or the passenger seat. The sound source information determined and estimated by the microphone array processing unit 104 is output together with information on the sound source direction.

  Next, the case where the certainty factor of the estimation of the sound source direction is medium (medium level) will be described. When the certainty of the estimation of the sound source direction is medium and in the range of + 15 ° to + 45 ° or −15 ° to −45 °, the seat of the sound source is the driver's seat 202a and the assistant according to the above-described real information. The sound source information that is determined to be the seat 203a or the rear seat 220, is determined to have been uttered by the user from the driver seat 202a, the passenger seat 203a, or the rear seat 220 and is estimated by the microphone array processing unit 104 in the direction of the sound source. Output with information.

  When the certainty of the estimation of the sound source direction is medium and the estimated sound source direction is outside the range of +15 degrees to +45 degrees or −15 degrees to −45 degrees, the utterance from the user Whether or not there is a problem cannot be determined with certainty. Therefore, in this case, the sound source information processing system 204 does not output the estimated sound source direction as it is, but outputs output information for requesting confirmation of the sound source. For example, predetermined information stored in advance in the information holding unit 107 (information indicating “not determined”, “sound source reconfirmation”, etc.) is output. By performing such processing, for example, the in-vehicle information system 205 can call the user “speak again” in order to request the user to confirm the sound source. Thereby, it is possible to prevent the in-vehicle information system 205 from malfunctioning due to indefinite information from the sound source information processing system 204.

  Next, a case where the certainty factor of the sound source direction estimation is low (low level) will be described. When the certainty of estimation of the sound source direction is low and the estimated sound source direction is in the range of +15 degrees to +45 degrees or −15 degrees to −45 degrees, the estimated sound source is the voice of the user. Although it may be, it cannot be judged reliably. Therefore, in this case, the sound source information processing system 204 does not output the estimated sound source direction as it is, but outputs output information for requesting confirmation of the sound source. For example, predetermined information stored in advance in the information holding unit 107 (information indicating “not determined”, “sound source reconfirmation”, etc.) is output. By performing such processing, the in-vehicle information system 205 can request the user to confirm the sound source as described above. Thereby, it is possible to prevent the in-vehicle information system 205 from malfunctioning due to indefinite information from the sound source information processing system 204.

  In addition, when the certainty factor of the estimation of the sound source direction is low and the estimated sound source direction is out of the range of +15 degrees to +45 degrees or -15 degrees to -45 degrees, the certainty degree is low. In addition, it is difficult to consider the direction of the sound source as the utterance from the user. Therefore, in this case, the sound source information processing system 204 does not output the estimated sound source direction as it is, but outputs output information that there is no utterance from the user, assuming that there is no utterance from the user (no useful sound source). Output. For example, predetermined information (information indicating “no voice input”, “no output” or the like) stored in advance in the information holding unit 107 is output. By performing such a process, the in-vehicle information system 205 can prevent the in-vehicle information system 205 from malfunctioning due to indefinite information from the sound source information processing system 204.

  In the sound source information processing system 204 according to the present embodiment as described above, the above-described extraction is performed using the estimated sound source angle and the certainty of estimation as the above-described sound source state information with respect to the utterance from the user. A predetermined process can be performed on the sound source information. The anthropomorphic agent using the sound source information processing system 204 can perform more appropriate processing by performing processing based on the output from the sound source information processing system 204. That is, when interacting with a user using an anthropomorphic agent in the in-vehicle information system 205, the anthropomorphic agent can make a more reliable response to the uttered user according to the surrounding situation. The effect is obtained.

  As described above, the sound source information processing apparatus according to the present invention is useful when collecting only a target sound in an environment with ambient noise, and is particularly suitable for applications such as a car navigation system and a video phone. .

It is a block diagram which shows one structural example of the sound source information processing system to which this invention is applied. It is a figure explaining the example which applied the sound source information processing system to which this invention is applied in the motor vehicle. It is a flowchart explaining the flow of a process in a sound source information processing system. It is a figure which shows the table which put together the procedure information which associates the judgment conditions at the time of making noise intensity into sound source status information, and the processing content with respect to sound source information. It is a flowchart which shows the judgment whether it is necessary to perform a predetermined | prescribed process with respect to the extracted sound source information, and the flow of the process. 2 is a diagram illustrating a hardware configuration of a sound source information processing system according to Embodiment 1. FIG. It is a characteristic view which shows the log | history of the information of the direction of the estimated target sound source memorize | stored in the information holding means. It is a figure which shows the state by which the pressure sensor which detects that the user sat down was arrange | positioned at the driver's seat. It is a flowchart explaining the flow of a process which judges whether it is necessary to perform a predetermined process with respect to sound source information using a personal sensor. It is a figure which shows the table which put together the procedure information which associates the judgment condition when the content of the sound source information of a target sound source and a non-target sound source is sound source status information, and the processing content for the sound source information. It is a flowchart which shows the judgment whether it is necessary to perform a predetermined | prescribed process with respect to the extracted sound source information, and the flow of the process. It is a figure showing the regulation of the sound source direction in the car. It is a figure which shows the table which put together the procedure information which associates the judgment condition at the time of making a sound source direction and the reliability of sound source information estimation into sound source status information, and the processing content with respect to sound source information.

Explanation of symbols

101-1 to 101-N microphone 102 input processing unit 103 signal processing unit 104 microphone array processing unit 105 sound source information processing unit 106 output 107 information holding unit 201 automobile 202 driver seat user 202a driver seat 203 passenger seat user 203a passenger seat 204 Sound source information processing system 205 In-vehicle information system 206 Car operation system

Claims (12)

A plurality of sound collecting means for collecting sound signals of a plurality of sound sources;
Sound source estimation means for estimating the direction of at least one sound source based on the sound signals collected by the plurality of sound collection means;
Sound source information extracting means for extracting sound source information related to a target sound source among the sound sources in the estimated direction;
Sound source status information acquisition means for acquiring sound source status information relating to at least one of one or a plurality of sound sources and surrounding conditions for selecting the processing content to be applied to the extracted sound source information;
Processing means for performing predetermined processing on the extracted sound source information based on the sound source status information;
A sound source information processing apparatus comprising: Holding means for holding processing content to be performed on the extracted sound source information based on the sound source status information;
The sound source information processing apparatus according to claim 1, wherein the processing unit calls a predetermined processing procedure from the holding unit to execute processing. The sound source status information acquisition unit holds a history of sound source information of the sound source estimated by the sound source estimation unit,
The sound source information processing apparatus according to claim 1, wherein the processing unit uses a history of the sound source information as the sound source state information. The sound source information processing apparatus according to claim 1, wherein the processing unit performs a predetermined process on the extracted sound source information at regular time intervals or at an arbitrary timing. The sound source information processing apparatus according to claim 1, wherein the sound source status information acquisition unit acquires the sound source status information from an information input unit other than the sound collection unit. The sound source status information acquisition means estimates the content of the sound source from the sound signal of the sound source estimated by the sound source estimation means,
The sound source information processing apparatus according to claim 1, wherein the processing unit uses the content of the sound source as the sound source status information. The sound source status information acquisition means estimates the content of the sound source other than the estimated sound source direction from the sound signal collected by removing the directivity with respect to the sound source direction of the sound source estimated by the sound source estimation means. ,
The sound source information processing apparatus according to claim 1, wherein the processing unit uses sound source content other than the estimated sound source direction as the sound source state information. The sound source status information acquisition means converts the sound source direction information estimated by the sound source estimation means into predetermined sound source status information as an input,
The sound source information processing apparatus according to claim 1, wherein the processing unit uses the converted sound source state information. The sound source estimation means estimates a sound source and estimates a certainty of estimation;
The sound source status information acquisition means acquires the certainty factor of the estimation as the sound source status information,
The sound source information processing apparatus according to claim 1, wherein the processing unit uses the certainty factor of the estimation as the sound source state information. The sound source information processing apparatus according to claim 1, wherein the processing unit outputs predetermined information stored in advance. A sound collection process for collecting audio signals of a plurality of sound sources;
A sound source estimating step of estimating a direction of at least one sound source based on the sound signal collected in the sound collecting step;
A sound source information extracting step for extracting sound source information related to an output target from among the sound sources in the estimated direction;
A sound source status information acquisition step for acquiring sound source status information related to at least one of one or a plurality of sound sources and surrounding conditions for selecting the processing content to be performed on the extracted sound source information;
A processing step of performing a predetermined process on the extracted sound source information based on the sound source status information;
A sound source information processing method comprising: A sound collection process for collecting audio signals of a plurality of sound sources;
A sound source estimating step of estimating a direction of at least one sound source based on the sound signal collected in the sound collecting step;
A sound source information extracting step for extracting sound source information related to an output target from among the sound sources in the estimated direction;
A sound source status information acquisition step for acquiring sound source status information related to at least one of one or a plurality of sound sources and surrounding conditions for selecting the processing content to be performed on the extracted sound source information;
A processing step of performing a predetermined process on the extracted sound source information based on the sound source status information;
A sound source information processing program for causing a computer to execute.

Images