JP2007187748A - Sound selective processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate and remove displeasure sound causing irritation out of mixed sounds generated asynchronously in a real environment. <P>SOLUTION: A sound selective processing device includes a sound acquisition part, a sound structure characteristic extraction part, a sound separation part, a displeasure detection part, a candidate sound selection determination part, a candidate sound presentation specifying part, a processing sound structure characteristic updating part, and a sound processing part. When detecting displeasure, displeasure sound being an irritating cause is estimated, and the number of presenting sounds is reduced as much as possible to determine an object sound after confirming it. By the composition, the displeasure sound being the irritating cause out of the mixed sounds is estimated and the displeasure is reduced quickly by processing to specify the sound presented by a user out of the candidate sound. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention selects a specific sound from a mixed sound composed of various sounds generated asynchronously around the user, and processes the sound while leaving a sound that is important for the user to be heard. The present invention relates to a sound selection processing device that makes it difficult to hear unpleasant sounds that cause frustration.

  We live in various sounds. Many of the miscellaneous sounds are of a nature that cannot be silenced by stopping the sound by itself, unless the sound is generated by the source or acquaintance or friend. For example, a stranger speaks loudly, or sounds of public transportation. People can feel frustrated and uncomfortable with such uncontrollable sounds.

  FIG. 18 is a diagram depicting a scene simulating sound generated in a train. Inside the train, various sounds such as engine (motor) sound, crossing sound, traveling sound such as train crossing track, passenger conversation, mobile phone operation sound, conductor announcement, etc. Is occurring asynchronously.

  Using this example scene, a situation example will be described in which the sound that the person wants to hear and the sound that he / she does not want to hear differ depending on the person's position.

  Person A was reading books intensively on the train. People B and C, who had newly boarded at the station where they stopped, stood in front of person A and started talking loudly. As a result, person A cannot concentrate on the book because of the loudness of his voice, and begins to get frustrated. Also, the person C, who is mainly the listener, is unlikely to hear the voice of the person B on the train, unlike the conversation in the station until the boarding, and has difficulty in grasping the flow of the conversation. Yes. On the other hand, the person D who gets on this route for the first time concentrates on the announcement in the car to hear the name of the next stop. However, the person D is also in trouble because the train sound and the conversation sound between the person B and the person C cannot hear the announcement well and cannot judge whether the next station is the station where he / she gets off. In addition, it is in a state of restlessness because it is not possible to judge when the next announcement will start when approaching the station. Such a person D is also writing a mail by operating a mobile phone, but the person B feels anxious about the operation sound.

  From this scene, it can be found that there are various unpleasant sounds that cause human frustration.

  For example, like a person B, in a train, even if the conversational sound spoken with a friend is loud, it is surprising that it does not bother itself. This is because a friend's voice is the target of the sound he / she wants to hear, and if the sound becomes difficult to hear due to the sound of traveling on the train, an excessively loud voice may be produced. On the other hand, for people A and D who are in different positions, the sound of the train itself is not so much anxious, and the conversation of the loud voice of the person B may feel noisy.

  FIG. 19A is a diagram showing each separated sound extracted from the mixed sound, and FIG. 19B is a diagram showing each sound such as the announcement, train sound, mobile phone operation sound, and person B's voice in FIG. It is the figure which summarized whether each person has the subjectivity of the sound which he wants to hear, the sound which is neither, and the sound which he does not want to hear by each subjectivity. As shown in the figure, people A and D share the same sound that they do not want to hear, that is, the sound that makes them frustrated, but person A does not have the sound he wants to hear, and person D wants to hear the announcement There is a difference. From this, it can be seen that the strategy of muting all sounds is not enough.

  Person B and person C have the same sound that they want to hear, but they do not want to hear the voice of person B or the mobile phone, but person C does not want to hear any sound other than person B's voice. State. From this, it can be seen that it is not enough to just eliminate the sounds you do not want to hear.

  Therefore, even if the same sound environment is shared at the same time and place, depending on the place and the person's context, it exists in the sound environment of the place, such as the sound that you want to hear, the sound that you do not want to hear, or the sound that is neither It can be said that the subjective sensation for each sound is a factor of unpleasant sound.

  In addition, the complexity of the human auditory mechanism is thought to have an impact on the irritating sensation of sound. A person has a function of selecting a sound called a so-called cocktail party effect, in which a person can listen to a sound by paying attention to the sound to be heard from a mixed sound in which various sounds are produced. In addition, when reading a book on the train like person A, the train called Gatangoton, which was supposed to be heard earlier, couldn't hear the sound that straddled the seam of the track or the motor sound. This is a phenomenon in which a steady and rarely changing sound is selectively made inaudible when it concentrates on other things. Therefore, even if you decide that the sound is frustrating once, you will not know if you will be frustrated at the same time.

  In order to reduce the discomfort caused by such uncontrollable sounds, for example, try to shut out the surrounding sounds with earplugs or listen to music with headphones. It is also common to create a sound environment.

  In addition, there is a headphone that eliminates ambient sounds that are disturbing sounds when listening to music and the like. For example, Patent Literature 1 discloses an active noise reduction device that reduces noise by causing signals having opposite phases and equal amplitudes to interfere with unpleasant noise generated in the passenger compartment as the engine rotates and the vehicle travels. It is disclosed. In particular, according to the present invention, the generation of abnormal noise due to the fact that normal noise reduction operation is not performed due to an abnormality in the output signal from the microphone that detects the noise reduction error is prevented.

  However, with such a countermeasure, it is almost impossible to hear surrounding sounds other than music output from headphones. Therefore, although discomfort received from ambient sounds can be reduced, there is a possibility of being missed when a sound desired to be heard is generated in the surroundings. Therefore, this method is effective only when the sound that you want to hear or the sound that you want to leave is determined, and when the sound you want to hear or the unpleasant sound changes one after another as in the situation of FIG. , There was a problem that can not be dealt with.

  Therefore, in order to realize the desire to eliminate only the unpleasant sound that causes the frustration as described above, the mixed sound is separated into sounds for each sound source, and a desired sound is selected and processed. Technology is needed. FIG. 19A is a schematic diagram showing that the mixed sound in the real environment is decomposed into sounds for each of the three sound sources from ch (channel) 1 to ch 3 using sound separation technology. A technique for identifying which of these separated sounds is an unpleasant sound that causes frustration is required.

  The conventional sound separation technology includes a plurality of directional microphones, array microphones, ICA, beam forming, and other techniques for separating sounds based on the estimated sound source direction, as a frequency analysis method, There is also a separation method using a sound analysis technique that can extract the target sound feature by reducing the overlapping of the sound in the time frequency domain by simultaneously and in detail detailing the resolution in the time frequency domain. Also, an auditory scene analysis method based on the human auditory mechanism proposed by Bergman has been proposed.

On the other hand, in the prior art for selecting a sound, a method of selecting a sound in which the position of the sound source generated in the monitor and the line-of-sight direction on the monitor match (for example, Patent Document 2), the importance of the sound source is set. There is a method for selecting a sound based on the method (for example, Patent Document 3), and a method for performing a voice conversation by changing the content of speech synthesis to be transmitted to the user according to the current location and situation (for example, Patent Document 4). Are known. Note that any of the methods described in Patent Documents 2 to 4 includes a technique for separating sound from an unfixed sound source because the input sound source is fixed depending on the sound source direction, the current location, and the importance of the sound source. Not.
JP 2004-352070 A JP-A-9-090963 Japanese Patent Laid-Open No. 9-275533 JP 11-351901 A

  20 (a) to 20 (d) are diagrams illustrating main factors that cause a situation in which the user is frustrated by sound. As shown in FIG. 20, the following four typical factors are conceivable as factors that create a situation in which the sound is frustrated. In this figure, when the sounds generated in the surrounding environment are classified according to the user's subjectivity: a) the sound that they want to hear, b) the sound that they do not want to hear, and c) the sound that they do not want to hear. It shows that subjectivity changes due to the relationship, and as a result, a sound that does not feel uncomfortable by itself changes to d) a sound that makes you feel uncomfortable. Here, each factor is demonstrated using FIG. 20 (a)-(d).

  In FIG. 20A, as factor 1, a case where the user is frustrated because a sound that he does not want to hear is sounding is shown. Here, since c) the sound that the user does not want to hear is sounding alone, d) discomfort occurs, and then, a) the sound that the user wants to hear and b) the sound that is neither is generated and overlapped. d) The discomfort is consistent with the section where the sound that you do not want to hear is sounding.

  In FIG. 20 (b), as factor 2, there is no frustration when listening alone, but there is a frustration overlying the sound you want to hear. Here, by itself, b) a sound that was neither sounded, but a) a desired sound was generated, so that it became difficult to hear the desired sound. As described above, since the sound overlapping section corresponds to a temporary sound c) that is not desired to be heard (dotted line), it indicates that discomfort has occurred.

  FIG. 20C shows a case where a plurality of sounds are generated at the same time as factor 3 and the user cannot perform sound selection by themselves, which is frustrating. In this example, a) three sounds to be heard occur almost simultaneously, and b) overlaps with none of the sounds, and it is not possible to determine which one to listen to. It indicates that discomfort has occurred because it corresponds to a sound that you do not want to hear.

  FIG. 20D shows a case where the information to be heard has already been heard as factor 4, but the sound continues to be irritated, and is frustrating. In this example, a) the sound that you want to hear continues to sound after the sound that b) is neither, so the latter half of the part you want to hear is equivalent to the sound you do not want to hear temporarily. It shows that you feel.

  When the above factors are analyzed, the factor 1 simply means that there is a sound that the individual user does not always want to hear, whereas the factors 2 to 4 indicate the separated sound separated from the mixed sound. It can be seen that due to the overlap with other separated sounds and the relationship with other separated sounds, the sound that is desired to be heard or the sound that was neither is temporarily changed to a sound that is not desired to be heard, resulting in unpleasant feeling.

  That is, using the conventional method based on the importance of sound, the method of determining the removal target in advance and learning the sound characteristics, and the method of selecting the sound according to the place and time, factor 1 always wants the user to hear As long as the separated sound can be learned, it is only necessary to detect and separate whether the sound that is not desired to be heard from the mixed sound is included in the separated sound. However, in Factors 2 to 4, there is a problem that even if a sound that the user does not want to hear is learned in advance, the subjectivity changes depending on the relationship of the separated sound, so that it cannot always be identified from the importance of the current location and the sound source.

  In addition, in the method of selecting the sound at the sound source position that matches the line-of-sight direction, the sound source position may not be accurately obtained in the first place when the sound is generated from the same direction or because of the diffuse sound field. In such a case, there is a problem that the frustrating sound itself cannot be specified.

  Therefore, the present invention solves the above-described conventional problems, and in order to quickly reduce discomfort due to sound, unpleasant sound that causes frustration from among various sounds that are generated asynchronously in the surroundings A sound selection processing device that makes it difficult to hear unpleasant sounds that cause annoyance while preserving important sounds to be identified while processing the sounds by identifying the sounds by estimating them and processing the sounds The purpose is to provide.

  In order to solve the conventional problem, detection of discomfort to the user's sound is performed by pressing a button when the user is frustrated with the sound, and the cause of frustration is due to something other than sound. There is no need to determine whether or not.

  With this configuration, it is possible to reduce the discomfort caused by the sound quickly identified from the candidate sounds as the undesired sounds that cause irritation from the mixed sounds.

  In addition to the above-described configuration, the sound selection processing device of the present invention having another configuration having the processing reference input unit and the search target sound structure feature DB performs sound separation based on the sound feature structure of the search target sound. For the specified unpleasant sound, it is possible to listen to the sound processed freely by inputting the processing conditions.

  With this configuration, when a sound that is determined according to time and place is generated from the mixed sound, the sound structure characteristics are extracted, so that other sounds that are more likely to fluctuate and feel uncomfortable can be obtained. It can be extracted with high accuracy. At this time, by determining whether or not the sound environment is the same as that represented by other separated sound configurations, it is possible to determine continuously or continuously from the sound structure when learned.

  Furthermore, in addition to the above configuration, information on unpleasant sounds is transmitted to and received from other sound selection processing devices.

  With this configuration, it is possible to more accurately remove unpleasant sounds by selecting and processing the sound based on information on the estimated unpleasant sounds of not only the user but also the other party.

  Note that the present invention can be realized not only as such a sound selection processing apparatus, but also as a sound selection processing method including steps characteristic of the sound selection processing apparatus. It can also be realized as a program for causing a computer to execute steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  According to the sound selection processing device of the present invention, it is possible to quickly identify an unpleasant sound that causes irritation from the mixed sound and process the sound so that it is difficult to hear the sound causing the unpleasant feeling. Discomfort and stress can be reduced, and self-management can be facilitated.

  In addition, by using a plurality of sound selection processing devices of the present invention and communicating information related to unpleasant sounds, it is possible to share the sound structure characteristics for canceling unpleasant sounds in common between the communications, Discomfort can be quickly suppressed.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a sound selection processing apparatus 100 according to the present embodiment. The sound selection processing device 100 is a sound selection processing device that selectively removes a sound that the user feels uncomfortable from the mixed sound generated around the user, and includes a sound acquisition unit 101, a sound structure feature extraction unit 102, A sound separation unit 103, a sound processing unit 104, a candidate sound selection determination unit 105, a candidate sound presentation specifying unit 106, a processed sound structure feature update unit 107, a processed sound structure feature DB (database) 108, and a discomfort detection unit 131 are provided. The candidate sound selection determination unit 105 includes a sound structure feature comparison unit 133 and a masker separation sound estimation unit 134 therein.

  When the discomfort detection unit 131 detects a user's discomfort, the sound selection processing device 100 estimates a candidate for an unpleasant sound that causes frustration, and presents the candidate with a reduced number of candidates as much as possible. The user confirms the presented target sound candidates and then selects and determines a sound to be processed from the candidates.

  As shown in FIG. 1, the separated sound processing configuration for separating and processing the mixed sound into sounds for each sound source and the separated sound that is the candidate for processing are presented to the user, and the separated sound is selected when the user's selection is accepted. Since it can be roughly divided into two parts according to the configuration, in the flowchart, the processing flow corresponding to each will be described with reference to FIG. 2, FIG. 3, FIG. 4, FIG.

First, the flow of processing at the time of separated sound processing will be described.
The sound acquisition unit 101 acquires sound by performing A / D conversion on the mixed sound generated around the user, which is input through the microphone. A plurality of microphones may be used as the sound acquisition unit 101 to acquire the processed sound based on the direction, or the acquired data may be acquired from the sound once recorded on the hard disk. Moreover, you may catch as a sound of receiving parts, such as a mobile telephone.

  The sound structure feature extraction unit 102 captures sound features using Fourier transform in order to express time-frequency information as sound structure features. Instead of FFT (Fast Fourier Transform), DCT (Discrete Cosine Transform) or the like may be used.

  The sound separation unit 103 corresponds to a sound separation unit that separates the mixed sound into sounds for each sound source. The sound separation unit 103 divides the mixed sound into a stream of separated sounds according to sound structure characteristics as shown in the related art. Disassembled into The sound separation method used in this embodiment is a method for extracting and decomposing a plurality of separated sounds from a mixed sound by evaluating the continuity of the sound characteristics from the mixed sound composed of the sounds of a plurality of sound sources. (For example, Nakatani, T. Computational Audit Scene Analysis based on Residue-Driving Architecture and Its Application to Mixed Spec Recognition. In this method, the start and end of a sound are grouped together, regarded as a peak group in which the rise and fall of the peak are synchronized, and a transition of the sound is tracked to extract one sound stream.

  The sound processing unit 104 corresponds to sound processing means for processing the specified separated sound to reconstruct the mixed sound. In the sound processing unit 104, the processed sound structure is processed in accordance with a processing flow (described later) at the time of sound selection. Processing a sound having a sound structure characteristic of a sound that is not desired to be stored, which is stored in the feature DB 108, and generating a reverse phase signal or the like of the identified separated sound based on the method described in Patent Document 1 or the like To prevent the specified separated sound from being heard.

  FIG. 2A is a flowchart of the processing during the separated sound processing in the first embodiment. The process at the time of separation sound processing is demonstrated using this figure.

  In S <b> 701, the sound selection processing device 100 sets a processing target section, and prepares sound structure characteristics for the processing target section of the mixed sound. This processing target section refers to the time from when the sound acquisition unit 101 acquires mixed sound, the sound separation unit 103 separates the mixed sound, and the sound processing unit 104 reconstructs the mixed sound. For example, the sound selection processing device 100 sets a processing target section for 3 seconds, and extracts the sound structure characteristics of the mixed sound in the 3 seconds. The length of the section to be processed may be determined in advance when the sound selection processing device 100 is shipped. Next, in S <b> 702, the sound separation unit 103 extracts the separated sound. In this embodiment, the method shown in FIG. 2B described later is used. After the separation sound extraction processing is performed, it is determined in step S703 whether a separable sound stream is obtained. For example, the mixed sound eventually becomes only one stream, that is, only the background noise can be obtained. In such a case, the mixed sound is output as it is in S704, and if there is a separated sound, the processing target sound already identified in S705 is read from the processed sound structure feature DB 108 so that it is difficult to hear. For example, the sound is processed by giving the opposite phase of the sound to be processed.

  FIG. 2B is a flowchart of the separated sound extraction peak search in the first embodiment. In S711 corresponding to the sound separation unit 103, a representative peak is extracted from the sound structure characteristics of the mixed sound, and the peak is traced in the time-frequency direction. In step S712, peaks whose rising and falling peaks fluctuate synchronously are grouped and regarded as one sound stream candidate. In S713, the number of peaks included in the sound stream candidate is verified, and if the number of peaks equal to or greater than the threshold is included, it is recognized as one sound. In S714, the extraction target sound is converted from a spectrum waveform to a time waveform for each group, and in S715, a difference between the mixed sound and the extraction target sound is obtained. As a result of S712, when a plurality of groups are generated, S713 to S715 are repeated and extracted as a separated sound for each group.

  When the separated sound is processed in S705, it is determined whether there is a separated sound that is stored in the processed sound structure feature DB 108 and that is identified as not to be heard, and then the identified separated sound exists. In that case, the specified separated sound may be processed so as to be subtracted from the mixed sound.

  FIG. 2C is a flowchart of separated sound processing for determining the presence or absence of a specific sound in the first embodiment. Here, the specific sound refers to a sound specified in advance when selecting a separated sound because it is not desired to hear. In S716, the sound structure feature amount of the input separated sound corresponding to the sound structure feature amount of the specific sound specified as not desired to be stored stored in the processed sound structure feature DB 108 is obtained. In S717, the vector between the feature amounts is calculated. The distance is obtained by, for example, the Euclidean distance. In S718, when the distance is within the predetermined determination value, it can be determined that the input separated sound is the same sound as the specific sound. If it is determined that the sound is the same, in step S719, the input separated sound is converted into a waveform, and the input separated sound is subtracted from the S720 mixed sound. By performing this operation for each input separated sound with respect to the specific sound, it is possible to reduce the specified undesired sound after confirming the presence or absence of the specific sound.

  Note that a normalized direction cosine may be used as the distance measure. Further, in consideration of the predetermined time direction, the predetermined distance value may be set so as to fluctuate with time in a range that considers tracking.

  With this configuration, even if a separated sound that has been identified as a sound that you do not want to hear temporarily stops, the presence or absence of the separated sound that has been identified as a sound that you do not want to hear has an effect. The sound can be processed so that there is no.

Next, a configuration related to separation sound selection will be described.
The discomfort detection unit 131 corresponds to discomfort detection means for detecting that the user is in an uncomfortable state. The discomfort detection unit 131 detects discomfort by detecting pressing of the discomfort button by the user.

  When the candidate sound selection determination unit 105 detects that the user is in an uncomfortable state by the discomfort detection means, the candidate sound selection determination unit 105 evaluates the relationship between each separated sound that is a separated sound, and based on the evaluation result, This corresponds to candidate sound selection determining means for estimating the separated sound of the candidate for processing. The candidate sound selection determining unit 105 estimates a candidate for an unpleasant sound, that is, an undesired sound, by evaluating the relationship between the separated sounds recorded for a predetermined time. The configuration of the present embodiment (FIG. 1) includes a sound structure feature comparison unit 133 and a masker separated sound estimation unit 134. The sound structure feature comparison unit 133 evaluates the relationship between the separated sounds by comparing the signals formed from the separated sounds. The masker separated sound estimation unit 134 estimates the separated sound of the candidate to be processed by estimating a dominant separated sound or a dominant frequency band among the separated sounds based on the evaluation result. To do. The masker of the masker separated sound estimation unit 134 refers to a dominant sound or band among the separated sounds to be compared.

  The processed sound presentation specifying unit 106 corresponds to a candidate sound presentation specifying unit that presents the estimated separation sound of the candidate to be processed to the user, accepts the selection, and specifies the selected separated sound. The processed sound presentation specifying unit 106 presents the candidate sound determined by the candidate sound selection determination unit 105 to the user, and makes the user press a confirmation button to specify the candidate sound presentation.

  The processed sound structure feature update unit 107 stores or updates the feature of the unpleasant sound that causes the frustration specified by the processed sound presentation specifying unit in the processed sound structure feature DB 108. The sound feature here refers to a value obtained by characterizing the peak frequency and time waveform grouped in S714. In the case where the processed sound structure feature DB 108 has the stored separated sound structure feature, processing is performed so that the sound is extinguished by creating a reverse phase waveform.

Next, the flow of processing when selecting a separated sound will be described.
FIG. 3 is a flowchart of a process when selecting a separated sound in the first embodiment. In S601 corresponding to the discomfort detection unit 131, when the user detects a discomfort using a button press or the like, the separated sound received from the sound separation unit 103 in S602 corresponding to the candidate sound selection determination unit 105 The candidate of the unpleasant sound which becomes a cause of irritation is estimated by evaluating the relationship between separated sounds. In S603 corresponding to the processed sound presentation specifying unit 106, the candidate sound is presented, and the candidate sound is presented and specified by causing the user to press a confirmation button or the like. In step S604 corresponding to the processed sound structure feature update unit 107, the specified processed sound structure feature is stored in the processed sound structure feature DB. The stored processed sound structure feature is used at the time of separated sound processing.

  If there is one candidate sound determined in S602, the sound structure feature may be stored in S604 without confirmation in S603.

From here, the detail of the process in each step of FIG. 3 is demonstrated.
FIG. 4A is a flowchart showing the operation of the discomfort detecting unit 131 in the first embodiment. Here, it is a simple configuration for detecting discomfort when it is detected in S301 that the user presses the discomfort button. The method of detecting user discomfort by detecting the user pressing the discomfort button is simple, but in order to actually capture the user's own subjectivity, the user's own subjectivity is reflected. Therefore, it can be said to be a reliable means. When a discomfort button is pressed while a plurality of sounds are being played, it can also be used to select an unpleasant sound only for sounds up to several seconds before that.

  FIG. 4B is a schematic diagram for explaining that the separated sound is selected as the candidate sound according to the time from the pressing timing of the discomfort detection button to the start rising time of the separated sound. In the figure, the downward arrow of the discomfort detection indicates the timing when the discomfort detection button is pressed.

  In this example, the separated sound A is regarded as having a weak influence on the frustration because the rise of the sound is more than a predetermined time away from the detection of the unpleasant feeling, and the unpleasant sound candidate causing the frustration is the separated sound. B and separated sound C can be limited.

  Note that this predetermined time may vary from person to person, so it can be variably determined, for example, based on the difference between the rise time of the specified unpleasant sound and the time when the button is pressed. It may be settable.

  In addition to the predetermined time determined in this way, a reference value is set for the loudness level that the user feels noisy, and when the separated sound is larger than this reference value, it is separated from the time of detection of discomfort. The candidate sounds may be narrowed down by setting the time until the rise, which is the start time of the sound, to be short. This is because, for example, when an extremely loud sound is generated that is surprising or when the background noise itself is at a level where it feels noisy, it can be estimated that the sound tends to be more sensitive and uncomfortable. That is, when an extremely loud sound that is surprising is generated, the user is likely to feel uncomfortable with the sound. Therefore, an extremely loud sound generated immediately before the time of discomfort detection is an unpleasant sound without searching for other separated sound that seems to be causing discomfort going back from the time of discomfort detection. This is because the certainty is high.

  With such a configuration, the candidate sounds can be narrowed down with high sensitivity to loud sounds, so that sounds that cause discomfort can be identified more quickly.

  FIG. 5A is a flowchart of processing of the candidate sound selection determination unit 105 in the first embodiment. Here, by evaluating the relationship between separated sounds, it is estimated which separated sounds are unpleasant sounds, that is, sounds that one does not want to hear.

  First, in S401, it is determined whether or not the number of separated sounds when detecting discomfort is one. If the number of separated sounds is one, the separated sound is identified as an unpleasant sound that causes annoyance, and the process ends. If there is one or more separated sounds, if a change is necessary in S402, a predetermined time used for selection of the separated sounds is designated in S403. In S403, as shown in FIG. By determining whether or not the interval between the discomfort detection time and the rising time of each separated sound is within a predetermined time (for example, 10 seconds), it is determined whether or not the separated sound is a candidate for selection as a processing target. The predetermined time indicating the interval between the uncomfortable feeling detection time and the rising time of each separated sound also varies depending on the time required for the user to press the button after hearing the unpleasant sound. Accordingly, for example, the user may be able to set the predetermined time by manually inputting the number of seconds indicating the predetermined time. As a result of this determination, if there is only one candidate separated sound in S404, the separated sound is specified as an unpleasant sound. At this stage, in the case of factor 1 and factor 4 in FIG.

  If there are a plurality of separated sounds, in S405, the candidate sound calculated as dominant in the dominant separated sound extraction routine is determined. In S406, it is determined whether or not a dominant candidate sound has been found. If it is determined that the sound has been found, it is determined in S407 that the sound is likely not to be heard and the sound is not desired to be heard. The order is changed so that it is presented preferentially to the user. Conversely, if no dominant candidate sound is determined, that is, each sound is dominant in a certain band with respect to each sound, it is determined that this corresponds to factor 3 in FIG. . In this case, in S407, it is necessary to present to the user one by one in order, and to give instructions on which to leave and which to erase. This stage corresponds to a case where a plurality of separated sounds are overlapped by any one of factors 1, 2, and 4.

  FIG. 5B is a flowchart showing a processing procedure in the dominant separated sound extraction of S405 in FIG. 5A, and corresponds to the function of the masker separated sound estimation unit 134. Here, the dominant sound is determined by evaluating the relationship between the separated sounds.

  First, in S502 to be compared, a signal having the maximum power is selected for each frequency band of the separated sound, and in S503, a band representative signal (a signal having the maximum power in the band) included in a pair of separated sounds. By comparing them with each other, and in S504, further, a transition in the time direction is observed, and a masker separation sound is determined in S505.

  Here, FIG. 6 is a diagram for explaining the dominant separated sound determination process of S405 in the first embodiment, showing this flow as a specific example.

  Here, the process flow for determining the dominant band or sound is that the mixed sound is composed of the following three types of sounds (A pure sound, B, sound, C, background noise), which are separated sounds. Will be described using an example extracted as Here, in order to overlap and compare waveforms as will be described later, each is represented by a solid line, a broken line, and a one-dot chain line. Further, in this figure, for simplicity, each waveform is represented by a line spectrum having the horizontal axis as the frequency. For example, a pure tone waveform is represented by a frequency spectrum of a single frequency. A voice waveform is represented by a collection of frequency spectra having peaks at the fundamental frequency and its harmonic components. Background noise is represented by a collection of frequency spectra having a substantially uniform size regardless of frequency.

  First, the representative value for each predetermined band of the sound structure signal obtained by dividing the frequency axis F into bands having a predetermined width from each separated sound waveform of FIG. B) is selected as the band representative signal, and b) the band representative signal is obtained as c) the sound structure comparison signal. That is, here, b) and c) have the same signal waveform, but have different names in order to simplify the comparison with FIG. Here, the horizontal axis of FIG. 6a) indicates the frequency, and the vertical axis indicates the spectral power of the corresponding frequency. 6b) to d) both have the same scale.

  In this schematic example, the predetermined band is assumed to have a linear bandwidth of five identical frequency bandwidths. However, a critical band having a non-linear bandwidth described in a Bark scale or the like, which is known as knowledge about the human auditory mechanism, may be used as the predetermined band. The method of selecting the representative value of the band may be a frequency component having the largest power in the band, or when the band is a critical band, the power of the center frequency may be used.

  Next, in the evaluation of the dominant band in FIG. 6d, it is determined whether or not the separated sound to be compared becomes dominant depending on which amplitude is larger for each band corresponding to the separated sound. Here, when the amplitude difference is within 3 dB, for example, it is determined that there is no dominant band. As a result, a band that cannot be determined as dominant is indicated by a “-” symbol in the figure. Here, when the waveform A and the waveform B are compared, they are expressed as A / B, and are used similarly as symbols for waveform comparison.

  In this example, in the comparison between A / C and A / B, A is dominant only in the band in which A's pure tone is scattered, and B and C are dominant in other bands. . Therefore, when comparing B / C, when comparing with the dominant number of bands, C is superior with 2 for B and 3 for C.

  6e) In determining the masker sound, the relationship between the separated sounds is evaluated over the entire band, and the dominant sound is determined based on the dominant sound standard.

  In this example, the results of evaluation of the dominant band of separated sound from the time T-2 to the current time T are listed, and a band with a change is extracted. As a result, in A / C and A / B, it can be determined that A is dominant. As a result of the comparison of B / C, it can be determined that there is no dominant separated sound because the number of changes of B and C is equal. Eventually, it can be determined that the sound of A is dominant.

  Here, the horizontal axis of FIG. 6e) shows a sequence of dominant separated sound identifiers for each band, and the vertical axis shows time T-2, time T-1, etc. in units of frames. ing. For example, the interval between time T-2 and time T-1 in frame units is preferably about 30 ms.

  In addition, as a criterion for determining a dominant sound different from the above, when the amplitude difference with the corresponding separated sound in a certain band is extremely large (for example, 20 dB), it is applied not only to the influence in the band but to the entire separated sound. The impact is considered large. Therefore, if there is a separated sound having an extremely large amplitude difference from the corresponding separated sound in at least one band, it may be determined as a dominant sound. In addition, as another criterion for determining a dominant sound, (1) when the temporal amplitude change of a predetermined band of separated sound is large (for example, when there is an amplitude change of 20 dB or more in a predetermined band), or (2) constant When there is a large amplitude fluctuation of a predetermined number or more in time (for example, when an amplitude fluctuation of 10 dB or more is a predetermined number (for example, 5 times) or more per predetermined time (for example, 10 seconds)) Judging that it is intense, it can be regarded as dominant. These criteria are based on the fact that humans are more interested in fluctuations such as louder sounds and rising and falling edges.

  In the case of a candidate sound whose priority order is not determined as in the comparison result of the B / C waveform in FIG. 6e), the candidate sound presentation specifying unit 106 starts with the separated sound having the dominant band. May be presented. Note that the number of bands dominant in the time direction may be counted. That is, among the separated sounds at a time closer to the current time (for example, a frame, etc.), the number of bands in which the separated sounds are dominant is counted, and the larger the number of counted bands, the more the candidate sounds to be processed. You may make it show previously. In such a case, the priority of the separated sound within a closer range may be increased from the timing of pressing the discomfort detection button in FIG. 4B. That is, in the case of FIG. 4B, assuming that a separated sound that rises in 5 seconds from the timing of pressing the discomfort detection button is a candidate for processing, as in this case, If priority is given to the candidate sound to be processed as the number of dominant bands increases, the number of dominant bands only needs to be examined for 3 seconds, for example, from the timing of pressing the discomfort detection button. .

  Alternatively, a dominant band may be determined and the determination may be performed only for that band. As a result, in all bands, for example, when the separated sound C is dominant, it can be determined that C is an unpleasant sound that causes irritation, so no sound is presented. Candidates may be identified and determined. The dominant band may be determined based on the user's hearing, especially the band that is easy to hear. Further, it may be determined from the band of the sound specified as an unpleasant sound. With this configuration, by calculating only the band necessary for unpleasant sound estimation, it is possible to reduce processing necessary for estimation without reducing the estimation accuracy.

  FIG. 7 is a flowchart showing the processing of the candidate sound presentation specifying unit 106 in the first embodiment. In S <b> 408, the candidate sounds determined by the candidate sound selection determination unit 105 with priorities are presented to the user, and the sound causing the irritation is specified. At this time, a selection button is provided in order to specify the sound that causes irritation, and the user presses the button to select the sound that causes irritation from the presented candidate sounds.

  Further, the content of the sound to be presented may be only the candidate sound, or may be a sound obtained by subtracting the candidate sound from the sound before separation.

  Furthermore, even when only one candidate sound is determined in S401 or S404, it is possible to prevent unnecessary updating of the sound characteristics of unpleasant sound by always presenting the candidate sound and confirming it to the user. Can do.

  If the order of dominant sounds is not determined, a sound obtained by removing all separated sounds as candidate sounds may be heard, and the user may be collectively confirmed as to whether or not the state is preferable. In this way, first of all, the user can quickly answer the desire to be freed from annoyance and quiet.

  According to such a configuration, by evaluating the relationship between the separated sounds immediately before detection of discomfort, it is possible to quickly identify the sound by estimating the discomfort sound that causes frustration, and to reduce discomfort Can be made.

  The candidate sound presentation specifying unit 106 may be provided with a selection button for specifying the sound that the user wants to hear. In this case, the candidate sound presentation specifying unit 106 wants to listen to the candidate sound selection determining unit 105, for example, with a priority order opposite to that of the unpleasant sound, in the same manner as the procedure of S408 shown in FIG. A candidate sound is presented to the user. Then, the user presses a button for selecting a sound to be heard from the presented candidate sounds, thereby determining the sound that the user wants to hear. The sound to be heard determined in this way may be excluded from the candidate sounds to be processed, and the user may select a sound that the user does not want to hear from the unpleasant sound candidates from which the sound to be heard is excluded. In addition, the sound structure of the determined sound to be heard is extracted by the processed sound structure feature updating unit 107 and stored in a region different from the specific sound in the processed sound structure feature DB 108. The sound processing unit 104 adds the in-phase waveform of the desired sound to the mixed sound from which the unpleasant sound has been removed, based on the sound structure characteristics of the desired sound stored in the processed sound structure characteristic DB 108, or wants to hear Only the waveform generated from the separated sound is amplified to reconstruct the mixed sound. As a result, there is an effect that it is possible to make it easier for the user to hear the desired sound.

  FIG. 8 is a diagram illustrating a correspondence relationship between Macheny's psychological model and the present invention. Below, the effect of the present invention will be described by associating the present invention with reference to Macheny's psychological model shown in FIG. 8 (described in New Physiological Psychology, Vol. 3, Chapter 2, pages 12-14, Kitaoji Shobo).

  According to this model, “Stress is the starting point of stress because of demands on oneself, changes in life, demands for roles, and frustration, and conditioned experiences like fear are conscious. Except for causing a direct stress response without awareness, first the requirements are noticed, and the intensity or importance of the awareness is estimated (temporary assessment) and compared with the resources to deal with them ( Two-dimensional assessment) If the person's coping resources address the demand, the demand is considered a challenge and remains healthy, but the demand and resource estimates do not match, On the other hand, if the request is too strong, the request is regarded as a stressor, and a stress response occurs and ultimately leads to stress symptoms. "

  Using this model, it will be explained that the present invention can be associated with the one that reduces the stress caused by sound. When you notice the presence of a stress source sound and become irritated as a stressor as described above, you can detect discomfort, identify the discomfort sound that causes irritation, and process the sound. Thus, it is possible to escape from the state in which the stress reaction has occurred, so that it is possible to avoid or reduce the stress caused by the unpleasant sound that is the source of stress.

  This also serves to increase a person's coping resources because the stress source can be avoided by specifying and extinguishing the stress source sound. Moreover, the impression with respect to the sound of a stress source can be changed by changing the timbre of the unpleasant sound used as a stress source by changing only a part of zone | band. In addition, by replacing the stress source with another sound, the coping resources can be increased.

(Second configuration diagram in Embodiment 1)
FIG. 9 is a block diagram showing a configuration of a sound selection processing apparatus 1100 that is the second configuration of the first embodiment. 9, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 9, the sound selection processing device 1100 includes a processing reference input unit 135 and a search target sound structure feature DB 136 in addition to the configuration of the sound selection processing device 100 of FIG. The search target sound structure feature DB 136 is a storage unit that stores the sound structure feature of the separated sound extracted by the above-described separated sound extraction method, and the sound structure feature of the separated sound targeted by the sound separation unit 103 is stored. To extract sound. The processing reference input unit 135 will be described in detail later.

  In the search target sound structure feature DB 136, when the processed sound structure feature update unit 107 updates a sound that the user does not want to hear, the separated sound other than the separated sound specified as the sound that the user does not want to hear is acquired. Stored in association with the time and place. As a result, in a train or the like, the background sound in the train whose sound structure feature is stored in the search target sound structure feature DB 136 is first removed from the acquired mixed sound, and the sound that you do not want to hear is removed. As a result, the candidate sounds to be processed can be reduced in advance.

  FIG. 10 is a flowchart showing the separation sound extraction accompanied by the determination of the separation sound search target sound in the sound separation unit 103 in the sound selection processing device 1100 according to the second configuration of the first embodiment. In this figure, the same components as those in FIG. 2C are denoted by the same reference numerals, and description thereof is omitted. While FIG. 2C determines whether the specific sound matches the separated sound, in FIG. 10, the purpose is different to extract the separated sound based on the search target sound. This is common in that the sound is processed by determining the degree of coincidence of the two separated sounds based on the distance between the two.

  In S721, the sound structure feature amount of the mixed sound corresponding to the search target sound is calculated. In S717, the distance between the mixed sound and the sound structure feature amount of the extraction target sound is obtained. If the distance is equal to or smaller than the predetermined threshold value in S718, it means that the search target sound has been found. In this case, the search target sound is developed into a time waveform in S722, and the difference between the mixed sound and the search target sound is obtained in S723. By repeating this operation for each search target sound, a separated sound that has already been registered as a search sound can be separated from the mixed sound.

  FIG. 11 is a flowchart showing a process when a separated sound is selected in the sound selection processing apparatus 1100 according to the second configuration of the first embodiment. A description of the same processing as in FIG. 3 is omitted. In S601, discomfort is detected, and the processing when no discomfort is detected is different from the configuration shown in FIG. At this time, in S621, the presence or absence of the sound structure feature corresponding to the search target sound is checked, and in S622, it is checked whether or not the sound structure feature constituting the mixed sound has changed, and the change is recognized. If so, the sound structure feature is updated as the search target sound in S623.

  The presence / absence of the sound structure feature is determined using the distance between the feature values in S718 described also in FIG. 2C and FIG. You may make it determine by adding the distance for a fixed time.

  In addition, the change in the sound structure feature means that, for example, the frequency characteristic is swept even if the sum of the scales of the distance for a certain time is in a range that matches the search sound feature amount. This indicates a case where a part of the distance is particularly deviated. The update of the search target sound compared to the predetermined distance may replace the sound structure feature amount of the search target sound itself, may newly add another search target sound, The distribution feature may be updated by providing a distribution of the sound structure feature quantity so as to express the degree of change of the sound structure. In addition, the user may be asked to confirm whether or not to actually update.

  According to such a configuration, after separating the mixed sound based on the determined sound as the sound that the user does not want to hear before, for example, the sound structure characteristic is searched for using the background noise peculiar to the background noise or the place as the target sound. By recording in the sound structure feature DB 136 and using it for sound separation during separation processing, separation can be performed more quickly and accurately. That is, when a search target sound that has been heard in the past is included in the currently acquired mixed sound, first, the target sound that has been heard in the past stored in the search target sound structure feature DB 136 is first separated from the mixed sound. Therefore, it becomes possible to easily estimate and present a new unpleasant sound included in the mixed sound. Thereby, the calculation load of the sound selection processing apparatus 1100 can be reduced.

  When a sound that is not desired to be heard is specified and the sound structure feature of the separated sound is stored in the processed sound structure feature DB 108, the currently stored sound structure feature amount and the sound structure to be newly stored When the distance of the feature amount is determined as in S171 of FIG. 2C and is equal to or less than a predetermined threshold value, new storage in the search target sound structure feature DB 136 may be made. Further, it is possible to confirm with the user whether the sound source is the same sound source, and to store a plurality of sound structure features due to the same unpleasant sound. Furthermore, the transition information of the plurality of unpleasant sounds may be included.

  With such a configuration, in internal processing such as comparison of separated sounds, a plurality of streams are registered in the processed sound structure feature DB 108 and presented to the user as sounds that one sound source does not want to hear. Become so.

  Further, when a sound that is not desired to be heard is specified and the sound structure feature value is stored in the processed sound structure feature DB 108, it is recorded in the search target sound structure feature DB 136 simultaneously with the sound structure feature of the separated sound. You may do it. You may record the result of having compared the separated sound structure whether it is the same as the sound environment expressed from the structure of the separated separated sound. By this determination, for example, when a sound that the user does not want to hear is continuously changing, even if the distance from the specific sound structure feature stored in the sound feature structure DB is outside a predetermined value, other separation is performed. It can be determined that most of the sound configurations (for example, 10) (for example, 8) coincide and the same environment continues. Then, for example, by confirming with the user and determining whether or not the sound is the same, it is possible to save information that the same sound that the user does not want to hear has changed, so unnecessary update of the sound feature structure DB Can be prevented.

  Furthermore, as shown in FIG. 10, when the separated sound is extracted by comparison with the search target sound, the search target sound structure feature DB 136 is stored together with the sound stored in the processed sound structure feature DB 108 as information on the search target sound. It is also possible to obtain the similarity of the place and time based on the information such as the place and time acquired by GPS and the like, and thereby narrow down the search target sound.

  According to this configuration, since sound separation based on the search target sound can be performed after the continuity of the mixed sound and the search target sound are narrowed down, the sound separation can be performed more quickly.

  Furthermore, the distance scale of S717 may be obtained in consideration of scenes such as reflection. For example, when the cross-correlation function in the time direction is calculated for the time waveform or the characteristic frequency component of the input sound and the sound to be searched before the distance scale determination in S717, and the phase lag is confirmed in the cross-correlation value, Assuming that the reflected waves have different arrival times from the sound source, the distance calculation for the reflected waves may be performed by subtracting the influence on the direct waves from the input sound. As long as the distance from the search target sound within a certain time is within a threshold, that is, the same, it is considered that the reflected wave is a change in the sound structure characteristics and can be deleted.

  According to such a configuration, even if the sound source direction cannot be accurately obtained due to the diffuse sound field, the reflected sound having a time delay and a slight change in sound characteristics is also determined based on the similarity to the search target sound. It is considered that the data can be erased more reliably by performing.

  It is known that when a person feels tension or discomfort, the sympathetic nerve function is activated and, for example, the heart rate, blood pressure, and respiratory rate are increased. Therefore, discomfort may be detected using a biological signal such as skin electrical resistance. For example, as a technique for estimating subjective evaluation of sound, there is a method of estimating pleasant unpleasant sound using a biological signal for collected sound (for example, Japanese Patent Application Laid-Open No. 2004-337294). It should be noted here that it is necessary to determine whether the stress source is due to sound. Therefore, it may be determined whether or not the user feels discomfort due to sound by combining an auditory evaluation scale (for example, JP-A-2001-165766) using a loudness change.

  FIG. 12 is a flowchart showing processing for detecting discomfort in the sound selection processing apparatus 1100 according to the second configuration of the first embodiment. Description of the same processing as in FIG. 4 is omitted. In this flowchart, a loudness change greater than or equal to a predetermined value occurs more than a predetermined number of times (for example, 5 times) per predetermined time (for example, 10 seconds). If any of the conditions is not satisfied, it is determined that the discomfort button is not pressed by mistake.

  According to such a configuration, it is possible to determine whether or not the button has been pressed by mistake, so that it is possible to suppress unnecessary calculation of the search target sound.

  The processing reference input unit 135 is a processing unit that inputs a processing condition for sound volume with respect to the sound specified as the processing sound by the candidate sound presentation specifying unit 106. This makes it possible to perform flexible sound processing rather than a clear processing method of whether to erase or leave. Furthermore, by processing the sound that you want to hear, or neither sound, you can process the sound you do not want to hear by processing the sound that you do not want to hear at -6dB, which is about half the volume of the other sounds. While controlling, it is possible to make sound of the environment close to the sound of the actual environment.

  According to such a configuration, unpleasant sounds that cause irritation can be generated by extracting sounds unique to a specific place, user's own voice, user's own property, etc. as sound structure features. Since it becomes possible to narrow down more, an unpleasant sound can be specified more reliably.

  In addition, in the candidate sound presentation specifying unit 106, in addition to the sound itself, attributes such as the type of the sound that are given to the search target sound DB may be presented together as the presented content. Further, the order of presentation may be changed according to attributes such as the type of sound. In the case of separated sounds that are not included in the search target DB, sound information is acquired using a means for identifying the sound through a model such as GMM.

  According to this configuration, for example, it is possible to select and delete all human voices at once without listening to the separated sounds presented one by one, which can be said to be very efficient. .

  Furthermore, the sound structure feature of the separated sound identified as the sound that is not desired to be heard when the processed sound structure DB 108 is updated is updated after the sound structure stored in the search target sound structure feature DB 136 is compared with the distance. Whether or not to do so may be presented to the user and determined. Alternatively, the separated sound group generated at the same time as the separated sound specified this time may be compared with the separated sound group in which the similarity between the search target DBs is recognized and the separated sound group may be determined as the same environment. It should be noted that the result of the sound type determination of each separated sound can be used for comparison of separated sound groups and environment. According to such a configuration, it becomes possible to detect that a sound that is not desired to be heard is switched, and at the same time, it is possible to perform sound processing in consideration of the replacement.

(Embodiment 2)
FIG. 13 is a block diagram illustrating a configuration of the sound selection processing device 1500 according to the second embodiment. In FIG. 13, the same components as those in FIGS. 1 and 9 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 13, in addition to the configuration shown in FIG. 9, the sound selection processing device 1500 includes a processed sound structure receiving unit 137 and a processed sound structure transmitting unit 138. The candidate sound selection determining unit 105 further includes a separated sound masking outline determining unit 132.

  The processed sound structure receiving unit 137 receives the processed sound identified as an unpleasant sound that causes the other person to be frustrated in the sound selection processing device used by the other person. Similarly, the processed sound structure transmission unit 138 transmits the processed sound specified by the user to the other party.

  By adding the separated sound masking outline determining unit 132 to the candidate sound selection determining unit 105, the selection of the candidate sound is performed by estimating the mask sound in consideration of the influence of the sound on human perception on another sound. Will be able to.

  FIG. 14 is a flowchart showing processing of the candidate sound selection determination unit 150 in the second embodiment. Description of the same processing as in FIG. 7 is omitted.

  The difference from the configuration of FIG. 7 is that in the formation of the sound structure comparison signal waveform in S502, after selecting a band representative signal for sound structure comparison in S510, the representative signal is estimated masking level waveform for each band in S511. In S512, the outlines of the masking levels are synthesized to obtain a sound structure comparison signal.

  FIG. 15 is a comparison flowchart of the candidate sound selection determination unit 105 according to the second embodiment. In this figure, a mixed sound is composed of the same three types of separated sounds as in FIG. The description of the same processing as in FIG. 6 will be omitted. FIG. 15 differs in that c) as a comparison signal waveform, b) a band representative signal is converted into f) a masking level outline, and an envelope signal is obtained.

  Thus, once converting to the masking level is equivalent to comparing separated sounds in consideration of auditory hearing. Therefore, it can be expected that a judgment closer to the user's hearing can be made compared with the comparison of separated sounds by signal analysis in FIG.

  FIG. 16A is a diagram showing FIG. 5 of a masking technique published in a patent document (Japanese Patent Laid-Open Publication No. 2000-506631 publication, two sound evaluation methods using masking). FIG. 16B is a view showing FIG. 19 of the masking technique published in the patent document (Japanese Patent Laid-Open No. 2000-506631). Note that the masking level is determined by considering the simultaneous masking obtained from knowledge as in the method shown in FIG. 16 (a) and the successive masking as shown in FIG. 16 (b). You may make it do.

  FIG. 16A is a curve of measured attenuation amounts in order to examine the influence of individual sound pressure levels on other frequency regions when a 1000 Hz sine wave is applied from 0 dB to 100 dB. This figure takes into account the transfer functions of the outer ear and the middle ear. The horizontal axis in FIG. 5 of the document shown in FIG. 16A represents the frequency in Hz, and the vertical axis represents the ear attenuation over the entire frequency domain, and is expressed in dB.

  From this figure, it can be seen that the influence of a 1000 Hz sine wave given with a loud sound of 0 dB affects not only 1000 Hz but also a fairly wide band. It can also be seen that even when given a small sound of 100 dB, there is an effect in the low frequency direction. This effect is interpreted as a physical structural characteristic that the cochlea vibrates in the direction of travel as sound propagates.

  In other words, for sounds with a single frequency in signal analysis, it is possible to make judgments closer to hearing by considering the influence (masking) on other frequency bands in terms of hearing. Conceivable.

  Note that the influence of the sound in the frequency direction shown in FIG. 16A is called simultaneous masking, and a high sound has a tendency to be easily erased by a low sound, and further, the connection time shown in FIG. In masking, sounds approaching in time affect each other. It has the characteristics of By reflecting such a person's hearing, it becomes possible to select a sound in consideration of the influence of a sound that is not desired to be heard. In other words, in order to take into account successive masking, it is necessary to look at the components in the time direction.

  Therefore, as shown in FIG. 15, in consideration of human hearing characteristics, the dominant sound estimation process uses the masked transformation outline, and finally considers the dominant band and time transition and controls The separation sound that becomes a specific band or masker is determined.

  In addition, as a result of evaluating the relationship between separated sounds, for example, when there is no dominant sound in which each of the three separated sounds is determined to be a masker, all of them are of the same magnitude and are nested. In the state, it is assumed that a plurality of sounds of the frustrating sound generation factor 3 in FIG. Therefore, the presentation may be performed in consideration of that.

  In the present embodiment, the sound acquisition unit 101 may be linked simultaneously to provide a direction suppression effect like an array microphone.

  With such a configuration, it is determined whether or not the other party's unpleasant sound is the same as that of the other person, and if it is the same, the sound can be selected using more directivity by operating as an array microphone. You can also select it from the cut end. In addition, by sending to the other party the sound structure characteristics recorded in a place closer to the sound source of the sound that is the cause, it is possible to perform processing as a sound that the other party does not want to hear more actively Become.

  In addition, the processed sound structure receiving unit 137 and the processed sound structure transmitting unit 138 transmit data that is identified as an unpleasant sound that causes annoyance, so that the other party identifies the sound identified as an unpleasant sound by itself. You may make it the structure which checks whether there exists any target sound which hears or receives from the other party in own search target sound structure characteristic DB.

  With such a configuration, for example, if it is found that the unpleasant sound from the other party is the sound that originates from you or your property, you can get an opportunity to make efforts to prevent that sound from coming out. it can.

  FIG. 17 is a diagram schematically showing an example in which two people A and B communicate discomfort sound information. In this example, the person A specifies the train straddling sound and the voice of the surrounding person X, and the person B specifies the voice of the person X as an unpleasant sound like the train motor sound, and sends information on the unpleasant sound to each other. ing. At this time, by comparing the unpleasant sounds, it can be seen that there is a voice of the person X as a common unpleasant sound. Therefore, in order to erase the voice of the person X, it is possible to extract the separated sound with higher accuracy by performing the sound separation and extraction process on the assumption that the input of the virtual microphone has increased as in the case of the array microphone.

  In addition, compared with the separated sound other than the unpleasant sound of the person A, it is determined that there is a sound that matches the separated sound that is occurring around the person. The sound may be presented to the person A. By doing in this way, the person A knows that the sound is generated by, for example, the decoration on the bag held by the person A due to the shaking of the train. As a result of this determination, the person A can be given an opportunity to take action so that the sound is not emitted.

  According to such a configuration, a shared sound environment such as an array microphone can be configured with another person using the same sound selection processing device via a communication network or the like. As a result, a mixed sound without an unpleasant sound can be reconstructed based on the other party's unpleasant sound and one's own unpleasant sound, so that a better sound environment can be constructed.

  Further, the data to be transmitted by the processed sound structure receiving unit 137 and the processed sound structure transmitting unit 138 may not be the sound itself specified as an unpleasant sound that causes irritation, but may be a pulse sound synchronized with the specific sound. Further, when the sound class is acquired by the discriminator, when the information such as the direction is known, the attribute information such as the type and direction of the sound may be transmitted.

  According to such a configuration, it is possible to suppress necessary calculation amount and data amount by giving attribute information such as the type and direction of sound as higher-level information than the acoustic signal itself. For example, by directly transmitting information such as the sound of XXX from the YY direction to the other user, the user can start from the judgment that the other user himself presses a new discomfort button.

  Each functional block in the block diagrams (FIGS. 1, 9, 13, etc.) is typically realized as an LSI which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  (For example, the functional blocks other than the memory may be integrated into one chip.)

  The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied.

  In addition, among the functional blocks, only the means for storing the data to be encoded or decoded may be configured separately instead of being integrated into one chip.

  Since the sound selection processing device according to the present invention has a mechanism for quickly selecting an unpleasant sound causing annoyance, it is necessary to select a sound by recording an image sound and editing an unnecessary sound. It is useful in the equipment to do. Further, by selecting a sound while recording the sound in an IC recorder or the like, it can be applied to a sound filtering device or the like in real time. For example, if you record a lecture and record it as the target sound feature structure of the sound you expect, such as the sound of a speaker or student moving a chair or desk, coughing, or sneezing, you can select the sound only for the lecture content. Can be recorded.

  Furthermore, by sharing information about unpleasant sounds with others, in addition to calls in remote areas such as mobile phones, it is possible to use hearing aids that reduce sounds that you do not want to hear from each other even in physically close locations such as noise. Application is also useful.

Configuration diagram of sound selection processing apparatus according to Embodiment 1 Flow chart of processing at the time of separated sound processing in the first embodiment Flow chart of processing when separated sound is selected in the first embodiment Flow chart of discomfort detection unit in the first embodiment Flow chart of candidate sound selection determination unit in the first embodiment Flow chart of candidate sound selection processing in the first embodiment Flow chart of candidate sound selection determination unit in the first embodiment Illustration of effect of the present invention by contrast of stress model Configuration diagram in the second configuration of the first embodiment Flow chart of comparison of separated sound extraction target sounds in the second configuration of the first embodiment Flowchart of processing when selecting separated sound in the second configuration of the first embodiment Flow chart of discomfort detection in the second configuration of the first embodiment Configuration diagram of sound selection processing apparatus according to the second embodiment Flowchart of candidate sound selection determination unit in the second embodiment Flow chart of comparison of candidate sound selection determination unit in the second embodiment Masking outline drawing Application example of the present invention Example of a sound scene in a train Example explanation of the sound you want to listen to subjectively Example of annoying factors caused by sound

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Sound selection processing apparatus 101 Sound acquisition part 102 Sound structure feature extraction part 103 Sound separation part 104 Sound processing part 105,150 Candidate sound selection determination part 106 Candidate sound presentation specific part 107 Processed sound structure feature update part 108 Processed sound structure characteristic DB
131 Discomfort detection unit 132 Separation sound masking outline determination unit 133 Sound structure feature comparison unit 134 Masker separation sound estimation unit 135 Processing reference input unit 136 Search target sound structure feature DB
137 Processed sound structure receiving unit 138 Processed sound structure transmitting unit 1100 Sound selection processing device

Claims (10)

A sound selection processing device that selectively removes unpleasant sounds from the mixed sound generated around the user,
Sound separation means for separating the mixed sound into sounds for each sound source;
Discomfort detection means for detecting that the user is in an uncomfortable state;
When it is detected by the discomfort detection means that the user is in the state, the relationship between each separated sound that is a separated sound is evaluated, and based on the evaluation result, the separated sound of the candidate for processing is obtained. A candidate sound selection determining means to be estimated;
Presenting the separated sound of the estimated processing target candidate to the user, accepting the selection, and candidate sound presentation specifying means for specifying the selected separated sound;
A sound selection processing apparatus comprising: sound processing means for processing the identified separated sound to reconstruct a mixed sound. The sound separation means separates the mixed sound by extracting a sound structure characteristic for each sound source included in the mixed sound,
The candidate sound selection determining means includes
A sound structure feature comparison unit that evaluates the relationship between the separated sounds by comparing the signals formed from the separated sounds;
Based on the evaluation result, a masker separated sound estimation unit that estimates the separated sound of the candidate to be processed by estimating a separated separated sound or a dominant frequency band among the separated sounds. The sound selection processing apparatus according to claim 1, further comprising: The discomfort detecting means detects that the user is in an uncomfortable state by using any one of input sound loudness change, skin electrical resistance fluctuation, and button press detection. Sound selection processing equipment. The candidate sound presentation specifying means presents the separated sound separated from the acquired mixed sound to the user and accepts selection of a sound to be heard from the presented separated sounds,
The sound selection processing apparatus according to claim 1, wherein the sound processing means performs processing so as to amplify the selected sound to be heard and reconstructs the mixed sound. The sound structure feature comparison unit compares the magnitudes of the signals formed from the separated sounds included in the mixed sound acquired within a predetermined time, thereby increasing the magnitude of the formed signals. Order the separated sounds as the dominant separated sounds,
The candidate sound presentation specifying unit presents a sound obtained by subtracting the separated sound of the candidate for processing from the separated sound or the mixed sound of the candidate for processing according to the order of dominant separated sounds. Item 2. The sound selection processing apparatus according to Item 2. The sound selection processing device further includes:
Search target sound structure feature storage means for accumulating separated sounds other than the separated sounds specified by the candidate sound presentation specifying means in the past;
A sound structure feature storage means for accumulating the separated sound newly specified by the candidate sound presentation specifying means,
When the acquired mixed sound includes a separated sound related to a location accumulated in the search target sound structure feature storage unit, the sound structure feature storage unit displays a newly specified separated sound. The sound selection processing device according to claim 1, wherein the sound selection processing device is not stored. The sound selection processing device further includes:
A discriminator for identifying the type of separated sound specified by the candidate sound presentation specifying means;
The sound selection processing apparatus according to claim 1, wherein the candidate sound presentation specifying unit presents the type of the sound along with the separated sound of the estimated processing target candidate. Sound acquisition means for acquiring mixed sound generated in the surroundings;
A processed sound structure transmitting unit that transmits the sound structure of the separated sound specified by the candidate sound presentation specifying means to another sound selection processing device;
A processing sound structure receiving unit that receives the sound structure of the separated sound specified in the other sound selection processing device from the other sound selection processing device;
The sound acquisition means is an array together with the sound acquisition means of the other sound selection processing device when the separated sound specified by the candidate sound presentation specification means matches the separation sound received from the other sound selection processing device. The sound selection processing apparatus according to claim 1, wherein the sound selection processing apparatus operates as a microphone. A sound selection processing method for selectively removing unpleasant sounds from the mixed sound generated around the user,
A sound separation step for separating the mixed sound into sounds for each sound source;
A discomfort detection step for detecting that the user is in an uncomfortable state;
When it is detected by the discomfort detection step that the user is in the state, the relationship between each separated sound that is a separated sound is evaluated, and based on the evaluation result, the separated sound of the candidate for processing is obtained. A candidate sound selection determination step to be estimated; and
Presenting the separated sound of the estimated processing target candidate to the user, accepting the selection, and a candidate sound presentation identifying step for identifying the selected separated sound;
And a sound processing step of processing the identified separated sound to reconstruct a mixed sound. This is a program for a sound selection processing device that selectively removes unpleasant sounds from the mixed sound generated around the user, and the computer separates the mixed sound into sounds for each sound source. A step, a discomfort detection step for detecting that the user is in an uncomfortable state, and each separated sound that is a separated sound when the user is detected in the state by the discomfort detection step A candidate sound selection determination step for estimating a separation sound of a candidate for processing based on the evaluation result, and presenting the separation sound of the estimated candidate for processing to the user for selection , A candidate sound presentation specifying step for specifying the selected separated sound, and a sound processing step for processing the specified separated sound to reconstruct the mixed sound.

Images