JP2017097160A - Speech processing device, speech processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that prevents the speaker of an unintended speech and/or the content of utterance by the speaker of the intended speech from being specified from an acoustic signal that includes an intended speech even when the unintended speech and the intended speech overlap.SOLUTION: A speech processing device of the present invention comprises: detection means for detecting the utterance section of utterance by the speaker of an unintended speech from a first input signal that is a sensor signal, and specifying time information that indicates the start time and end time of the utterance section; and processing means which, in order for at least one of the content of speech and the speaker to be not specified by a signal, out of a second input signal that is an acoustic signal including the speech of the speaker of an intended speech and the unintended speech, that is a portion equivalent to a period that corresponds to the specified time information, processes the portion of the second input signal that is equivalent to the period.SELECTED DRAWING: Figure 15

Description

  The present invention relates to a voice processing device, a voice processing method, and a program.

  There is a technique for emphasizing and detecting target speech by noise canceller, beam forming, or the like, and suppressing and removing other than the target speech. As such a technique, for example, the technique of Patent Document 1 is known.

  In the technique described in Patent Document 1, an input signal is converted from a time domain to a frequency domain, and based on a signal having a first directivity and a signal having a second directivity each having a blind spot in a predetermined direction, A coherence value is obtained, and a coherence gradient is obtained from the value. In the technique described in Patent Document 1, when the target sound section detection unit determines that the coherence value is larger than a predetermined target sound section determination threshold or the coherence gradient is smaller than the coherence gradient determination threshold, the target sound section is determined. Otherwise, it is determined as a non-target sound section, and a gain is set according to the determination result.

JP 2013-1206026 A

  In the technique described in Patent Document 1, it is determined using a coherence value whether or not a target speech section (target sound section) that is a speech section of the target speech. Therefore, when the target sound (target voice) and the non-target voice overlap at the same time, the target voice and the non-target voice may exist in the target voice section. Therefore, in the technique of Patent Document 1, there is a possibility that the non-target voice speaker and the utterance content remain in the target voice section to the extent that the person can be understood.

  In addition, for example, a case is assumed in which the store manager confirms the utterance content of a store clerk who talks with a customer at a store payment counter by listening to sound later, or automatically processing and analyzing using speech recognition or other techniques. . In this case, the clerk's voice is the target sound and the customer's voice is the non-target voice. In the above case, when a voice section is detected using the clerk's voice as the target voice, when the clerk and the customer speak at the same time, the voice of the customer as the non-target voice is mixed into the detected voice section.

  In this way, the non-target sound remains or is mixed in the sound signal of the sound section to be output so that a person can understand, so that the third party who has listened to the sound of the sound signal of the sound section There is a problem that the utterance content and the speaker who made the utterance are specified.

  An object of the present invention is to prevent the utterance contents of a non-target voice speaker and / or a non-target voice speaker from being identified from an acoustic signal including the target sound even when the non-target voice overlaps with the target sound. It is to provide technology to do.

  The speech processing apparatus according to an aspect of the present invention detects a speech segment of an utterance by a speaker of a non-target speech from a first input signal that is a sensor signal, and represents time information indicating a start time and an end time of the speech segment Utterance contents and a speaker by a signal in a period corresponding to the specified time information out of a second input signal which is an acoustic signal including a target sound and the voice of the non-target voice speaker Processing means for processing the signal of the period in the second input signal so that at least one of the above is not specified.

  In addition, the speech processing method according to an aspect of the present invention detects an utterance section of an utterance by a speaker of non-target speech from a first input signal that is a sensor signal, and represents a start time and an end time of the utterance section. Of the second input signal that is the acoustic signal including the target sound and the voice of the non-target speaker, the time information is specified, and the content of the utterance and the speaker's voice are determined according to the signal of the period corresponding to the specified time information. The signal of the period in the second input signal is processed so that at least one is not specified.

  Note that a computer program for realizing the above apparatus or method by a computer and a computer-readable non-transitory recording medium storing the computer program are also included in the scope of the present invention.

  According to the present invention, even when the non-target voice overlaps with the target sound, the utterance content by the non-target voice speaker and / or the non-target voice speaker cannot be specified from the acoustic signal including the target sound. can do.

It is a figure for demonstrating an example of the application scene of the audio processing apparatus which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 1st Embodiment of this invention. It is a figure which shows the image of a 1st input signal, a 2nd input signal, and an utterance area process signal. It is a flowchart which shows an example of operation | movement of the speech processing unit which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the modification 1 of the 1st Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech recognition apparatus which concerns on the 2nd Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 3rd Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the speech processing unit which concerns on the 3rd Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 4th Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the speech processing unit which concerns on the 4th Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 5th Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the speech processing unit which concerns on the 5th Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 6th Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the speech processing unit which concerns on the 6th Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the speech processing unit which concerns on the 7th Embodiment of this invention. It is a figure which illustrates illustartively the hardware constitutions of the computer (information processing apparatus) which can implement | achieve each embodiment of this invention.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
First, an example of an application scene of the sound processing apparatus according to the first embodiment of the present invention will be described with reference to FIG. In the present embodiment, the input signal input to the sound processing device is a sensor signal. The sensor signal is a signal output from a sensor such as a microphone or a camera. In this embodiment, a microphone is employed as an example of a sensor. Since the microphone collects sound, the sensor signal output from the microphone is an acoustic signal.

  Further, in the present embodiment, the input signal input to the voice processing device is a signal representing the sound collected in the conversation scene between the store clerk and the customer at the store payment counter as shown in FIG. Suppose. In the present embodiment, an example is described in which there are one clerk and one customer, but each clerk and customer is not limited to one. Moreover, in this embodiment, the said conversation scene assumes the conversation scene in the state where the salesclerk and the customer face each other across the payment counter as shown in FIG. In other embodiments described below, unless otherwise specified, a conversation scene between a store clerk and a customer at a store payment counter as shown in FIG. 1 is assumed as in this embodiment.

  As shown in FIG. 1, a microphone (second microphone) m2 that mainly collects the voice of the store clerk operating the Point Of Sale (POS) register provided on the payment counter, and the voice of the customer are mainly collected. It is assumed that a microphone (first microphone) m1 is provided. In FIG. 1, the microphone m1 is provided at a position closer to the customer than the store clerk on the POS register in order to mainly collect the voice of the customer. In FIG. 1, the microphone m2 is provided on the POS register at a position closer to the store clerk than the customer in order to mainly collect the store clerk's voice.

  Thus, by providing the microphone m1 and the microphone m2, the acoustic signal output from the microphone m1 mainly includes a customer's voice signal (voice signal). In addition, the sound signal of the clerk's voice is also mixed into the acoustic signal output from the microphone m1 due to the sound wrapping around the microphone m1. The acoustic signal output from the microphone m2 mainly includes a voice signal of a store clerk's voice. In addition, the sound signal output from the microphone m2 includes the sound signal of the customer's voice due to the sound wrapping around the microphone m2.

  In this embodiment, it is assumed that the clerk's utterance content is confirmed. That is, the target sound is the clerk's voice. The sound source of the target sound is a store clerk. As described above, the sound signal output from the microphone m2 includes the sound signal of the store clerk's voice and the sound signal of the customer's voice. For this reason, if the acoustic signal output from the microphone m2 is converted into sound and output, the output sound includes the customer's sound. Therefore, in the present embodiment, in order to protect the privacy of the customer, the signal in the section including the voice signal of the customer's voice is processed among the acoustic signals output from the microphone m2. In the present embodiment, the non-target voice is the customer voice. In other embodiments described below, the target sound is the clerk's voice and the non-target voice is the customer's voice, as in this embodiment, unless otherwise specified.

  In addition to the clerk's utterance, for example, a third party may want to evaluate the clerk's work by listening to the payment sound or announcement of electronic money, so the target sound is not limited to the voice of the clerk's utterance. In the embodiment, the following description will be made assuming that the target sound is a voice generated by a store clerk.

  In the following description, the acoustic signal output from the microphone m1 and input to the speech processing device is referred to as a first input signal, and the acoustic signal output from the microphone m2 and input to the speech processing device is referred to as a second input signal. Also, it is assumed that signals (first input signal and second input signal) collected by the microphone m1 and the microphone m2 at the same timing and output from the microphone m1 and the microphone m2 are associated with each other.

(Speech processor 100)
A speech processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings.

  FIG. 2 is a functional block diagram showing an example of the functional configuration of the speech processing apparatus 100 according to the first embodiment of the present invention. The speech processing apparatus 100 according to the present embodiment processes an acoustic signal including a target sound and a non-target voice so that the utterance content by the non-target voice speaker and / or the speaker of the non-target voice cannot be specified. It is.

  As shown in FIG. 2, the speech processing apparatus 100 includes an utterance interval detection unit (detection unit) 101 and an utterance interval processing unit (processing unit) 102.

(Speech section detection unit 101)
The utterance section detection unit 101 receives a first input signal that is an acoustic signal output from the microphone m1. The utterance section detection unit 101 performs utterance section detection for detecting a section (speaking section) in which a person speaks from the received first input signal. The utterance section detection unit 101 uses an existing voice detection technique for utterance section detection. As a voice detection technique, for example, among voice signals (acoustic signals), a technique for determining a section of a signal whose sound volume corresponding to the voice signal (sound signal) is a predetermined value or more as a speech section, For example, a technique for determining an utterance section by comparing a modeled in advance with a feature amount of an input signal may be used. Since the first input signal mainly includes the voice signal of the customer's voice, the utterance period detected by the utterance period detection unit 101 is a section including the voice signal of the customer's voice, that is, the non-target voice. Note that the utterance interval detected by the utterance interval detection unit 101 is not limited to one.

  The utterance section detection unit 101 specifies start time information indicating the start time of the utterance section in the first input signal and end time information indicating the end time of the utterance section as time information of the utterance section. Then, the utterance section detection unit 101 outputs the specified time information to the utterance section processing unit 102. When there are a plurality of detected utterance sections, the utterance section detection unit 101 specifies time information for all of them. Note that the time information output from the utterance section detection unit 101 is associated with information representing the first input signal that is the detection source of the time information.

(Speech section processing unit 102)
The speech segment processing unit 102 receives the time information output from the speech segment detection unit 101 and the second input signal that is an acoustic signal output from the microphone m2. The speech segment processing unit 102 compares the time information of the speech segment in the first input signal related to the second input signal with respect to the second input signal related to the first input signal associated with the time information. , A corresponding period in the second input signal corresponding to the time information is specified. The time information output from the utterance section detection unit 101 includes the start time information and the end time information of the utterance section as described above. The utterance section processing unit 102 corresponds to the time (referred to as the first time) corresponding to the start time represented by the start time information and the end time represented by the end time information in the second input signal. The time (referred to as the second time) is specified. That is, the utterance section processing unit 102 specifies a period (referred to as a time section) from the first time to the second time in the second input signal. Then, the utterance section processing unit 102 specifies a signal in the specified time section from the second input signal. When the time information output from the utterance section detection unit 101 includes time information for a plurality of utterance sections, the utterance section processing unit 102 specifies a corresponding time section for each of the plurality of time information, and the second Identify signals of multiple time intervals in the input signal.

  Since this time section is a period during which the customer speaks, it is the signal of this time section that includes the voice signal of the customer's voice in the second input signal. Therefore, the speech section processing unit 102 processes the signal of the specified time section in the second input signal. Note that the speech section processing unit 102 does not process signals other than the specified time section in the second input signal. That is, the utterance section processing unit 102 generates a signal obtained by processing only the signal in the specified time section with respect to the second input signal. Then, the utterance section processing unit 102 sets the generated signal as the utterance section processing signal. As a method of processing the acoustic signal performed by the speech section processing unit 102, a method in which the acoustic signal in the time section is replaced with a zero signal, the acoustic signal in the time section is cut out, and the acoustic signals before and after the cut portion are connected. The method of outputting things, etc. are mentioned.

  Note that the method of processing the acoustic signal performed by the speech section processing unit 102 is not limited to this. For example, the processing method of the acoustic signal performed by the speech section processing unit 102 may be a method of replacing the acoustic signal in the time section with noise such as white noise, or when a person listens to the acoustic signal in the time section. A method may be used in which a noise signal representing noise such as white noise having a volume at which the original utterance content cannot be specified is added. In this case, the volume of the noise depends on the volume of the sound corresponding to the first input signal in the utterance section detected by the utterance section detection unit 101 or the volume of the sound corresponding to the signal in the time section of the second input signal. The volume may be a volume obtained by multiplying the volume by a predetermined constant. Further, the speech segment processing unit 102 may perform voice quality conversion processing on the acoustic signal in the time segment of the second audio signal. Here, the voice quality conversion process is to convert the voice quality of the customer's voice represented by the acoustic signal into a voice quality of a third party different from the voice quality of the customer. Thereby, the utterance section processing unit 102 can process the acoustic signal in the time section so that the speaker cannot be specified even if the utterance content can be specified. Therefore, even if a third party converts the acoustic signal output from the utterance section processing unit 102 into speech and listens to the speech, the third party can identify the speaker, that is, the customer from the speech. Disappear. Therefore, the sound processing apparatus 100 of the present embodiment can output an acoustic signal that protects the privacy of the customer.

  An example of the speech segment processing signal when the speech segment processing unit 102 replaces the acoustic signal of the specified time segment in the second input signal with a zero signal will be described with reference to FIG. FIG. 3 is a diagram illustrating an image of the first input signal, the second input signal, and the speech segment processing signal. In FIG. 3, the first input signal is shown in the upper stage, the second input signal is shown in the center, and the speech segment processing signal is shown in the lower stage. The horizontal axis in each stage indicates time.

  In FIG. 3, t0 represents the start time of the utterance section, and t1 represents the end time of the utterance section. Therefore, the utterance section processing unit 102 specifies the period from t0 to t1 of the second input signal as a time section, and processes the acoustic signal in the specified time section. As described above, in the example of FIG. 3, the utterance section processing unit 102 replaces the acoustic signal in the time section with a zero signal, so that the utterance section processing signal is a zero signal during the period from t0 to t1 of the second speech signal. It becomes the replaced acoustic signal.

  Next, the operation of the speech processing apparatus 100 according to the present embodiment will be described using FIG. FIG. 4 is a flowchart showing an example of the operation of the speech processing apparatus 100 according to the present embodiment.

  As shown in FIG. 4, first, the utterance section detecting unit 101 detects the utterance section of the utterance by the customer from the first input signal (step S101). Then, the utterance section detection unit 101 specifies the start time information indicating the start time of the detected utterance section and the end time information indicating the end time of the detected utterance section as time information of the utterance section (step S102).

  Next, the utterance section processing unit 102 specifies a time section corresponding to the time information specified in step S102 in the second input signal (step S103). Then, the utterance section processing unit 102 processes the signal of the specified time section in the second input signal (step S104).

  Thus, the voice processing apparatus 100 ends the process.

  As described above, in the speech processing apparatus 100 according to the present embodiment, the utterance section detection unit 101 detects the utterance section of the utterance by the non-target voice speaker from the first input signal, and the non-target voice speaker is detected. The time information indicating the start time and end time of the utterance section of the utterance is identified. Then, the utterance section processing unit 102 selects at least one of the utterance content and the speaker based on the signal in the period corresponding to the specified time information among the second input signals including the target sound and the non-target speaker voice. The signal of the period in the second input signal is processed so that is not specified.

  Of the second input signal, the period represented by the time information specified by the utterance section detection unit 101 includes a non-target voice signal. A state in which the utterance content of the utterance of the non-target voice speaker and / or the non-target voice speaker cannot be specified from the sound corresponding to the signal of this period by the utterance section processing unit 102 processing the signal of this period become. Therefore, even when the non-target voice overlaps with the target sound, the voice processing device 100 cannot identify the utterance content by the non-target voice speaker and / or the non-target voice speaker from the acoustic signal including the target sound. Can be.

  The first input signal is an audio signal collected and output by the microphone m1 located closer to the non-target voice speaker (eg, customer) than the target sound source (eg, store clerk). Thereby, the utterance section of the utterance by the speaker of the non-target voice to be processed can be obtained more accurately. The second input signal is a voice signal that is collected and output by the microphone m2 that is closer to the sound source (for example, a clerk) of the target sound than the speaker (for example, the customer) of the non-target voice. Thereby, the utterance section processing unit 102 can more clearly include the clerk's voice as the target sound in the voice corresponding to the utterance section processing signal.

(Modification 1 of the first embodiment)
Next, Modification 1 of the present embodiment will be described with reference to FIG. FIG. 5 is a functional block diagram illustrating an example of a functional configuration of the speech processing apparatus 110 according to the first modification of the present embodiment. For convenience of explanation, members having the same functions as those included in the drawings described in the first embodiment are given the same reference numerals, and detailed descriptions thereof are omitted.

  As shown in FIG. 5, the speech processing apparatus 110 includes an utterance section detection unit 101, an utterance section processing unit 102, and an acoustic signal storage unit 103. That is, the sound processing apparatus 110 according to the first modification has a configuration in which the acoustic signal storage unit 103 is added to the sound processing apparatus 100 according to the first embodiment. In this modification, it is assumed that a person listens to the speech segment processing signal later.

  The utterance section processing unit 102 stores the utterance section processing signal in the acoustic signal storage unit 103.

  The acoustic signal storage unit 103 stores an utterance section processing signal. The acoustic signal storage unit 103 is realized by a storage device such as a hard disk drive, for example.

  For example, when a playback instruction or the like is transmitted from an external device of the speech processing device 110 to the speech segment processing signal stored in the acoustic signal storage unit 103, the speech processing device 110 outputs the speech segment processing signal as a recording signal. To do. The external device that has received the recording signal can convert the recording signal into sound and output it.

  As described above, since the speech processing apparatus 110 according to the present embodiment temporarily stores the utterance section processed signal in the acoustic signal storage unit 103, when the person wants to listen later, the utterance section processed signal is used as a recording signal. Can be output.

(Modification 2 of the first embodiment)
Next, a second modification of the present embodiment will be described. In the first embodiment described above, as an example of the first input signal input to the sound processing device 100, the acoustic signal from the microphone m1 has been described. However, the first input signal is not limited to an acoustic signal. In the second modification of the present embodiment, another example of the first input signal will be described. Note that the functional configuration of the speech processing apparatus 100 in this modification is the same as that in FIG.

  In the present modification, for example, a video signal acquired by an imaging device (sensor) such as a camera installed in a store is used as the first input signal. The utterance period detection unit 101 detects an utterance period from the video signal. Hereinafter, a method in which the utterance section detection unit 101 detects the utterance section from the video signal will be described.

  First, the utterance section detection unit 101 receives a video signal from a camera. The utterance section detection unit 101 determines a customer from the video represented by the received video signal. When the camera is a fixed camera, the positions of the payment counter and the POS register as shown in FIG. 1, for example, in the video represented by the received video signal are fixed. Further, as shown in FIG. 1, the position of the clerk is a position where the POS register can be operated, and the position of the customer is at a position opposite to the position of the clerk with the payment counter as the center. The utterance section detection unit 101 uses the information indicating these positions to determine a customer from the video represented by the received video signal. The information indicating the position may be stored in a storage unit (not shown) in the utterance section detection unit 101 or the voice processing device 100, or may be received from outside the voice processing device 100.

  The utterance section detection unit 101 uses the video representing the video signal to identify the mouth of the person determined to be a customer. Since a general image processing technique is applied to a method for identifying a part (in this case, the mouth) included in the face, detailed description thereof is omitted in this modification. Then, the utterance section detection unit 101 detects the movement of the mouth from the video representing the video signal, and detects from the time when the mouth starts to the time when the movement of the mouth stops as the utterance section. Then, the utterance section detection unit 101 sets the time when the mouth starts to move as the start time of the utterance section and the time when the movement of the mouth stops as the end time of the utterance section. Then, the utterance section detection unit 101 specifies start time information indicating the start time and end time information indicating the end time as time information of the utterance section.

  As described above, the audio processing device 100 according to the present modification can obtain the same effects as those of the audio processing device 100 according to the first embodiment described above even when the first input signal is a video signal. .

<Second Embodiment>
A speech recognition apparatus according to the second embodiment of the present invention will be described with reference to FIG. In the present embodiment, an example in which the speech processing apparatus 100 according to the first embodiment described above is applied to a speech recognition apparatus is shown. For convenience of explanation, members having the same functions as those included in the drawings described in the first embodiment are given the same reference numerals, and detailed descriptions thereof are omitted.

  FIG. 6 is a functional block diagram illustrating an example of a functional configuration of the speech recognition apparatus according to the present embodiment. As shown in FIG. 6, the speech recognition apparatus 600 according to the present embodiment includes an utterance section detection unit 101, an utterance section processing unit 102, and a speech recognition unit 601. The speech recognition apparatus 600 according to the present embodiment is an apparatus that includes therein the speech processing apparatus 100 according to the first embodiment described above.

  The voice recognition unit 601 receives the second input signal. The voice recognition unit 601 performs voice processing such as voice recognition based on the second input signal. Then, the voice recognition unit 601 generates a voice recognition result. In this embodiment, the speech recognition result generated by the speech recognition unit 601 is expressed by text.

  For example, the case where speech recognition is performed using the speech segment processing signal generated by the speech segment processing unit 102 will be described. As described above, the speech segment processing signal is a signal obtained by processing a part of the second input signal. Therefore, when speech recognition is performed on this speech segment processing signal, the target speech that is the target of speech recognition may be deleted. Therefore, the voice recognition accuracy of the target voice may be deteriorated.

  However, as described above, the speech recognition apparatus 600 according to the present embodiment can output a speech recognition result separately from the speech segment processing signal. Thereby, the speech recognition apparatus 600 has both a speech recognition result that is not deteriorated due to the processing of the acoustic signal and an utterance section processing signal in which the utterance content by the non-target speech speaker and / or the non-target speech speaker cannot be specified. Can be obtained. Speech recognition results expressed in text may include utterances by non-target speech speakers. However, since the speech recognition result is text, the speaker of the non-target speech cannot be specified from the speech recognition result. Therefore, the speech recognition apparatus 600 can protect the privacy of non-target speakers.

<Third Embodiment>
A speech processing apparatus according to a third embodiment of the present invention will be described with reference to the drawings.

  In the first embodiment described above, the first input signal is described as a signal output from the microphone m1, and the second input signal is described as a signal output from the microphone m2. However, the number of microphones may be plural. For example, a plurality of microphones may be provided on the payment counter and / or the POS terminal shown in FIG. In the present embodiment, a case where n (n is a natural number of 2 or more) microphones are used will be described.

(First input signal and second input signal)
First, the first input signal and the second input signal used in the speech processing apparatus according to this embodiment will be described. As described above, in this embodiment, sound is collected using a plurality of microphones (m1 to mn).

  In this embodiment, a signal in which one or a plurality of acoustic signals output from at least one of the plurality of microphones (m1 to mn) is combined is referred to as a first input signal and a second input signal. The first input signal and the second input signal may be signals obtained by collecting all the acoustic signals output from each of the n microphones (m1 to mn). Further, the first input signal and the second input signal may be signals obtained by collecting one or more selected acoustic signals. The first input signal and the second input signal may or may not include an acoustic signal output from the same microphone. However, the first input signal includes at least one acoustic signal output from a microphone capable of collecting the customer's voice (non-target voice), and the second input signal includes the clerk's voice (target sound). It is assumed that at least one acoustic signal output from a microphone that can collect sound is included.

  As described above, the first input signal includes the acoustic signal output from the microphone that can mainly collect the voice of the customer, and the second input signal includes the acoustic signal output from the microphone that can mainly collect the clerk's voice. . Thereby, the sound processing apparatus according to the present embodiment can process the signal of the customer in the second input signal, that is, the signal in the period including the non-target sound, with higher accuracy.

(Voice processing device 200)
FIG. 7 is a functional block diagram illustrating an example of a functional configuration of the voice processing device 200 according to the present embodiment. For convenience of explanation, members having the same functions as those included in the drawings described in the first embodiment are given the same reference numerals, and detailed descriptions thereof are omitted.

  As shown in FIG. 7, the speech processing apparatus 200 according to the present embodiment includes a speech signal generation unit (generation unit) 210, a speech segment detection unit 101, and a speech segment processing unit 102. The audio processing device 200 according to the present embodiment is configured to further include an audio signal generation unit 210 in the audio processing device 100 according to the first embodiment described above. The audio signal generation unit 210 includes a first audio signal generation unit 201 and a second audio signal generation unit 202.

(First audio signal generation unit 201)
The first audio signal generation unit 201 receives the first input signal input to the audio processing device 200. And the 1st audio | voice signal production | generation part 201 produces | generates the 1st audio | voice signal which is one acoustic signal mainly including the audio | voice signal of a customer's audio | voice from the received 1st input signal. As a method for generating the first audio signal, for example, an audio enhancement technique is applied from an acoustic signal output from one or more microphones (m1 to mn) included in the first input signal, and a non-target audio (customer's voice) is applied. A method of generating an acoustic signal in which (speech) is emphasized is conceivable. The speech enhancement technique may be a general one, and for example, beam forming may be used. In this case, the 1st audio | voice signal generation part 201 forms a beam with respect to a customer's direction with respect to the 1 or several acoustic signal contained in a 1st input signal. And the 1st audio | voice signal production | generation part 201 produces | generates the acoustic signal (1st audio | voice signal) by which the component of the customer's audio | voice was emphasized from one or several acoustic signals by forming a beam. Information indicating the direction of the customer may be stored in advance in a storage unit in the first audio signal generation unit 201. As described above, in this embodiment, a conversation scene as shown in FIG. 1 is assumed. Accordingly, the direction of the customer can be determined in advance with respect to the microphones (m1 to mn). Thus, the 1st audio | voice signal production | generation part 201 produces | generates the acoustic signal with which the customer's audio | voice was emphasized accurately by forming a beam toward the direction of the customer in which a customer is assumed to exist in a shop. Can do.

  Further, the method for generating the first audio signal is not limited to this. The first audio signal generation unit 201 outputs an acoustic signal output from a microphone assumed to be closest to the customer among one or more acoustic signals included in the first input signal, to a sound corresponding to the acoustic signal. The selected sound signal may be the first sound signal. In this case, the first sound signal generation unit 201 may perform noise suppression processing on the selected sound signal, and may use the sound signal whose noise is suppressed as the first sound signal.

  In addition, the first audio signal generation unit 201 may suppress ambient noise with respect to the first input signal using a noise suppression technique such as a noise canceller, and may use the suppressed signal as the first audio signal. The first audio signal generation unit 201 may apply both the noise suppression technique and the voice enhancement technique to the first input signal. When the first input signal is an acoustic signal output from one microphone, the first audio signal generation unit 201 applies a noise suppression technique for one input such as a spectral subtraction method to the first input signal. Also good. Thus, the 1st audio | voice signal production | generation part 201 can produce | generate a 1st audio | voice signal by emphasizing the customer's audio | voice which is non-purpose audio | voice, or suppressing other than a customer's audio | voice.

  Then, the first audio signal generation unit 201 supplies the generated first audio signal to the utterance section detection unit 101.

(Second audio signal generation unit 202)
The second audio signal generation unit 202 receives the second input signal input to the audio processing device 200. And the 2nd audio | voice signal production | generation part 202 produces | generates the 2nd audio | voice signal which is one acoustic signal mainly including a store clerk's audio | voice from the received 2nd input signal. As a method for generating the second sound signal, a sound enhancement technique such as beam forming may be applied in the same manner as the method for generating the first sound signal to generate an acoustic signal in which the target sound (store clerk's sound) is emphasized. . In this case, the second audio signal generation unit 202 forms a beam in the direction of the store clerk for one or more acoustic signals included in the second input signal. And the 2nd audio | voice signal production | generation part 202 produces | generates the acoustic signal (2nd audio | voice signal) by which the component of the salesclerk's audio | voice was emphasized from one or several acoustic signals by forming a beam. Information indicating the direction of the store clerk may be stored in advance in the storage unit in the second audio signal generation unit 202. As described above, in this embodiment, a conversation scene as shown in FIG. 1 is assumed. Therefore, the direction of the store clerk can be determined in advance with respect to the microphones (m1 to mn).

  Further, the method of generating the second audio signal is not limited to this. The second audio signal generation unit 202 outputs an acoustic signal output from a microphone assumed to be closest to the store clerk among one or more acoustic signals included in the second input signal, as a sound corresponding to the acoustic signal. The selected signal may be the second audio signal. In this case, the second sound signal generation unit 202 may perform noise suppression processing on the selected sound signal and use the sound signal in which noise is suppressed as the second sound signal.

  Similarly to the first audio signal generation unit 201, the second audio signal generation unit 202 may generate a second audio signal using a noise suppression technique such as a noise canceller.

  Further, the first audio signal generation method and the second audio signal generation method may be the same or different. For example, a method using a speech enhancement technique may be employed as the first sound signal generation method, and a method using a noise suppression technique may be employed as the second sound signal generation method.

  By such a method, the second audio signal generation unit 202 can generate the second audio signal in which the clerk's voice, which is the target sound, is emphasized or other than the clerk's voice is suppressed.

  The speech segment detection unit 101 in the speech processing apparatus 200 according to the present embodiment receives the first speech signal output from the first speech signal generation unit 201 as an input, and the speech processing apparatus 100 according to the first embodiment described above. Processing similar to that performed by the utterance section detection unit 101 is performed. Further, the speech segment processing unit 102 in the speech processing apparatus 200 according to the present embodiment inputs the second speech signal output from the second speech signal generation unit 202 and the time information output from the speech segment detection unit 101. And Then, the utterance section processing unit 102 performs the same processing as the utterance section processing unit 102 of the speech processing apparatus 100 according to the first embodiment described above.

  Next, the operation of the speech processing apparatus 200 according to the present embodiment will be described using FIG. FIG. 8 is a flowchart showing an example of the operation of the speech processing apparatus 200 according to the present embodiment.

  As shown in FIG. 8, first, the first audio signal generation unit 201 of the audio signal generation unit 210 generates a first audio signal from the first input signal (step S201). Further, the second audio signal generation unit 202 of the audio signal generation unit 210 generates a second audio signal from the second input signal (step S202). Note that step S201 and step S202 may be performed simultaneously or in reverse order.

  Thereafter, the utterance section detection unit 101 detects the utterance section of the utterance by the customer from the first voice signal (step S203). Then, the utterance section detection unit 101 specifies the start time information indicating the start time of the detected utterance section and the end time information indicating the end time of the detected utterance section as time information of the utterance section (step S204).

  Next, the speech segment processing unit 102 identifies a time segment corresponding to the time information identified in Step S204 in the second audio signal (Step S205). Then, the utterance section processing unit 102 processes the signal of the specified time section in the second voice signal (step S206).

  Thus, the voice processing device 200 ends the process.

  According to the sound processing apparatus 200 according to the present embodiment, in addition to the effects of the first embodiment described above, it is possible to specify non-target sound with high accuracy. This is because the first sound signal generation unit 201 generates a first acoustic signal that is a sound signal mainly including the non-target sound from the first input signal. As a result, the speech segment detected by the speech segment detection unit 101 from the first acoustic signal includes the acoustic signal corresponding to the non-target speech with higher accuracy. Therefore, the utterance section processing unit 102 can perform processing on a signal in a section in which non-target speech is included with higher accuracy.

  Therefore, the speech corresponding to the speech segment processing signal processed by the speech segment processing unit 102 includes the target speech more easily. When speech recognition processing or the like is performed on such a speech segment processing signal, the accuracy of speech recognition becomes higher.

  The second audio signal may be plural. For example, when there are a plurality of salesclerks in a conversation scene between a salesclerk and a customer, the second audio signal generation unit 202 may generate a second audio signal for each salesclerk. In this case, the utterance section processing unit 102 may specify a time section for each of the plurality of second audio signals, and may process the signal of the specified time section. Thereby, the utterance section processing unit 102 can generate an utterance section processing signal for each of a plurality of salesclerks.

<Fourth Embodiment>
A speech processing apparatus according to a fourth embodiment of the present invention will be described with reference to the drawings. First, an input signal input to the speech processing apparatus according to this embodiment will be described. In the present embodiment, as in the third embodiment, the first input signal and the second input signal in which one or a plurality of acoustic signals output from at least one of the plurality of microphones (m1 to mn) are combined are audio. Input to the processing unit.

  Further, the third input signal is further input to the speech processing apparatus according to the present embodiment. This third input signal is a sensor signal. The sensor signal is, for example, a video signal output from a camera that captures a video near the POS register.

(Speech processor 300)
FIG. 9 is a functional block diagram illustrating an example of a functional configuration of the voice processing device 300 according to the present embodiment. For convenience of explanation, members having the same functions as those included in the drawings described in the above-described embodiments are given the same reference numerals, and detailed descriptions thereof are omitted.

  As shown in FIG. 9, the speech processing apparatus 300 according to the present embodiment includes a signal selection unit (selection unit) 301, a speech signal generation unit 210, a speech segment detection unit 101, and a speech segment processing unit 102. ing. The speech processing apparatus 300 according to the present embodiment is configured to further include a signal selection unit 301 in the speech processing apparatus 200 described above. The audio signal generation unit 210 includes a first audio signal generation unit 201 and a second audio signal generation unit 202.

(Signal selection unit 301)
The signal selection unit 301 receives the first input signal, the second input signal, and the third input signal. And the signal selection part 301 estimates the position where a speaker (a salesclerk, a customer) exists using a 3rd input signal. First, the signal selection unit 301 identifies a speaker from the video represented by the third input signal. Then, the signal selection unit 301 determines whether the identified speaker is a store clerk or a customer. At this time, the signal selection unit 301 determines the store clerk using, for example, the store clerk uniform information and the store clerk face information stored in the signal selection unit 301 or in a storage unit (not shown). Then, the signal selection unit 301 determines the remaining speakers as customers. Since the salesclerk discrimination method performed by the signal selection unit 301 uses a general person detection technique or the like, detailed description is omitted in this embodiment.

  And the signal selection part 301 estimates the position of the speaker who discriminate | determined as a salesclerk from the 3rd input signal. Further, the signal selection unit 301 estimates the position of the speaker determined as the customer from the third input signal.

  Then, the signal selection unit 301 selects one or a plurality of acoustic signals from the first input signal based on the estimated customer position. Further, the signal selection unit 301 selects one or a plurality of acoustic signals from the second input signal based on the estimated position of the store clerk.

  Hereinafter, the one or more acoustic signals selected from the first input signal by the signal selection unit 301 are referred to as first selection signals, and the one or more acoustic signals selected from the second input signals are referred to as second selection signals.

  The selection method of the first selection signal by the signal selection unit 301 includes, for example, preliminarily storing position information indicating the positions of a plurality of microphones (m1 to mn), and based on the position information, the estimated position of the customer is stored. A method of selecting a predetermined number of acoustic signals output from a close microphone may be used.

  Similarly, the selection method of the second selection signal by the signal selection unit 301 is, for example, preliminarily holding position information indicating the positions of the plurality of microphones (m1 to mn) and estimating the salesclerk based on the position information. A method of selecting a predetermined number of acoustic signals output from a microphone close to the position may be used.

  The signal selection unit 301 outputs the first selection signal to the first audio signal generation unit 201. Further, the signal selection unit 301 outputs the second selection signal to the second audio signal generation unit 202.

  The example in which the video signal from one camera is used as the third input signal has been described above, but the third input signal may be a video signal output from each of a plurality of cameras. For example, the video signal used for estimating the position of the store clerk may be different from the video signal used for estimating the position of the customer. Further, the third input signal is not limited to the video signal from the camera as long as the position of the store clerk or the customer can be estimated. For example, the audio processing device 300 may use a signal from a position estimation system using a wireless tag or the like as the third input signal. The audio processing device 300 may use a signal from the position estimation system and a video signal from the camera in combination as the third input signal.

  The first audio signal generation unit 201 of the audio signal generation unit 210 in the audio processing device 300 according to the present embodiment receives the first selection signal output from the signal selection unit 301 as an input, and relates to the above-described third embodiment. The same processing as the first audio signal generation unit 201 of the audio processing device 200 is performed. Further, the second audio signal generation unit 202 of the audio signal generation unit 210 in the audio processing device 300 according to the present embodiment receives the second selection signal, and the second audio signal generation unit 202 of the audio processing device 200 according to the third embodiment described above. The same processing as that of the 2 audio signal generation unit 202 is performed.

  Further, the utterance section detection unit 101 and the utterance section processing unit 102 perform the same processes as the utterance section detection unit 101 and the utterance section processing unit 102 in the third embodiment described above, respectively.

  Next, the operation of the speech processing apparatus 300 according to the present embodiment will be described using FIG. FIG. 10 is a flowchart showing an example of the operation of the speech processing apparatus 300 according to this embodiment.

  As shown in FIG. 10, first, the signal selection unit 301 selects a first selection signal from the first input signal (step S301). Further, the signal selection unit 301 selects the second selection signal from the second input signal (step S302). Note that step S301 and step S302 may be performed simultaneously or in reverse order.

  Next, the first audio signal generation unit 201 of the audio signal generation unit 210 generates a first audio signal from the first selection signal (step S303). Also, the second audio signal generation unit 202 of the audio signal generation unit 210 generates a second audio signal from the second selection signal (step S304). Note that step S303 and step S304 may be performed simultaneously or in the reverse order.

  Thereafter, the utterance section detection unit 101 detects the utterance section of the utterance by the customer from the first voice signal (step S305). Then, the utterance section detecting unit 101 specifies the start time information indicating the start time of the detected utterance section and the end time information indicating the end time of the detected utterance section as time information of the utterance section (step S306).

  Next, the speech segment processing unit 102 identifies a time segment corresponding to the time information identified in step S306 in the second audio signal (step S307). Then, the utterance section processing unit 102 processes the signal of the specified time section in the second audio signal (step S308).

  Thus, the voice processing device 300 ends the process.

  According to the speech processing apparatus 300 according to the present embodiment, the same effects as those in the first embodiment described above can be achieved. In addition, the voice processing device 300 according to the present embodiment can process the signal of the section including the voice of the customer with high accuracy even if the position of the customer or the store clerk varies. This is because the signal selection unit 301 selects one or more acoustic signals from the first input signal based on the third input signal, and selects one or more acoustic signals from the second input signal based on the third input signal. Because it does. Thereby, even if the position of the customer or the store clerk moves, the signal selection unit 301 selects an acoustic signal mainly including a signal corresponding to the voice of the customer from the first input signal according to the moved position, An acoustic signal mainly including a signal corresponding to the clerk's voice can be selected from the second input signal. And the audio | voice signal production | generation part 210 can produce | generate a 1st acoustic signal and a 2nd acoustic signal from the selected acoustic signal.

  In addition, since the signal selection unit 301 selects an acoustic signal from the first input signal and the second input signal, the number of acoustic signals processed by the audio signal generation unit 210 can be reduced. Therefore, the speech processing apparatus 300 according to the present embodiment can reduce the processing amount as compared with the speech processing apparatus 200.

  In the above description, it is assumed that the conversation between the store clerk and the customer at the payment counter is collected by the microphone installed in the POS register. However, the installation location of the microphone is not limited to this. For example, a scene in which conversation between a store clerk and a customer at an arbitrary location on a store floor is collected by a plurality of microphones installed at various locations such as a ceiling or a shelf in the store can be considered. In such a case, it is desirable to use video signals from a plurality of cameras installed at different locations in the floor as the third input signal in the present embodiment. In this case, the signal selection unit 301 detects a customer and a store clerk and estimates a position from the image of the video signal of each camera. And the signal selection part 301 may select the acoustic signal output from the microphone arrange | positioned within the predetermined distance from the estimated customer's position among the first input signals as the first selection signal. Similarly, the signal selection unit 301 may select, as the second selection signal, an acoustic signal output from a microphone arranged within a predetermined distance from the estimated position of the store clerk among the second input signals. Further, the signal selection unit 301 selects the first selection signal when the position estimated for the detected customer is more than a predetermined distance from the estimated position for the detected store clerk. May be excluded from the acoustic signal. When the signal selection unit 301 detects a plurality of store clerk, the signal selection unit 301 may select the second selection signal for each detected store clerk. In this case, the processing of the voice signal generation unit 210 and the utterance section detection unit 101 is also performed for each clerk detected by the signal selection unit 301, and the utterance section processing unit 102 can generate the utterance section processing signal for each clerk.

(Modification)
In the present embodiment, the third input signal is described as a signal different from the first input signal and the second input signal. However, the third input signal is a signal similar to the first input signal and the second input signal. May be.

  That is, the first input signal and the second input signal may be used as the third input signal. For example, the signal selection unit 301 compares the sound volume corresponding to each of the one or more sound signals included in the first input signal, selects a predetermined number of sound signals corresponding to the sound having a high sound volume, and It may be a selection signal. Further, when comparing the volume, the signal selection unit 301 applies a noise suppression technique, a noise suppression technique, or the like to each of one or a plurality of acoustic signals included in the first input signal, so that the signals other than the non-target voices are applied. Voice and noise may be removed. In addition, the signal selection unit 301 may apply a sound source separation technique that separates a sound source and extracts only the sound to be extracted for each of one or a plurality of acoustic signals included in the first input signal. Thereby, the signal selection part 301 can select a 1st selection signal from the acoustic signal corresponding to a non-target audio | voice. Similarly, the signal selection unit 301 compares the sound volume corresponding to each of the one or more sound signals included in the second input signal, selects a predetermined number of sound signals corresponding to the sound having a high sound volume, Two selection signals may be used.

<Fifth Embodiment>
A fifth embodiment of the present invention will be described with reference to the drawings. FIG. 11 is a functional block diagram illustrating an example of a functional configuration of the voice processing device 400 according to the present embodiment. For convenience of explanation, members having the same functions as those included in the drawings described in the above-described embodiments are given the same reference numerals, and detailed descriptions thereof are omitted. As shown in FIG. 11, the speech processing apparatus 400 according to the present embodiment includes an utterance section detection unit 101, an utterance section processing unit 102, an audio signal generation unit 410, and a position estimation unit (estimation unit) 420. The audio signal generation unit 410 includes a first audio signal generation unit 401 and a second audio signal generation unit 402. The position estimation unit 420 includes a first speaker position estimation unit 403 and a second speaker position estimation unit 404.

  The speech processing apparatus 400 according to the present embodiment includes a position estimation unit 420 instead of the signal selection unit 301 of the speech processing apparatus 300 according to the fourth embodiment, and replaces the speech signal generation unit 210 with a speech signal generation unit 410. It is the structure provided with. Note that the first input signal, the second input signal, and the third input signal that are input to the speech processing apparatus 400 according to the present embodiment are the same as those in the fourth embodiment.

  The first speaker position estimation unit 403 of the position estimation unit 420 estimates the position of the customer using the first input signal and / or the third input signal. The customer position estimation method performed by the first speaker position estimation unit 403 using the first input signal or the third input signal is the same method as the customer position estimation method performed by the signal selection unit 301 described above. Therefore, detailed description is omitted. Further, when the first speaker position estimation unit 403 estimates the position of the customer using the first input signal, if the first input signal includes a plurality of acoustic signals, the first speaker position estimation unit 403 A technique for estimating a general sound source may be applied to identify the sound source and estimate the position of the customer. Further, when the first speaker position estimation unit 403 estimates the position of the customer using the first input signal and the third input signal, the sound source candidate is identified from the first input signal, and the third input signal is used. Alternatively, a customer may be selected from the sound source candidates and the position of the selected customer may be estimated.

  The first speaker position estimation unit 403 outputs first position information indicating the estimated customer position (hereinafter referred to as a first speaker position) to the first audio signal generation unit 401. For example, when the first position information is estimated using the third input signal, the first position information may be information (for example, xy coordinates) indicating the position on the video represented by the third input signal. The information (for example, latitude, longitude, etc.) which shows the position which converted the position into the position in the real world may be sufficient. The first position information may be information indicating a relative position from a microphone installed in the store. In this case, information indicating the position of the microphone may be stored in the storage unit in the first speaker position estimation unit 403.

  The second speaker position estimation unit 404 of the position estimation unit 420 estimates the position of the store clerk using the second input signal and / or the third input signal. The method of estimating the position of the clerk performed by the second speaker position estimating unit 404 is the same as the method used when the first speaker position estimating unit 403 estimates the position of the customer. The second speaker position estimation unit 404 outputs second position information indicating the estimated position of the store clerk (hereinafter referred to as a second speaker position) to the second audio signal generation unit 402.

  Note that the first speaker position estimation unit 403 and the second speaker position estimation unit 404 may estimate the position of the customer and the clerk by the same estimation method, or the position of the customer and the clerk by different estimation methods. It may be estimated.

  The first audio signal generation unit 401 of the audio signal generation unit 410 receives the first input signal and the first position information. The first audio signal generation unit 401 applies an audio enhancement technique such as beam forming to the first input signal, and generates a first audio signal that is an acoustic signal in which the customer's voice is emphasized. The first audio signal generation unit 401 determines the direction of the beam to be formed based on the first position information when performing beamforming.

  Then, the first audio signal generation unit 401 outputs the generated first audio signal to the utterance section detection unit 101.

  The second audio signal generation unit 402 of the audio signal generation unit 410 receives the second input signal and the second position information. The second sound signal generation unit 402 applies a sound enhancement technique such as beam forming to the second input signal, and generates a second sound signal that is an acoustic signal in which the clerk's sound is emphasized. The second audio signal generation unit 402 determines the direction of the beam to be formed based on the second position information when performing beamforming.

  Then, the second voice signal generation unit 402 outputs the generated second voice signal to the utterance section processing unit 102.

  Further, the utterance section detection unit 101 and the utterance section processing unit 102 perform the same processes as the utterance section detection unit 101 and the utterance section processing unit 102 in the third embodiment described above, respectively.

  Next, the operation of the speech processing apparatus 400 according to the present embodiment will be described using FIG. FIG. 12 is a flowchart showing an example of the operation of the speech processing apparatus 400 according to this embodiment.

  As shown in FIG. 12, first, the position estimation unit 420 estimates the first speaker position and the second speaker position. Specifically, the first speaker position estimation unit 403 of the position estimation unit 420 estimates the first speaker position from the third input signal and / or the first input signal, and the second speaker position estimation unit 404 A second speaker position is estimated from the third input signal and / or the second input signal (step S401).

  Next, the first audio signal generation unit 401 of the audio signal generation unit 410 generates a first audio signal from the first input signal based on the first speaker position (step S402). Further, the second audio signal generation unit 402 of the audio signal generation unit 410 generates a second audio signal from the second input signal based on the second speaker position (step S403). Note that step S402 and step S403 may be performed simultaneously or in reverse order.

  Thereafter, the utterance section detecting unit 101 detects the utterance section of the utterance by the customer from the first voice signal (step S404). Then, the utterance section detection unit 101 specifies the start time information indicating the start time of the detected utterance section and the end time information indicating the end time of the detected utterance section as time information of the utterance section (step S405).

  Next, the utterance section processing unit 102 specifies a time section corresponding to the time information specified in step S405 in the second audio signal (step S406). Then, the utterance section processing unit 102 processes the signal of the specified time section in the second audio signal (step S407).

  Thus, the voice processing device 400 ends the process.

  In addition, although the audio | voice processing apparatus 400 which concerns on this embodiment demonstrated the structure provided with the position estimation part 420 instead of the signal selection part 301, the audio | voice processing apparatus 400 is provided with the signal selection part 301 and the position estimation part 420. It may be.

  As described above, according to the voice processing device 400 according to the present embodiment, the position estimation unit 420 estimates the position of the customer and the salesclerk, and the voice signal generation unit 410 determines the estimated customer position and the salesclerk's position. Based on the position, a first audio signal and a second audio signal are generated. The first audio signal and the second audio signal thus generated are signals in which the voice of the customer or the store clerk is further emphasized. As a result, the utterance section processing unit 102 can process the period in which the customer's voice is included more accurately.

<Sixth Embodiment>
A sixth embodiment of the present invention will be described with reference to the drawings. In this embodiment, it is assumed that the target sound is the voice of a specific speaker, and the speaker and the voice are hereinafter referred to as “target speaker” and “target voice”, respectively. Note that the target speaker may include a plurality of speakers.

  FIG. 13 is a functional block diagram illustrating an example of a functional configuration of the voice processing device 500 according to the present embodiment. For convenience of explanation, members having the same functions as those included in the drawings described in the above-described embodiments are given the same reference numerals, and detailed descriptions thereof are omitted.

  The speech processing apparatus 500 according to the present embodiment includes a speech signal generation unit 210, a speech segment detection unit 101, a target speaker speech segment detection unit 501, a speech segment processing unit 502, and a storage unit 503.

  In the present embodiment, the second audio signal generated by the second audio signal generation unit 202 of the audio signal generation unit 210 is input to the target speaker utterance interval detection unit 501 and the utterance interval processing unit 502.

(Storage unit 503)
The storage unit 503 stores the feature amount of the acoustic signal of the target speaker's voice. The feature amount may be expressed by, for example, a mixed Gaussian distribution model (GMM), or may be expressed by other models. For example, the feature amount may be a feature amount used in general speaker recognition technology.

  Further, for example, when the target speaker is a store clerk, the storage unit 503 stores the feature amount of the sound signal of the voice of all the store clerk in the store.

(Target speaker utterance section detector 501)
The target speaker utterance section detection unit 501 receives the second audio signal from the second audio signal generation unit 202. Then, the target speaker utterance section detector 501 detects the utterance section of the target speaker from the received second voice signal. Since the second audio signal is the same as the second audio signal in the second embodiment, the second audio signal includes audio signals of one or a plurality of speakers including the target speaker.

  Specifically, the target speaker utterance section detection unit 501 converts the second audio signal into a time-series feature value, and stores the converted feature value (referred to as input feature value) and the storage unit 503. Check against feature quantity. The target speaker utterance section detection unit 501 may perform feature amount collation at regular intervals, for example, in units of 10 milliseconds. In addition, the target speaker utterance section detection unit 501 detects the utterance section of the second voice signal by applying a voice detection technique to the second voice signal, and performs feature amount collation for each detected utterance section. Alternatively, it may be determined whether or not each utterance section is the utterance of the target speaker.

  For example, the target speaker utterance section detection unit 501 collates the input feature quantity with the feature quantity stored in the storage unit 503, and obtains the likelihood of the stored feature quantity with respect to the input feature quantity. Then, the target speaker utterance section detecting unit 501 compares the likelihood with a predetermined threshold, and the section in which the target speaker utters the section of the second speech signal having the input feature amount having the likelihood higher than the predetermined threshold. It is determined that

  Then, the target speaker utterance section detection unit 501 identifies section information that is time information of a section in which the target speaker is speaking based on the determination result, and outputs the section information to the utterance section processing unit 502. . The section information includes start end time information indicating the start time of the section and end time information indicating the end time. When there are a plurality of uttered sections, the target speaker utterance section detecting unit 501 outputs start time information and end time information of all the sections as section information.

<Speech section processing unit 502>
The utterance section processing unit 502 receives section information from the target speaker utterance section detection unit 501. Further, the utterance section processing unit 502 receives the second audio signal from the second audio signal generation unit 202. Further, the utterance section processing unit 502 receives time information from the utterance section detection unit 101. Based on the received section information, the utterance section processing unit 502 extracts a signal of a section expressed by the section information from the second audio signal. Specifically, the utterance section processing unit 502 generates a target speaker utterance section speech signal that leaves only the section signal represented by the section information in the second speech signal, based on the received section information. Then, the utterance section processing unit 502 performs the same processing as the utterance section processing unit 102 in each embodiment described above on the target speaker utterance section voice signal, and generates a utterance section processing signal.

  Next, the operation of the speech processing apparatus 500 according to the present embodiment will be described using FIG. FIG. 14 is a flowchart showing an example of the operation of the speech processing apparatus 500 according to this embodiment.

  As shown in FIG. 15, first, the first audio signal generation unit 201 of the audio signal generation unit 210 generates a first audio signal from the first input signal (step S501). Further, the second audio signal generation unit 202 of the audio signal generation unit 210 generates a second audio signal from the second input signal (step S502). Note that step S501 and step S502 may be performed simultaneously or in the reverse order.

  Thereafter, the utterance section detection unit 101 detects the utterance section of the utterance by the speaker of the non-target voice from the first voice signal (step S503). Then, the utterance section detection unit 101 specifies start time information indicating the start time of the detected utterance section and end time information indicating the end time of the detected utterance section as time information of the utterance section of the non-target speech ( Step S504).

  Next, the target speaker utterance section detection unit 501 detects the utterance section of the target speaker from the second audio signal (step S505). Then, the target speaker utterance section detection unit 501 identifies section information that is time information of the target speaker's utterance section (step S506). Note that step S503 and step S504 and step S505 and step S506 may be performed at the same time, or step S503 and step S504 may be performed after step S506 ends.

  Next, the utterance section processing unit 502 extracts a signal in the time section corresponding to the time information specified in step S506 from the second voice signal (step S507). Then, the speech segment processing unit 502 identifies the time segment corresponding to the time information identified in Step S503 from the signal extracted in Step S507 (Step S508). Then, the utterance section processing unit 502 processes the signal of the time section specified in step S508 in the extracted signal (step S509).

  Thus, the voice processing device 500 ends the process.

  According to the speech processing apparatus 500 according to the present embodiment, the same effects as those in the first embodiment described above can be achieved. Furthermore, speech processing apparatus 500 according to the present embodiment specifies a time interval represented by time information of the speech interval of the target speech using the feature amount of the acoustic signal of the target speaker's speech. Then, the audio processing device 500 extracts only the audio signal in the specified time interval from the second audio signal. Then, the speech processing apparatus 500 processes a signal in a period corresponding to the speech section of the non-target speech among the extracted speech signals. Thereby, the audio processing device 500 can further reduce the possibility that a signal corresponding to the non-purpose audio leaks from the processing target and remains.

  The method by which the utterance section processing unit 502 generates the utterance section processing signal is not limited to the above. For example, the utterance section processing unit 502 specifies a set of sections other than the section expressed by the received section information from the second audio signal. Then, the utterance section processing unit 502 obtains a union of the identified set and the section set represented by the received time information. Then, the utterance section processing unit 502 may perform the same processing as the utterance section processing unit 102 on the period included in the union of the second audio signal.

<Seventh Embodiment>
Next, a seventh embodiment of the present invention will be described. In this embodiment, a minimum configuration that solves the problems of the present invention will be described.

  FIG. 15 is a functional block diagram illustrating an example of a functional configuration of the voice processing device 10 according to the present embodiment. As shown in FIG. 15, the speech processing apparatus 10 according to the present embodiment includes a detection unit 11 and a processing unit 12.

  The detection unit 11 corresponds to the utterance section detection unit 101 in each embodiment. The detection unit 11 detects an utterance section of an utterance by a non-target voice speaker from a first input signal that is a sensor signal. And the detection part 11 specifies the time information showing the start time and end time of an utterance area. The detection unit 11 supplies the specified time information to the processing unit 12.

  The processing unit 12 corresponds to the utterance section processing unit (102, 502) in each embodiment. The processing unit 12 receives time information from the detection unit 11. The processing unit 12 also receives a second input signal that is an acoustic signal including the target sound and the non-target speaker's voice. The processing unit 12 specifies a signal in a period corresponding to time information among the second input signals. And the process part 12 processes the signal of the period in a 2nd input signal so that at least any one of utterance content and a speaker may not be specified by the signal of the specified period. The signal processing method performed by the processing unit 12 is not particularly limited, and the signal in the period may be deleted, or the signal in the period may be replaced with a zero signal. The processing unit 12 may process the signal of the period so that the utterance content and / or the speaker is not specified by the signal of the specified period, and the method is not particularly limited.

  The period of time specified by the detection unit 11 includes a non-target audio signal. Therefore, for example, when the processing unit 12 replaces the signal of the above period with a zero signal, the signal processed by the processing unit 12 does not include non-target sound. Therefore, the utterance content and the speaker by the non-target speaker cannot be specified from the processed signal. Further, when the voice quality conversion process is performed on the signal in the above period, the speaker cannot be specified from the processed signal. In this manner, the processing unit 12 can output an acoustic signal in which the utterance content of a non-target voice speaker and / or a non-target voice speaker is processed. Therefore, even when the non-target voice overlaps with the target sound, the voice processing device 10 cannot specify the utterance content by the non-target voice speaker and / or the non-target voice speaker from the acoustic signal including the target sound. Can be.

(About hardware configuration)
In each embodiment of the present invention, each component of each device represents a functional unit block. A part or all of each component of each device is realized by an arbitrary combination of an information processing device 900 and a program as shown in FIG. 16, for example. The information processing apparatus 900 includes the following configuration as an example.

CPU (Central Processing Unit) 901
ROM (Read Only Memory) 902
-RAM (Random Access Memory) 903
A program 904 loaded into the RAM 903
A storage device 905 that stores the program 904
A drive device 907 that reads / writes data from / to the recording medium 906
A communication interface 908 connected to the communication network 909
・ Input / output interface 910 for inputting / outputting data
-Bus 911 connecting each component
Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. The program 904 that realizes the function of each component of each device is stored in advance in the storage device 905 or the RAM 903, for example, and is read by the CPU 901 as necessary. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in the recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.

  There are various modifications to the method of realizing each device. For example, each device may be realized by an arbitrary combination of an information processing device 900 and a program that are different for each component. A plurality of constituent elements included in each device may be realized by an arbitrary combination of one information processing device 900 and a program.

  In addition, some or all of the components of each device are realized by other general-purpose or dedicated circuits, processors, or combinations thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus.

  Part or all of each component of each device may be realized by a combination of the above-described circuit and the like and a program.

  When some or all of the constituent elements of each device are realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributedly arranged. Also good. For example, the information processing apparatus, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system and a cloud computing system.

  Each of the above-described embodiments is a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention only to the above-described each embodiment. It is possible to construct a form in which various modifications are made by modifying or substituting the embodiment.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

  (Supplementary Note 1) Detection means for detecting a speech section of a speech by a non-target speech speaker from a first input signal that is a sensor signal, and identifying time information indicating a start time and an end time of the speech section; Of the second input signal that is an acoustic signal including sound and the voice of the non-target speaker, at least one of the utterance content and the speaker is not specified by a signal in a period corresponding to the specified time information And processing means for processing the signal of the period in the second input signal.

(Appendix 2)
The first input signal is an audio signal mainly including the non-purpose audio,
The audio processing apparatus according to appendix 1, wherein the second input signal is an audio signal mainly including the target sound.

(Appendix 3)
The first input signal is an acoustic signal that is collected by a first microphone provided at a position closer to a speaker of the non-target sound than the sound source of the target sound and output from the first microphone;
The second input signal is collected from a speaker of the non-target voice by a second microphone different from the first microphone provided at a position close to the sound source of the target sound, and output from the second microphone. The audio processing device according to attachment 2, wherein the audio processing device is an acoustic signal that has been recorded.

(Appendix 4)
Each of the first input signal and the second input signal includes one or a plurality of acoustic signals output from at least one of a plurality of microphones, and the sound processing device receives the non-purpose from the first input signal. Generating means for generating a first acoustic signal that is an audio signal mainly including sound, and generating a second acoustic signal that is an audio signal mainly including the target sound from the second input signal; Any one of appendices 1 to 3, wherein the detecting means specifies the time information from the first acoustic signal, and the processing means processes the signal of the period in the second acoustic signal. The voice processing apparatus according to 1.

(Appendix 5)
Based on the third input signal that is a sensor signal and the first input signal, one or more acoustic signals are selected as the first selection signal from the first input signal, and the third input signal and the second input signal are selected. Based on the second input signal, further comprising selection means for selecting one or a plurality of acoustic signals as a second selection signal, the generation means generates the first acoustic signal from the first selection signal, The speech processing apparatus according to appendix 4, wherein the second acoustic signal is generated from the second selection signal.

(Appendix 6)
The apparatus further comprises estimation means for estimating the position of the sound source of the target sound and the position of the speaker of the non-target voice from a third input signal which is a sensor signal, and the generation means includes a speaker of the estimated non-target voice Generating the first acoustic signal from the first input signal based on the position of the second input signal, and generating the second acoustic signal from the second input signal based on the estimated position of the sound source of the target sound. The speech processing apparatus according to supplementary note 4, wherein

(Appendix 7)
The first input signal is a video signal representing a video of the non-target voice speaker, and the detection means determines the non-target voice speaker from the video represented by the video signal. The speech processing apparatus according to appendix 1, wherein an utterance section of the utterance by the speaker is detected using the determined speaker image.

(Appendix 8)
The processing means replaces a signal in a period corresponding to the specified time information in the second input signal with a zero signal or a predetermined noise signal. The voice processing apparatus according to 1.

(Appendix 9)
The processing means includes at least a volume of the utterance section of the sensor signal and a volume of the period of the second input signal in a signal corresponding to the specified time information in the second input signal. The sound processing apparatus according to any one of appendices 1 to 7, wherein a noise with a volume at which the content of the sound during the period cannot be specified is added according to any of the above.

(Appendix 10)
The voice according to any one of appendices 1 to 7, wherein the processing means performs voice quality conversion processing on a signal in a period corresponding to the specified time information in the second input signal. Processing equipment.

(Appendix 11)
An utterance section of an utterance by a non-target speech speaker is detected from a first input signal that is a sensor signal, time information indicating a start time and an end time of the utterance section is specified, and the target sound and the non-target speech The second input signal so that at least one of the utterance contents and the speaker is not specified by the signal in the period corresponding to the specified time information among the second input signals which are acoustic signals including the voice of the speaker. A voice processing method, wherein the signal in the period is processed.

(Appendix 12)
A process of detecting an utterance section of an utterance by a speaker of a non-target voice from a first input signal which is a sensor signal, specifying time information indicating a start time and an end time of the utterance section, a target sound and the non-purpose The second input signal, which is an acoustic signal including the voice of the voice speaker, is not specified by at least one of the utterance contents and the speaker by a signal in a period corresponding to the specified time information. A program for causing a computer to execute processing for processing a signal of the period in an input signal.

DESCRIPTION OF SYMBOLS 10 Speech processing apparatus 11 Detection part 12 Processing part 100 Speech processing apparatus 101 Speech area detection part 102 Speech area processing part 103 Acoustic signal preservation | save part 110 Speech processing apparatus 200 Speech processing apparatus 201 1st speech signal generation part 202 2nd speech signal generation Unit 210 audio signal generation unit 300 audio processing device 301 signal selection unit 400 audio processing device 401 first audio signal generation unit 402 second audio signal generation unit 403 first speaker position estimation unit 404 second speaker position estimation unit 410 audio Signal generation unit 420 Position estimation unit 500 Speech processing device 501 Target speaker utterance section detection section 502 Speaking section processing section 503 Storage section 600 Speech recognition apparatus 601 Speech recognition section

Claims (10)

Detecting means for detecting an utterance section of an utterance by a non-target voice speaker from a first input signal that is a sensor signal, and identifying time information indicating a start time and an end time of the utterance section;
Of the second input signal that is an acoustic signal including the target sound and the voice of the non-target speaker, at least one of the utterance content and the speaker is not specified by the signal in the period corresponding to the specified time information And a processing means for processing the signal of the period in the second input signal. The first input signal is an audio signal mainly including the non-purpose audio,
The audio processing apparatus according to claim 1, wherein the second input signal is an audio signal mainly including the target sound. Each of the first input signal and the second input signal includes one or a plurality of acoustic signals output from at least one of a plurality of microphones,
The sound processing device generates a first acoustic signal that is a sound signal mainly including the non-target sound from the first input signal, and a sound signal mainly including the target sound from the second input signal. A generator for generating a second acoustic signal;
The detection means identifies the time information from the first acoustic signal,
The audio processing apparatus according to claim 1, wherein the processing unit processes a signal of the period in the second acoustic signal. Based on the third input signal that is a sensor signal and the first input signal, one or more acoustic signals are selected as the first selection signal from the first input signal, and the third input signal and the second input signal are selected. And further comprising a selection means for selecting one or more acoustic signals as the second selection signal from the second input signal,
The sound processing apparatus according to claim 3, wherein the generation unit generates the first acoustic signal from the first selection signal and generates the second acoustic signal from the second selection signal. An estimation means for estimating a position of a sound source of the target sound and a position of a speaker of the non-target sound from a third input signal which is a sensor signal;
The generating means generates the first acoustic signal from the first input signal based on the estimated position of the speaker of the non-target speech, and based on the position of the estimated sound source of the target sound, The audio processing apparatus according to claim 3, wherein the second acoustic signal is generated from the second input signal. The first input signal is a video signal representing a video of a speaker of the non-purpose voice;
The detecting means determines a speaker of the non-target voice from the video represented by the video signal, and uses the determined video of the speaker to detect an utterance section of the utterance by the speaker; The speech processing apparatus according to claim 1, wherein   The processing means replaces a signal of a period corresponding to the specified time information in the second input signal with a zero signal or a predetermined noise signal. The speech processing apparatus according to the item.   The said process means performs a voice quality conversion process to the signal of the period corresponding to the said specified time information among the said 2nd input signals, The any one of Claim 1 to 6 characterized by the above-mentioned. Audio processing device. Detecting an utterance section of an utterance by a non-target speaker from a first input signal that is a sensor signal, and identifying time information indicating a start time and an end time of the utterance section;
Of the second input signal that is an acoustic signal including the target sound and the voice of the non-target speaker, at least one of the utterance content and the speaker is not specified by the signal in the period corresponding to the specified time information As described above, the speech processing method characterized by processing the signal of the period in the second input signal. A process of detecting an utterance section of an utterance by a speaker of a non-target voice from a first input signal which is a sensor signal, and specifying time information indicating a start time and an end time of the utterance section;
Of the second input signal that is an acoustic signal including the target sound and the voice of the non-target speaker, at least one of the utterance content and the speaker is not specified by the signal in the period corresponding to the specified time information As described above, a program for causing a computer to execute processing for processing a signal of the period in the second input signal.

Images