JP2017208682A - Target sound collection device, target sound collection method, program, and recording media - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a target sound collection device that can estimate a direction of unnecessary sound to execute directional characteristics sound collection with a direction other than that of the unnecessary sound as a sound collection direction.SOLUTION: A target sound collection device includes: a direction estimation unit for estimating a direction of a sound source on the basis of a plurality of acoustic signals collected from a plurality of microphones; a sound generation frequency measurement unit for measuring frequency estimated as a sound source direction regarding each direction; an unnecessary sound direction estimation unit for estimating some direction as an unnecessary sound direction when comparison results between frequency in some direction in each direction and a predetermined threshold value satisfies a predetermined condition; a sound collection direction determination unit for determining a direction of a sound source that is a direction different from the unnecessary sound direction as a sound collection direction when a direction different from the unnecessary sound direction is estimated as the sound collection direction; and a directional characteristics sound collection unit for emphasizing sound in a determined sound collection direction to collect it.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a target sound collection device, a target sound collection method, a program, and a recording medium that are used in sound collection technology using a plurality of microphones.

  FIG. 1 and FIG. 2 schematically show the configuration and operation of the target sound collecting device 9 of Patent Document 1. As shown in FIG. 1, the target sound collection device 9 of Patent Document 1 includes a direction estimation unit 91, a sound collection direction control unit 92, a directivity sound collection unit 93, and a storage unit 94, and the sound collection direction control unit 92. Includes a desired direction setting unit 922 and a sound collection direction determination unit 923. The direction estimating unit 91 estimates the direction of the sound source based on a plurality of acoustic signals collected from the plurality of microphones 8-1,..., 8-N (N is an integer of 2 or more) (S91). The direction estimation unit 91 estimates the direction of the sound source based on the time difference or the amplitude difference generated between the microphones 8-1,. The desired direction setting unit 922 sets in advance a direction in which sound collection is desired (desired direction) or an angle range in which sound collection is desired (desired angle range) (S922). The sound collection direction determination unit 923 determines the sound source direction as the sound collection direction when the direction of the sound source estimated in step S91 matches a preset desired direction (or a desired angle range) (S923). . The directivity sound collection unit 93 emphasizes the sound in the sound collection direction determined in step S923 and executes directivity sound collection (S93). Step S93 can be realized by the method disclosed in Patent Document 2 and the like. The acoustic signal collected in step S93 may be stored in the storage unit 94 as shown in the figure, or may be output to the outside of the apparatus.

JP 2005-64968 A JP 2009-44588 A

  The method of Patent Document 1 is effective if the direction and angle range in which sound collection is desired are determined in advance. On the other hand, when the direction and angle range in which the sound is desired to be collected are not determined in advance in the method of Patent Document 1, all directions are set as the sound collection direction. In this case, if there is a person or person who emits unnecessary sounds around, there is a problem that this sound is also collected as a necessary sound. For example, when the method of Patent Document 1 is applied to a robot that performs dialogue using voice recognition or a remote controller that operates a device using voice recognition, voice recognition is performed in response to unwanted sounds, The remote control may malfunction.

  Therefore, an object of the present invention is to provide a target sound collecting device that can estimate a direction of unnecessary sound and execute directional sound collection using a direction other than the unnecessary sound direction as a sound collecting direction.

  The target sound collection device of the present invention includes a direction estimation unit, a pronunciation frequency measurement unit, an unnecessary sound direction estimation unit, a sound collection direction determination unit, and a directional sound collection unit.

  The direction estimation unit estimates the direction of the sound source based on a plurality of acoustic signals collected from a plurality of microphones. The pronunciation frequency measurement unit measures the frequency estimated as the direction of the sound source in each direction. The unnecessary sound direction estimation unit estimates any direction as an unnecessary sound direction when a comparison result between the frequency in any one of the directions and a predetermined threshold satisfies a predetermined condition. When the direction different from the unnecessary sound direction is estimated as the direction of the sound source, the sound collection direction determination unit determines the direction of the sound source that is different from the unnecessary sound direction as the sound collection direction. The directional sound collection unit collects sound by emphasizing the sound in the determined sound collection direction.

  According to the target sound collecting device of the present invention, it is possible to estimate a direction of unnecessary sound and execute directional sound collection using a direction other than the unnecessary sound direction as a sound collecting direction.

The block diagram which shows the structure of the target sound collector of a prior art. The flowchart which shows operation | movement of the target sound collector of a prior art. FIG. 3 is a block diagram illustrating a configuration of a target sound collection device according to the first embodiment. 3 is a flowchart showing the operation of the target sound collection device of the first embodiment. FIG. 5 is a block diagram illustrating a configuration of a target sound collection device according to a second embodiment. 9 is a flowchart showing a synthetic voice generation and reproduction operation of the target sound collection device according to the second embodiment. 9 is a flowchart illustrating a sound collection direction control operation of the target sound collection device according to the second embodiment. The block diagram which shows the structure of the target sound collector of the modification 1. FIG. The flowchart which shows the speech detection operation | movement of the target sound collector of the modification 1. 10 is a flowchart showing a sound collection direction control operation of the target sound collection device of Modification 1; FIG. 9 is a block diagram illustrating a configuration of a target sound collecting device according to a third embodiment. 10 is a flowchart showing the operation of the target sound collecting device of the third embodiment. The block diagram which shows the structure of the target sound collector of the modification 2. FIG. 10 is a flowchart showing the operation of a target sound collecting device according to a second modification.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  Hereinafter, the configuration and operation of the target sound collecting apparatus according to the first embodiment will be described with reference to FIGS. 3 and 4. As shown in FIG. 3, the target sound collection device 1 of the present embodiment includes a direction estimation unit 91, a sound collection direction control unit 12, a directional sound collection unit 93, and a storage unit 94, and the sound collection direction The control unit 12 includes a sound generation frequency measurement unit 121, an unnecessary sound direction estimation unit 122, and a sound collection direction determination unit 123. Since the direction estimation unit 91, the directivity sound collection unit 93, and the storage unit 94 have the same functions as the constituent elements of the same name and the same number as the target sound collection device 9 of Patent Document 1, description thereof is omitted.

  The pronunciation frequency measurement unit 121 measures the frequency estimated as the direction of the sound source for each direction (S121). That is, the sound generation frequency measuring unit 121 measures how often the sound is generated from which direction within a certain time. Whether or not there is a pronunciation can be known from the output of the direction estimation unit 91. For example, if the total time when the direction estimated by the direction estimation unit 91 is θ during the past T seconds is A (θ) seconds, the sound generation frequency measurement unit 121 calculates the sound generation frequency in the θ direction as a ratio thereof. It can be obtained as D (θ) = A (θ) / T. The pronunciation frequency measuring unit 121 calculates all the frequencies in each direction. For example, if it is assumed that the noise source is a TV or a speaker for listening to music, they will not be silent for a long time, and sound will continue to arrive at the microphones 8-1,. It will be. When such a noise source is in the θ direction, the sound generation frequency D (θ) takes a large value close to 1.

The unnecessary sound direction estimation unit 122 estimates any direction as an unnecessary sound direction when a comparison result between the frequency in any one of the directions and a predetermined threshold satisfies a predetermined condition (S122). ). For example, when the aforementioned sound generation frequency D (θ) exceeds a preset threshold value E (0 ≦ E ≦ 1), the unnecessary sound direction estimation unit 122 sets that direction as the unnecessary sound direction. The unnecessary sound direction estimation unit 122 performs the same estimation for all directions, and estimates one direction or a plurality of directions as unnecessary sound directions. Unwanted sound direction estimating unit 122, the all directions within a predetermined angular range determined based on the set direction theta _N as unnecessary sound direction (e.g. all directions within the range of θ _{N +} Δθ~θ _N -Δθ) An unnecessary sound direction may be used. Δθ is a preset setting range of the unnecessary sound direction, and is set based on the accuracy of direction estimation. For example, when the direction estimation accuracy is 10 degrees, by setting a value of 10 degrees or more, unnecessary sounds are not determined as necessary sounds due to direction estimation errors.

  When the direction different from the unnecessary sound direction is estimated as the direction of the sound source, the sound collection direction determination unit 123 determines the direction of the sound source that is different from the unnecessary sound direction as the sound collection direction (S123).

  According to the target sound collecting device 1 of the present embodiment, a sound source that continuously generates sound is estimated as an unnecessary sound, and other sound sources are handled as target sounds. Even if there is a noise source, the target sound can be appropriately identified and the target sound can be emphasized and collected.

  Hereinafter, the configuration and operation of the target sound collecting apparatus according to the second embodiment will be described with reference to FIGS. 5, 6, and 7. As shown in FIG. 5, the target sound collection device 2 of this embodiment includes a direction estimation unit 91, a sound collection direction control unit 22, a directivity sound collection unit 93, a storage unit 94, and a speech control unit 24. The sound collection direction control unit 22 includes a sound generation frequency measurement unit 221, an unnecessary sound direction estimation unit 122, and a sound collection direction determination unit 123. The constituent elements other than the pronunciation frequency measuring unit 221, the speech control unit 24, and the speech synthesizing unit 25 have the same functions as those of the target sound collecting device 1 of the first embodiment and have the same functions, and thus the description thereof is omitted. To do. Assuming that the target sound collecting device 2 of the present embodiment is a device that interacts with the user, the target sound collecting device 2 has an utterance control unit 24 and a voice synthesis unit 25, thereby having an utterance control and a voice synthesis function. Yes. The utterance control unit 24 controls the utterance (S24). The speech synthesizer 25 generates and reproduces synthesized speech (S25). The utterance frequency measuring unit 221 according to the present embodiment measures the frequency only during the time when the synthesized speech is being reproduced (S221).

  During the synthetic voice playback, it is assumed that the user who is the other party of the conversation only has a chance to talk, and the user is unlikely to speak frequently. Therefore, when the pronunciation frequency is high during the reproduction of the synthesized speech, the sound source is likely not a user but a noise source. Based on this assumption, the target sound collecting device 2 of the present embodiment has the same effect as that of the first embodiment even in a device that interacts with the user.

<Modification 1>
Hereinafter, with reference to FIG. 8, FIG. 9, and FIG. 10, a modification in which the utterance control unit 24 and the speech synthesis unit 25 are replaced with the utterance detection unit 26a, and the pronunciation frequency measurement unit 221 is replaced with the pronunciation frequency measurement unit 221a. The configuration and operation of the target sound collection device 2a will be described.

  The utterance detection unit 26a detects an utterance from the sound signal for speaker reproduction (S26a). The utterance detection unit 26a compares the level of the sound signal for speaker reproduction with a preset threshold value, and detects the utterance as uttered when the level of the sound signal for speaker reproduction exceeds the threshold value. The utterance frequency measuring unit 221a measures the frequency only during the time when the utterance is detected (S221a).

  Hereinafter, the configuration and operation of the target sound collecting apparatus according to the third embodiment will be described with reference to FIGS. 11 and 12. As shown in FIG. 11, the target sound collection device 3 of this embodiment includes a direction estimation unit 91, a sound collection direction control unit 32, a directivity sound collection unit 93, a storage unit 94, and an utterance control unit 24. The sound collection direction control unit 32 includes a sound generation frequency measurement unit 221, an unnecessary sound direction estimation unit 122, a sound collection direction determination unit 323, and a sound generation timing measurement unit 324. Since the components other than the sound collection direction determination unit 323 and the sound generation timing measurement unit 324 have the same functions as the component having the same name and the same number as the target sound collection device 2 of the second embodiment, description thereof will be omitted.

  The pronunciation timing measurement unit 324 measures the time from when the synthesized speech is paused to when the direction of the sound source is first estimated when the synthesized speech is paused (S324).

  The sound collection direction determination unit 323 determines the sound source that is estimated first when the direction of the sound source estimated first is different from the unnecessary sound direction and the time measured in step S324 satisfies a predetermined condition. Is determined as the sound collection direction (S323). Specifically, the sound collection direction determination unit 323 performs the sound generation timing measurement unit 324 in addition to the condition that the direction of the sound source estimated by the direction estimation unit 91 is not estimated as the unnecessary sound direction by the unnecessary sound direction estimation unit 122. If the measured time is between the preset minimum and maximum values, the sound collection direction is determined for the estimated direction (S323). However, the minimum and maximum values may be negative or positive.

  If the user is interacting with the apparatus, it is assumed that it is natural to utter immediately after the reproduction of the conversation voice (synthetic voice) ends. Based on this assumption, the target sound collection device 3 of the present embodiment generates a sound within a short time immediately after the reproduction of the dialogue voice (synthetic voice) is paused. Whether or not the sound source is a user can be determined by paying attention to whether or not the sound source is started.

<Modification 2>
Hereinafter, referring to FIG. 13 and FIG. 14, the utterance control unit 24 and the voice synthesis unit 25 are replaced with the utterance detection unit 26 a, the pronunciation frequency measurement unit 221 is replaced with the pronunciation frequency measurement unit 221 a, and the pronunciation timing measurement unit The configuration and operation of the target sound collecting device 3a according to the modified example 2 in which the sound generation timing measuring unit 324a is replaced with 324 will be described. When the utterance is not detected, the pronunciation timing measuring unit 324a measures the time from when the utterance is not detected to when the direction of the sound source is first estimated (S324a). In another expression, the pronunciation timing measurement unit 324a measures the time from when the output of the utterance detection unit 26a shifts from the presence of utterance to the absence of utterance until the direction of the sound source is first estimated (S324a).

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (8)

A direction estimation unit that estimates the direction of a sound source based on a plurality of acoustic signals collected from a plurality of microphones;
A pronunciation frequency measuring unit that measures the frequency estimated as the direction of the sound source for each direction;
An unnecessary sound direction estimating unit that estimates any one of the directions as an unnecessary sound direction when a comparison result between the frequency in any one of the directions and a predetermined threshold satisfies a predetermined condition;
When the direction different from the unnecessary sound direction is estimated as the direction of the sound source, a sound collection direction determining unit that determines the direction of the sound source that is different from the unnecessary sound direction as the sound collection direction;
A directional sound collection unit that emphasizes and collects the sound in the determined sound collection direction;
A target sound collecting device. The target sound collecting device according to claim 1,
A speech synthesizer that generates and reproduces synthesized speech;
The speech frequency measurement unit
A target sound collecting device that measures the frequency only during the time that the synthesized speech is being reproduced. The target sound collecting device according to claim 1,
An utterance detection unit for detecting an utterance from a predetermined sound signal;
The speech frequency measurement unit
A target sound collecting device that measures the frequency only during the time when the utterance is detected. The target sound collecting device according to claim 2,
When the playback of the synthesized speech is paused, further includes a sounding timing measuring unit that measures a time from immediately after the playback of the synthesized speech pauses until the estimation of the direction of the sound source is first performed,
The sound collection direction determination unit
When the first estimated sound source direction is different from the unnecessary sound direction and the measured time satisfies a predetermined condition, the first estimated sound source direction is determined as the sound collection direction. Sound collecting device. The target sound collecting device according to claim 3,
When the utterance is not detected, it further includes a pronunciation timing measurement unit that measures the time from immediately after the utterance is no longer detected until the first direction of the sound source is estimated,
The sound collection direction determination unit
When the first estimated sound source direction is different from the unnecessary sound direction and the measured time satisfies a predetermined condition, the first estimated sound source direction is determined as the sound collection direction. Sound collecting device. A target sound collection method executed by the target sound collection device,
Estimating a direction of a sound source based on a plurality of acoustic signals collected from a plurality of microphones;
Measuring the frequency estimated as the direction of the sound source for each direction;
A step of estimating any one of the directions as an unnecessary sound direction when a comparison result between the frequency in any one of the directions and a predetermined threshold satisfies a predetermined condition;
Determining a direction of the sound source that is different from the unnecessary sound direction as a sound collection direction when a direction different from the unnecessary sound direction is estimated as the direction of the sound source;
Enhancing the sound in the determined sound collection direction and collecting the sound;
Purpose sound collection method including.   A program for causing a computer to function as the target sound collecting device according to any one of claims 1 to 5.   A computer-readable recording medium on which the program according to claim 7 is recorded.

Images